Skip to content

Commit a6cdb69

Browse files
psiddhCopilot
andauthored
Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
1 parent 26a00cb commit a6cdb69

1 file changed

Lines changed: 3 additions & 3 deletions

File tree

examples/qualcomm/oss_scripts/llama/wrappers/llm_wrappers.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -181,9 +181,9 @@ def _prepare_model(self): # noqa: C901
181181
k.replace("_orig_mod.", ""): v for k, v in state_dict.items()
182182
}
183183

184-
# Gemma RMSNorm computes (1 + w) * x but ExecuTorch's RMSNorm computes
185-
# w * x, so add the +1 offset to norm weights regardless of load path.
186-
# See https://github.com/huggingface/transformers/pull/29402
184+
# Gemma checkpoints store RMSNorm weights as deviations from 1 and use (1 + w) * x.
185+
# This model/export path uses standard RMSNorm semantics (w * x), so add +1 to
186+
# norm weights regardless of load path. See https://github.com/huggingface/transformers/pull/29402
187187
if self.control_args.decoder_model in {
188188
"gemma-2b",
189189
"gemma2-2b",

0 commit comments

Comments
 (0)