Skip to content

[fix] Normalization compute step for non-nvidia hardware#84

Merged
mikepapadim merged 2 commits intomainfrom
hotfix/addnorm-non-nvidia
Dec 11, 2025
Merged

[fix] Normalization compute step for non-nvidia hardware#84
mikepapadim merged 2 commits intomainfrom
hotfix/addnorm-non-nvidia

Conversation

@mikepapadim
Copy link
Copy Markdown
Member

No description provided.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a conditional finalization step for RMS normalization operations to support non-NVIDIA hardware. The fix ensures proper normalization computation across different GPU vendors by adding a hardware-specific final reduction step when needed.

  • Adds conditional reductionFinalNormalization task after initial RMS reduction
  • Uses shouldUseFinalNormalization() to determine when the additional step is required
  • Applies the fix consistently across multiple model architectures and quantization formats

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
Qwen3Q8_0FFNLayers.java Adds conditional finalization task for attention RMS normalization in Q8_0 quantized Qwen3 model
Phi3Q8_0FFNLayers.java Adds conditional finalization task for attention RMS normalization in Q8_0 quantized Phi3 model
Qwen3FP16FFNLayers.java Adds conditional finalization task for attention RMS normalization in FP16 Qwen3 model
Qwen2FP16FFNLayers.java Adds conditional finalization task for attention RMS normalization in FP16 Qwen2 model
Phi3FP16FFNLayers.java Adds conditional finalization task for attention RMS normalization in FP16 Phi3 model

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…/Phi3Q8_0FFNLayers.java

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@mikepapadim mikepapadim merged commit 8422058 into main Dec 11, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants