Skip to content

[Qwen3MoE] Fix wrong return type annotation in Qwen3MoeSparseMoeBlock.forward#45211

Closed
matdou wants to merge 2 commits intohuggingface:mainfrom
matdou:fix/qwen3-moe-sparse-moe-block-return-type
Closed

[Qwen3MoE] Fix wrong return type annotation in Qwen3MoeSparseMoeBlock.forward#45211
matdou wants to merge 2 commits intohuggingface:mainfrom
matdou:fix/qwen3-moe-sparse-moe-block-return-type

Conversation

@matdou
Copy link
Copy Markdown

@matdou matdou commented Apr 3, 2026

Fixes #45208

What does this PR do?

This PR corrects an incorrect return type in Qwen3MoeSparseMoeBlock.forward.

The method was annotated as returning tuple[torch.Tensor, torch.Tensor], while the implementation returns a torch.Tensor:

return final_hidden_states.reshape(batch_size, sequence_length, hidden_dim)

In particular, downstream usage (e.g. in Qwen3MoeDecoderLayer) expects a tensor:

hidden_states = self.mlp(hidden_states)
hidden_states = residual + hidden_states

This PR updates the annotation to:

-> torch.Tensor

in both:

  • modeling_qwen3_moe.py
  • modular_qwen3_moe.py

No functional changes are introduced, this is a typing correction only.


Checklist

  • I confirm that this is not a pure code agent PR.
  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

Who can review?

@ArthurZucker @Cyrilvallez

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 3, 2026

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_moe, qwen3_omni_moe, qwen3_vl_moe

@matdou matdou force-pushed the fix/qwen3-moe-sparse-moe-block-return-type branch from a0a6199 to 2c36a74 Compare April 3, 2026 08:16
@Rocketknight1
Copy link
Copy Markdown
Member

No drive-by code agent PRs on other people's issues please!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Qwen3MoE] Potentially a bug on Qwen3MoeSparseMoeBlock

2 participants