Skip to content

[None][fix] Fix moe backend setting#15861

Open
shuyixiong wants to merge 1 commit into
NVIDIA:mainfrom
shuyixiong:user/shuyix/fix_moe_backend_setting
Open

[None][fix] Fix moe backend setting#15861
shuyixiong wants to merge 1 commit into
NVIDIA:mainfrom
shuyixiong:user/shuyix/fix_moe_backend_setting

Conversation

@shuyixiong

@shuyixiong shuyixiong commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • Fix
    resolve_moe_cls unconditionally downgraded an unquantized (BF16) MoE from TRTLLMGenFusedMoE to CutlassFusedMoE, so moe_config.backend="TRTLLM" was silently ignored for BF16 checkpoints. This PR removes that downgrade so the user-requested TRTLLM backend is actually used on its supported BF16 path.
  • Root cause
    The downgrade rule (if moe_cls == TRTLLMGenFusedMoE and not has_quant: moe_cls = CutlassFusedMoE) was originally added as a fallback: some BF16 cases lacked FlashInfer support, so it fell back to Cutlass for those. It was gated by _supports_flashinfer_bf16_routing_method(routing_method), which kept TRTLLMGenFusedMoE whenever the FlashInfer BF16 path was actually available. That guard was dropped in [TRTLLM-13052][feat] Enable TRTLLM moe backend for nemotron-h BF16 ckpt #14944, turning a conditional fallback into an unconditional downgrade that discards a valid TRTLLM selection.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

@shuyixiong shuyixiong requested a review from a team as a code owner July 2, 2026 02:32
@shuyixiong shuyixiong requested a review from QiJune July 2, 2026 02:32
Signed-off-by: shuyixiong <219646547+shuyixiong@users.noreply.github.com>
@shuyixiong shuyixiong force-pushed the user/shuyix/fix_moe_backend_setting branch from 59fff7c to a2b6e66 Compare July 2, 2026 02:33
@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Removed a fallback conditional in resolve_moe_cls that previously downgraded the selected MoE class from TRTLLMGenFusedMoE to CutlassFusedMoE when the effective quantization config had no applicable quantization. Backend selection now relies solely on get_moe_cls's output.

Changes

MoE Backend Selection

Layer / File(s) Summary
Remove quant-based downgrade fallback
tensorrt_llm/_torch/modules/fused_moe/create_moe.py
Removed the has_quant check and conditional reassignment that forced CutlassFusedMoE when the effective quant config had no applicable quantization, leaving get_moe_cls's selection as final.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is still the template with no actual title, issue summary, or test coverage details. Add a proper PR title plus short Description and Test Coverage sections that explain the problem, solution, and relevant tests.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title matches the PR's MOE backend selection fix and is specific enough to identify the main change.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@shuyixiong

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57096 [ run ] triggered by Bot. Commit: a2b6e66 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57096 [ run ] completed with state SUCCESS. Commit: a2b6e66
/LLM/main/L0_MergeRequest_PR pipeline #45885 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@shuyixiong

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

1 similar comment
@shuyixiong

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57202 [ run ] triggered by Bot. Commit: a2b6e66 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57202 [ run ] completed with state SUCCESS. Commit: a2b6e66
/LLM/main/L0_MergeRequest_PR pipeline #45974 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@shuyixiong

Copy link
Copy Markdown
Collaborator Author

/bot run --disable-fail-fast

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57383 [ run ] triggered by Bot. Commit: a2b6e66 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #57383 [ run ] completed with state SUCCESS. Commit: a2b6e66
/LLM/main/L0_MergeRequest_PR pipeline #46132 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants