[TRTLLM-12500][feat] Add support for Qwen3.5 VL MoE (with the MTP fixes) by moraxu · Pull Request #14599 · NVIDIA/TensorRT-LLM

moraxu · 2026-05-26T23:19:17Z

Summary by CodeRabbit

Release Notes

New Features
- Added support for Qwen3.5 Mixture-of-Experts multimodal vision-language model (Qwen3.5-35B-A3B).
- Extended speculative decoding to support vision-language model workflows.
- Added FP8 block-scales quantization support for the new model.
Documentation
- Updated feature support matrix to include Qwen3.5-MoE multimodal model capabilities.
Tests
- Added accuracy validation tests for Qwen3.5-MoE multimodal model on standard benchmarks.
- Added configuration and forward-parity verification tests.

Description

Completes Qwen3.5-MoE-VL (Qwen3_5MoeForConditionalGeneration) on top of #12611.
Switches the VLM config path to HF's native transformers.Qwen3_5MoeConfig (present in 5.3.0), adds a thin post-load normalizer that materializes the handful of aliases the reused Qwen3Next runtime expects on text_config (intermediate_size from the MoE fields, rope_theta/partial_rotary_factor/rope_scaling from rope_parameters), and centralizes hybrid-cache dtype resolution in two helpers.
284291a: Makes Qwen3VLModelBase MTP/eagle-compatible: threads spec_metadata / resource_manager / pre-fusion orig_input_ids to the inner LM. Unblocks TestQwen3_5_35B_A3B::test_bf16_mtp[mtp_on], which started failing after [TRTLLM-11547][feat] Add Qwen3.5 MTP support. #12646 added the MTP test on top of the original [TRTLLM-12500][feat] Add support for Qwen3.5 VL MoE - REVERTED by #14599 #14164 and the same Qwen3.5-35B-A3B checkpoint started routing through the VLM wrapper - see nvbugs/6206179 for details
Commits specific to this PR follow after the first two commits

Test Coverage

Accuracy & unit tests

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

moraxu · 2026-05-26T23:19:51Z

/bot run

tensorrt-cicd · 2026-05-26T23:25:36Z

PR_Github #50402 [ run ] triggered by Bot. Commit: 284291a Link to invocation

tensorrt-cicd · 2026-05-27T02:14:47Z

PR_Github #50402 [ run ] completed with state SUCCESS. Commit: 284291a
/LLM/main/L0_MergeRequest_PR pipeline #39926 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-05-27T04:44:38Z

/bot run

tensorrt-cicd · 2026-05-27T04:50:39Z

PR_Github #50459 [ run ] triggered by Bot. Commit: 284291a Link to invocation

tensorrt-cicd · 2026-05-27T08:52:24Z

PR_Github #50459 [ run ] completed with state SUCCESS. Commit: 284291a
/LLM/main/L0_MergeRequest_PR pipeline #39975 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-05-27T17:52:22Z

/bot run

tensorrt-cicd · 2026-05-27T17:58:37Z

PR_Github #50590 [ run ] triggered by Bot. Commit: 17c7a00 Link to invocation

tensorrt-cicd · 2026-05-27T18:36:31Z

PR_Github #50590 [ run ] completed with state FAILURE. Commit: 17c7a00
/LLM/main/L0_MergeRequest_PR pipeline #40088 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-05-27T18:42:55Z

/bot run

tensorrt-cicd · 2026-05-27T18:49:22Z

PR_Github #50604 [ run ] triggered by Bot. Commit: 17c7a00 Link to invocation

tensorrt-cicd · 2026-05-27T19:26:07Z

PR_Github #50604 [ run ] completed with state FAILURE. Commit: 17c7a00
/LLM/main/L0_MergeRequest_PR pipeline #40099 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tburt-nv · 2026-05-27T20:11:11Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-05-27T20:18:02Z

PR_Github #50613 [ run ] triggered by Bot. Commit: 17c7a00 Link to invocation

tensorrt-cicd · 2026-05-28T04:11:16Z

PR_Github #50613 [ run ] completed with state SUCCESS. Commit: 17c7a00
/LLM/main/L0_MergeRequest_PR pipeline #40108 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-05-28T06:53:40Z

/bot run

tensorrt-cicd · 2026-05-28T07:00:59Z

PR_Github #50720 [ run ] triggered by Bot. Commit: 17c7a00 Link to invocation

tensorrt-cicd · 2026-05-28T07:40:46Z

PR_Github #50720 [ run ] completed with state SUCCESS. Commit: 17c7a00
/LLM/main/L0_MergeRequest_PR pipeline #40202 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-05-28T07:44:59Z

/bot run

tensorrt-cicd · 2026-05-28T07:50:35Z

PR_Github #50739 [ run ] triggered by Bot. Commit: 17c7a00 Link to invocation

tensorrt-cicd · 2026-05-28T08:07:38Z

PR_Github #50739 [ run ] completed with state FAILURE. Commit: 17c7a00
/LLM/main/L0_MergeRequest_PR pipeline #40219 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-05-28T16:41:15Z

/bot run

tensorrt-cicd · 2026-05-28T16:46:32Z

PR_Github #50834 [ run ] triggered by Bot. Commit: 17c7a00 Link to invocation

tensorrt-cicd · 2026-05-28T19:13:47Z

PR_Github #50834 [ run ] completed with state FAILURE. Commit: 17c7a00
/LLM/main/L0_MergeRequest_PR pipeline #40303 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-05-28T19:19:48Z

/bot run

tensorrt-cicd · 2026-05-28T19:27:20Z

PR_Github #50866 [ run ] triggered by Bot. Commit: 17c7a00 Link to invocation

tensorrt-cicd · 2026-05-28T19:46:09Z

PR_Github #50866 [ run ] completed with state FAILURE. Commit: 17c7a00
/LLM/main/L0_MergeRequest_PR pipeline #40332 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-05-28T19:49:20Z

/bot run

moraxu · 2026-06-12T21:56:22Z

/bot run

tensorrt-cicd · 2026-06-12T22:02:32Z

PR_Github #53962 [ run ] triggered by Bot. Commit: e3e87d2 Link to invocation

tensorrt-cicd · 2026-06-13T00:49:37Z

PR_Github #53962 [ run ] completed with state FAILURE. Commit: e3e87d2
/LLM/main/L0_MergeRequest_PR pipeline #43053 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

2ez4bz · 2026-06-22T23:47:40Z

/bot run

venkywonka

lgtm from doc-owners

tensorrt-cicd · 2026-06-22T23:53:05Z

PR_Github #55103 [ run ] triggered by Bot. Commit: 260144a Link to invocation

tensorrt-cicd · 2026-06-23T01:14:48Z

PR_Github #55103 [ run ] completed with state FAILURE. Commit: 260144a
/LLM/main/L0_MergeRequest_PR pipeline #44089 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

moraxu · 2026-06-23T05:37:46Z

/bot run

tensorrt-cicd · 2026-06-23T05:43:30Z

PR_Github #55170 [ run ] triggered by Bot. Commit: 260144a Link to invocation

tensorrt-cicd · 2026-06-23T06:38:47Z

PR_Github #55170 [ run ] completed with state SUCCESS. Commit: 260144a
/LLM/main/L0_MergeRequest_PR pipeline #44141 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

moraxu · 2026-06-24T16:15:41Z

/bot run

tensorrt-cicd · 2026-06-24T16:23:23Z

PR_Github #55537 [ run ] triggered by Bot. Commit: 260144a Link to invocation

tensorrt-cicd · 2026-06-24T18:21:38Z

PR_Github #55537 [ run ] completed with state SUCCESS. Commit: 260144a
/LLM/main/L0_MergeRequest_PR pipeline #44465 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

tburt-nv · 2026-06-24T20:56:57Z

please rebase or merge origin/main to resolve the GB200-4_GPUs-PyTorch-PerfSanity-* failures with 91dc145

Signed-off-by: Michal Guzek <mguzek@nvidia.com>

moraxu · 2026-06-25T00:00:54Z

/bot run

tensorrt-cicd · 2026-06-25T00:07:28Z

PR_Github #55617 [ run ] triggered by Bot. Commit: 4035427 Link to invocation

tensorrt-cicd · 2026-06-25T02:09:02Z

PR_Github #55617 [ run ] completed with state FAILURE. Commit: 4035427
/LLM/main/L0_MergeRequest_PR pipeline #44534 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-06-25T05:02:19Z

/bot run

tensorrt-cicd · 2026-06-25T05:08:41Z

PR_Github #55701 [ run ] triggered by Bot. Commit: 4035427 Link to invocation

tensorrt-cicd · 2026-06-25T06:14:58Z

PR_Github #55701 [ run ] completed with state FAILURE. Commit: 4035427
/LLM/main/L0_MergeRequest_PR pipeline #44604 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

moraxu · 2026-06-25T07:31:03Z

/bot run

tensorrt-cicd · 2026-06-25T07:42:55Z

PR_Github #55742 [ run ] triggered by Bot. Commit: 4035427 Link to invocation

tensorrt-cicd · 2026-06-25T08:33:36Z

PR_Github #55742 [ run ] completed with state SUCCESS. Commit: 4035427
/LLM/main/L0_MergeRequest_PR pipeline #44644 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

github-actions Bot assigned moraxu May 26, 2026

moraxu mentioned this pull request May 26, 2026

[TRTLLM-12500][feat] Add support for Qwen3.5 VL MoE - REVERTED by #14599 #14164

Merged

1 task

venkywonka approved these changes Jun 22, 2026

View reviewed changes

moraxu added 6 commits June 24, 2026 14:27

Original Qwen 3.5 VL MoE PR commit

0671e31

Signed-off-by: Michal Guzek <mguzek@nvidia.com>

Fix Qwen3 MTP test

4b67ff2

Signed-off-by: Michal Guzek <mguzek@nvidia.com>

Exclude some tests from l40s

b2a389d

Signed-off-by: Michal Guzek <mguzek@nvidia.com>

Update test decorators

f897f73

Signed-off-by: Michal Guzek <mguzek@nvidia.com>

Update _is_disagg to _is_mm_disagg

fa3878b

Signed-off-by: Michal Guzek <mguzek@nvidia.com>

Fixes to mrope_config. fields

4035427

Signed-off-by: Michal Guzek <mguzek@nvidia.com>

moraxu force-pushed the dev-mguzek-qwen3_5-moe-vl-v2 branch from 260144a to 4035427 Compare June 25, 2026 00:00

Uh oh!

Conversation

moraxu commented May 26, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Release Notes

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

moraxu commented May 26, 2026

Uh oh!

tensorrt-cicd commented May 26, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

moraxu commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

moraxu commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

moraxu commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tburt-nv commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 27, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

moraxu commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

moraxu commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

moraxu commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

moraxu commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

tensorrt-cicd commented May 28, 2026

Uh oh!

moraxu commented May 28, 2026

Uh oh!

moraxu commented Jun 12, 2026

Uh oh!

tensorrt-cicd commented Jun 12, 2026

Uh oh!

tensorrt-cicd commented Jun 13, 2026

Uh oh!

2ez4bz commented Jun 22, 2026

Uh oh!

venkywonka left a comment

Choose a reason for hiding this comment

Uh oh!

tensorrt-cicd commented Jun 22, 2026

Uh oh!

moraxu commented May 26, 2026 •

edited by coderabbitai Bot

Loading