Skip to content

Use official TRT-LLM image (1.3.0rc15.post1) for DSv4 B300 TRT (non-MTP + MTP) #1636

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Oseltamivir wants to merge 14 commits into
base: main
Choose a base branch
from
+26 −2
Open

Use official TRT-LLM image (1.3.0rc15.post1) for DSv4 B300 TRT (non-MTP + MTP) #1636

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
4bcefb7
Update DSv4 TRT image for B200/B300 (non-MTP) to feat-deepseek_v4-2dd…
Oseltamivir Jun 1, 2026
6b7558c
Backfill PR number in changelog pr-link
Oseltamivir Jun 1, 2026
bd3c94c
Merge branch 'main' into update-dsv4-trt-image-2dd03e6
Oseltamivir Jun 1, 2026
f441f9f
Try official TRT-LLM release image 1.3.0rc15.post1 for DSv4 B200/B300…
Oseltamivir Jun 1, 2026
4bc5592
Revert to custom feat/deepseek_v4-2dd03e6 image for DSv4 B200/B300 (n…
Oseltamivir Jun 2, 2026
1b0afeb
Merge branch 'main' into update-dsv4-trt-image-2dd03e6
Oseltamivir Jun 2, 2026
14a1bb3
Point DSv4 B200/B300 TRT (non-MTP) at the SWA-scratch-fix image
Oseltamivir Jun 2, 2026
242ab88
Merge remote-tracking branch 'origin/main' into update-dsv4-trt-image…
Oseltamivir Jun 2, 2026
e23a541
Revert DSv4 B200/B300 TRT (non-MTP) to 2dd03e6 + disable SWA scratch …
Oseltamivir Jun 2, 2026
6118a76
Merge remote-tracking branch 'origin/main' into update-dsv4-trt-image…
Oseltamivir Jun 3, 2026
5adfeb3
Scope DSv4 TRT non-MTP change to B300 only
Oseltamivir Jun 3, 2026
ad529fb
Use official TRT-LLM image (1.3.0rc15.post1) for DSv4 B300 TRT (non-M…
Oseltamivir Jun 3, 2026
c2381b7
Re-add TRTLLM_DSV4_ENABLE_SWA_SCRATCH_REUSE=0 on rc15.post1 to test D…
Oseltamivir Jun 5, 2026
b09619e
Merge branch 'main' into update-dsv4-trt-image-2dd03e6
Oseltamivir Jun 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1801,7 +1801,7 @@ dsv4-fp4-b200-vllm-agentic:
- { tp: 8, ep: 8, dp-attn: true, offloading: cpu, conc-list: [64, 128, 256] }

dsv4-fp4-b200-trt:
image: ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-9aa3715
image: ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-2dd03e6
Copy link
Copy Markdown
Collaborator

@functionstackx functionstackx Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there any official nvidia RC that works...

Copy link
Copy Markdown
Collaborator Author

@Oseltamivir Oseltamivir Jun 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image is from dsv4 branch: https://github.com/NVIDIA/TensorRT-LLM/tree/feat/deepseek_v4

Main dsv4 failing DPA: https://github.com/SemiAnalysisAI/InferenceX/actions/runs/26786937394

Comment thread
cursor[bot] marked this conversation as resolved.
Outdated
model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: b200-dsv4
Expand Down Expand Up @@ -3053,7 +3053,7 @@ dsv4-fp4-b300-vllm-agentic:
- { tp: 8, ep: 8, dp-attn: true, offloading: cpu, conc-list: [128, 256, 512] }

dsv4-fp4-b300-trt:
image: ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-9aa3715
image: ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-2dd03e6
model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: b300
Expand Down
7 changes: 7 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3354,3 +3354,10 @@
description:
- "Add MTP speculative-decoding sibling for dsv4-fp4-mi355x-vllm (model: deepseek-ai/DeepSeek-V4-Pro) on vllm/vllm-openai-rocm:v0.22.0, per vllm-project/vllm#43385"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1630

- config-keys:
- dsv4-fp4-b200-trt
- dsv4-fp4-b300-trt
description:
- "Update the TensorRT-LLM DeepSeek-V4-Pro image to ghcr.io/semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-2dd03e6"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX

Check warning on line 3363 in perf-changelog.yaml

View check run for this annotation

Claude / Claude Code Review

perf-changelog entry has placeholder pr-link pull/XXXX 

The new `perf-changelog.yaml` entry leaves the `pr-link` as the unfilled template placeholder `https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX`. It should be `pull/1636` to match the actual PR number and the convention of every preceding entry, otherwise the pr-link is broken for both humans and any tooling that consumes the changelog.
Copy link
Copy Markdown
Contributor

@claude claude Bot Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The new perf-changelog.yaml entry leaves the pr-link as the unfilled template placeholder https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. It should be pull/1636 to match the actual PR number and the convention of every preceding entry, otherwise the pr-link is broken for both humans and any tooling that consumes the changelog.

Extended reasoning...

What the bug is. The diff appends a new entry to perf-changelog.yaml (lines 3358-3363) for the DSv4 TRT image bump on dsv4-fp4-b200-trt and dsv4-fp4-b300-trt. The last field of that entry is pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. XXXX is the literal template placeholder — it was never filled in with the actual PR number.

Why this is a real issue. Every other recent entry in the same file follows the convention of using the real PR number — the five entries immediately above this one link to pull/1602, pull/1624, pull/1616, pull/1626, and pull/1630 respectively. The PR metadata for this change shows it is PR #1636, so the value should be https://github.com/SemiAnalysisAI/InferenceX/pull/1636. With XXXX left in place, the link does not resolve to any PR, breaking the traceability that the pr-link field exists to provide.

Impact. This does not affect the actual image bump or any sweep behavior — the runtime is unchanged. The damage is to the changelog's documentation/audit value: anyone trying to find the originating PR for these two config-key changes from the changelog hits a dead 404, and any tooling that parses pr-link (e.g., to cross-link sweep results back to PRs, generate release notes, or validate entries) will either fail or produce a broken link.

Why existing checks didn't prevent it. There appears to be no schema validation that rejects XXXX as a PR number; the YAML is valid syntactically and the link is a syntactically valid URL — it just points nowhere meaningful. The placeholder is the kind of thing only a reviewer or a numeric-PR-id linter would catch.

Fix. Replace the placeholder with the real PR number:

  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1636

Step-by-step proof.

  1. Open perf-changelog.yaml at line 3363.
  2. Observe the literal line: pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX.
  3. Compare to the five entries immediately above (lines ending around 3332, 3338, 3344, 3350, 3356), which read pull/1602, pull/1624, pull/1616, pull/1626, pull/1630 — all real PR numbers.
  4. Check the PR metadata in this review: PR number is 1636.
  5. Click (or curl) https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX — it does not resolve to a PR. Click https://github.com/SemiAnalysisAI/InferenceX/pull/1636 — it resolves to this PR. The placeholder thus makes the field useless for its stated purpose.

Loading