Skip to content

Commit 5adfeb3

Browse files
Oseltamivirclaude
andcommitted
Scope DSv4 TRT non-MTP change to B300 only
B200 reverts to feat-deepseek_v4-9aa3715: the 2dd03e6 image OOMs on B200's smaller HBM at conc-256 once SWA scratch reuse is disabled. Only B300 moves to 2dd03e6 + TRTLLM_DSV4_ENABLE_SWA_SCRATCH_REUSE=0 in its launcher. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 6118a76 commit 5adfeb3

3 files changed

Lines changed: 2 additions & 10 deletions

File tree

.github/configs/nvidia-master.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1801,7 +1801,7 @@ dsv4-fp4-b200-vllm-agentic:
18011801
- { tp: 8, ep: 8, dp-attn: true, offloading: cpu, conc-list: [64, 128, 256] }
18021802

18031803
dsv4-fp4-b200-trt:
1804-
image: ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-2dd03e6
1804+
image: ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-9aa3715
18051805
model: deepseek-ai/DeepSeek-V4-Pro
18061806
model-prefix: dsv4
18071807
runner: b200-dsv4

benchmarks/single_node/fixed_seq_len/dsv4_fp4_b200_trt.sh

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -47,13 +47,6 @@ sanitize_slurm_mpi_env_for_trtllm
4747
export NCCL_NVLS_ENABLE="${NCCL_NVLS_ENABLE:-0}"
4848
echo "NCCL_NVLS_ENABLE: $NCCL_NVLS_ENABLE"
4949

50-
# Disable DSv4 SWA scratch reuse: with attention-DP the V2 scheduler grows ctx KV
51-
# (incl. SWA scratch) before delay batching defers a request, and the resulting
52-
# revert_allocate_context -> resize(shrink) can't release the scratch of a
53-
# never-forwarded request (no finish_event), crashing every dpa=true job.
54-
export TRTLLM_DSV4_ENABLE_SWA_SCRATCH_REUSE="${TRTLLM_DSV4_ENABLE_SWA_SCRATCH_REUSE:-0}"
55-
echo "TRTLLM_DSV4_ENABLE_SWA_SCRATCH_REUSE: $TRTLLM_DSV4_ENABLE_SWA_SCRATCH_REUSE"
56-
5750
if [[ "$MODEL" != /* ]]; then
5851
hf download "$MODEL"
5952
fi

perf-changelog.yaml

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3387,10 +3387,9 @@
33873387
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1630
33883388

33893389
- config-keys:
3390-
- dsv4-fp4-b200-trt
33913390
- dsv4-fp4-b300-trt
33923391
description:
3393-
- "Revert the non-MTP TensorRT-LLM DeepSeek-V4-Pro image to ghcr.io/semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-2dd03e6 and disable DSv4 SWA scratch reuse via TRTLLM_DSV4_ENABLE_SWA_SCRATCH_REUSE=0 in the launcher. Root cause of the prior attention-DP hangs/crashes: the V2 scheduler grows a context request's KV cache (incl. SWA scratch slots) before delay batching can defer it, so revert_allocate_context -> resize(shrink) must release scratch slots of a never-forwarded request, which has no finish_event -> crash on every dpa=true job. Disabling scratch reuse stops those slots from being allocated so the revert shrinks cleanly."
3392+
- "Update the non-MTP B300 TensorRT-LLM DeepSeek-V4-Pro image to ghcr.io/semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-2dd03e6 and disable DSv4 SWA scratch reuse via TRTLLM_DSV4_ENABLE_SWA_SCRATCH_REUSE=0 in the B300 launcher. Without this, attention-DP (dpa=true) jobs hang/crash: the V2 scheduler grows a context request's KV cache (incl. SWA scratch slots) before delay batching can defer it, so revert_allocate_context -> resize(shrink) must release scratch slots of a never-forwarded request, which has no finish_event -> crash on every dpa=true job. Disabling scratch reuse stops those slots from being allocated so the revert shrinks cleanly. B200 stays on feat-deepseek_v4-9aa3715 for now (the 2dd03e6 image OOMs on B200's smaller HBM at conc-256 with scratch reuse disabled)."
33943393
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1636
33953394

33963395
- config-keys:

0 commit comments

Comments
 (0)