Skip to content

Commit ff5c115

Browse files
xinli-swAnkur-singh
xinli-sw
and
Ankur-singh
authored
[NV] minimax-m3-gb200-dynamo: use vLLM nightly image / [NV] minimax-m3-gb200-dynamo：使用 vLLM nightly 镜像 (#2339)
* [NV] minimax-m3-gb200-dynamo: vLLM nightly + dynamo bump; collapse 2xDEP8 decode to 1xDEP16 minimaxm3-fp8-gb200-dynamo-vllm: - image -> vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 - dynamo -> 1.3.0.dev20260710 (recipe files) - drop minimax-m3-gb200-vllm-fixes.sh setup patch + launcher wiring (shipped in the nightly image) - every 2xDEP8 decode -> 1xDEP16 (decode_workers 2->1, gpus_per_decode 8->16, data-parallel-size 8->16); recipes renamed Xp2d-dep4-dep8 -> Xp1d-dep4-dep16 * [NV] minimax-m3-gb200-dynamo: add perf-changelog pr-link --------- Co-authored-by: Ankur-singh <ankusingh@nvidia.com>
1 parent 4ea94c9 commit ff5c115

10 files changed

Lines changed: 52 additions & 87 deletions

File tree

‎benchmarks/multi_node/srt-slurm-recipes/configs/minimax-m3-gb200-vllm-fixes.sh‎

Lines changed: 0 additions & 38 deletions
This file was deleted.

benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/1p2d-dep4-dep8-8k1k.yaml renamed to benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/1p1d-dep4-dep16-8k1k.yaml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
1-
name: "minimax-m3-vllm-disagg-gb200-1p2d-dep4-dep8-fp8-8k1k"
1+
name: "minimax-m3-vllm-disagg-gb200-1p1d-dep4-dep16-fp8-8k1k"
22

33
# 1P DEP4 prefill (TP1 DP4 EP, 4 GPU/worker) + 2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each).
44
# GB200 has 4 GPUs/node.
55
# Nodes: 1 prefill + 4 decode (+ head/infra).
66

77
model:
88
path: "minimax-m3-mxfp8"
9-
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
9+
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
1010
precision: "fp8"
1111

1212

1313
dynamo:
1414
install: true
15-
version: 1.3.0.dev20260614
15+
version: 1.3.0.dev20260710
1616

1717
health_check:
1818
max_attempts: 720
@@ -31,9 +31,9 @@ resources:
3131
prefill_nodes: 1
3232
decode_nodes: 4
3333
prefill_workers: 1
34-
decode_workers: 2
34+
decode_workers: 1
3535
gpus_per_prefill: 4
36-
gpus_per_decode: 8
36+
gpus_per_decode: 16
3737

3838
frontend:
3939
type: dynamo
@@ -84,7 +84,7 @@ backend:
8484
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
8585
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
8686
tensor-parallel-size: 1
87-
data-parallel-size: 8
87+
data-parallel-size: 16
8888
data-parallel-rpc-port: 13345
8989
enable-expert-parallel: true
9090
trust-remote-code: true

‎benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/1p2d-dep4-tep4-8k1k.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb200-1p2d-dep4-tep4-fp8-8k1k"
66

77
model:
88
path: "minimax-m3-mxfp8"
9-
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
9+
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
1010
precision: "fp8"
1111

1212

1313
dynamo:
1414
install: true
15-
version: 1.3.0.dev20260614
15+
version: 1.3.0.dev20260710
1616

1717
health_check:
1818
max_attempts: 720

‎benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/1p2d-dep4-tep8-8k1k.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@ name: "minimax-m3-vllm-disagg-gb200-1p2d-dep4-tep8-fp8-8k1k"
66

77
model:
88
path: "minimax-m3-mxfp8"
9-
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
9+
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
1010
precision: "fp8"
1111

1212

1313
dynamo:
1414
install: true
15-
version: 1.3.0.dev20260614
15+
version: 1.3.0.dev20260710
1616

1717
health_check:
1818
max_attempts: 720

benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/2p2d-dep4-dep8-8k1k.yaml renamed to benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/2p1d-dep4-dep16-8k1k.yaml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
1-
name: "minimax-m3-vllm-disagg-gb200-2p2d-dep4-dep8-fp8-8k1k"
1+
name: "minimax-m3-vllm-disagg-gb200-2p1d-dep4-dep16-fp8-8k1k"
22

33
# 2P DEP4 prefill (TP1 DP4 EP, 4 GPU/worker) + 2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each).
44
# GB200 has 4 GPUs/node.
55
# Nodes: 1 prefill + 4 decode (+ head/infra).
66

77
model:
88
path: "minimax-m3-mxfp8"
9-
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
9+
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
1010
precision: "fp8"
1111

1212

1313
dynamo:
1414
install: true
15-
version: 1.3.0.dev20260614
15+
version: 1.3.0.dev20260710
1616

1717
health_check:
1818
max_attempts: 720
@@ -31,9 +31,9 @@ resources:
3131
prefill_nodes: 2
3232
decode_nodes: 4
3333
prefill_workers: 2
34-
decode_workers: 2
34+
decode_workers: 1
3535
gpus_per_prefill: 4
36-
gpus_per_decode: 8
36+
gpus_per_decode: 16
3737

3838
frontend:
3939
type: dynamo
@@ -84,7 +84,7 @@ backend:
8484
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
8585
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
8686
tensor-parallel-size: 1
87-
data-parallel-size: 8
87+
data-parallel-size: 16
8888
data-parallel-rpc-port: 13345
8989
enable-expert-parallel: true
9090
trust-remote-code: true

benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/3p2d-dep4-dep8-8k1k.yaml renamed to benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/3p1d-dep4-dep16-8k1k.yaml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
1-
name: "minimax-m3-vllm-disagg-gb200-3p2d-dep4-dep8-fp8-8k1k"
1+
name: "minimax-m3-vllm-disagg-gb200-3p1d-dep4-dep16-fp8-8k1k"
22

33
# 3P DEP4 prefill (TP1 DP4 EP, 4 GPU/worker) + 2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each).
44
# GB200 has 4 GPUs/node.
55
# Nodes: 2 prefill + 4 decode (+ head/infra).
66

77
model:
88
path: "minimax-m3-mxfp8"
9-
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
9+
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
1010
precision: "fp8"
1111

1212

1313
dynamo:
1414
install: true
15-
version: 1.3.0.dev20260614
15+
version: 1.3.0.dev20260710
1616

1717
health_check:
1818
max_attempts: 720
@@ -31,9 +31,9 @@ resources:
3131
prefill_nodes: 3
3232
decode_nodes: 4
3333
prefill_workers: 3
34-
decode_workers: 2
34+
decode_workers: 1
3535
gpus_per_prefill: 4
36-
gpus_per_decode: 8
36+
gpus_per_decode: 16
3737

3838
frontend:
3939
type: dynamo
@@ -84,7 +84,7 @@ backend:
8484
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
8585
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
8686
tensor-parallel-size: 1
87-
data-parallel-size: 8
87+
data-parallel-size: 16
8888
data-parallel-rpc-port: 13345
8989
enable-expert-parallel: true
9090
trust-remote-code: true

benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/5p2d-dep4-dep8-8k1k.yaml renamed to benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8/8k1k/5p1d-dep4-dep16-8k1k.yaml

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,18 @@
1-
name: "minimax-m3-vllm-disagg-gb200-5p2d-dep4-dep8-fp8-8k1k"
1+
name: "minimax-m3-vllm-disagg-gb200-5p1d-dep4-dep16-fp8-8k1k"
22

33
# 5P DEP4 prefill (TP1 DP4 EP, 4 GPU/worker) + 2D DEP8 decode (TP1 DP8 EP, 8 GPU/worker = 2 nodes each).
44
# GB200 has 4 GPUs/node.
55
# Nodes: 3 prefill + 4 decode (+ head/infra).
66

77
model:
88
path: "minimax-m3-mxfp8"
9-
container: "vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223"
9+
container: "vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9"
1010
precision: "fp8"
1111

1212

1313
dynamo:
1414
install: true
15-
version: 1.3.0.dev20260614
15+
version: 1.3.0.dev20260710
1616

1717
health_check:
1818
max_attempts: 720
@@ -31,9 +31,9 @@ resources:
3131
prefill_nodes: 5
3232
decode_nodes: 4
3333
prefill_workers: 5
34-
decode_workers: 2
34+
decode_workers: 1
3535
gpus_per_prefill: 4
36-
gpus_per_decode: 8
36+
gpus_per_decode: 16
3737

3838
frontend:
3939
type: dynamo
@@ -84,7 +84,7 @@ backend:
8484
kv-transfer-config: '{"kv_connector": "NixlConnector", "kv_role": "kv_both"}'
8585
attention-config: '{"backend": "FLASHINFER", "use_trtllm_attention": true, "indexer_kv_dtype": "fp8"}'
8686
tensor-parallel-size: 1
87-
data-parallel-size: 8
87+
data-parallel-size: 16
8888
data-parallel-rpc-port: 13345
8989
enable-expert-parallel: true
9090
trust-remote-code: true

‎configs/nvidia-master.yaml‎

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -7463,7 +7463,7 @@ qwen3.5-fp4-b200-trt:
74637463
# All prefill DEP4 (TP1 DP4 EP, 4 GPU/worker). Decode: Marlin, TEP8, DEP8, TEP4.
74647464
# 4 GPU/node (GB200 NVL72). FLASHINFER attention with FP8 KV cache.
74657465
minimaxm3-fp8-gb200-dynamo-vllm:
7466-
image: vllm/vllm-openai:minimax-m3-perf-arm64-13.0.1-7a67223
7466+
image: vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9
74677467
model: MiniMaxAI/MiniMax-M3-MXFP8
74687468
model-prefix: minimaxm3
74697469
runner: gb200
@@ -7486,11 +7486,11 @@ minimaxm3-fp8-gb200-dynamo-vllm:
74867486
ep: 4
74877487
dp-attn: true
74887488
additional-settings:
7489-
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb200-fp8/8k1k/1p2d-dep4-dep8-8k1k.yaml"
7489+
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb200-fp8/8k1k/1p1d-dep4-dep16-8k1k.yaml"
74907490
decode:
7491-
num-worker: 2
7492-
tp: 8
7493-
ep: 8
7491+
num-worker: 1
7492+
tp: 16
7493+
ep: 16
74947494
dp-attn: true
74957495

74967496
# 1p2d DEP4+TEP4, 3n: conc 4
@@ -7531,11 +7531,11 @@ minimaxm3-fp8-gb200-dynamo-vllm:
75317531
ep: 4
75327532
dp-attn: true
75337533
additional-settings:
7534-
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb200-fp8/8k1k/2p2d-dep4-dep8-8k1k.yaml"
7534+
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb200-fp8/8k1k/2p1d-dep4-dep16-8k1k.yaml"
75357535
decode:
7536-
num-worker: 2
7537-
tp: 8
7538-
ep: 8
7536+
num-worker: 1
7537+
tp: 16
7538+
ep: 16
75397539
dp-attn: true
75407540

75417541
# 3p2d DEP4+DEP8, 7n: conc 4096
@@ -7546,11 +7546,11 @@ minimaxm3-fp8-gb200-dynamo-vllm:
75467546
ep: 4
75477547
dp-attn: true
75487548
additional-settings:
7549-
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb200-fp8/8k1k/3p2d-dep4-dep8-8k1k.yaml"
7549+
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb200-fp8/8k1k/3p1d-dep4-dep16-8k1k.yaml"
75507550
decode:
7551-
num-worker: 2
7552-
tp: 8
7553-
ep: 8
7551+
num-worker: 1
7552+
tp: 16
7553+
ep: 16
75547554
dp-attn: true
75557555

75567556
# 5p2d DEP4+DEP8, 9n: conc 4096
@@ -7561,11 +7561,11 @@ minimaxm3-fp8-gb200-dynamo-vllm:
75617561
ep: 4
75627562
dp-attn: true
75637563
additional-settings:
7564-
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb200-fp8/8k1k/5p2d-dep4-dep8-8k1k.yaml"
7564+
- "CONFIG_FILE=recipes/vllm/minimax-m3-gb200-fp8/8k1k/5p1d-dep4-dep16-8k1k.yaml"
75657565
decode:
7566-
num-worker: 2
7567-
tp: 8
7568-
ep: 8
7566+
num-worker: 1
7567+
tp: 16
7568+
ep: 16
75697569
dp-attn: true
75707570

75717571
qwen3.5-fp4-b200-trt-mtp:

‎perf-changelog.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5216,3 +5216,10 @@
52165216
- "Bump image to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 and dynamo to 1.3.0.dev20260710; drop the minimax-m3-vllm-fixes.sh setup patch (shipped in the nightly image)"
52175217
- "conc 4096: drop the 4p2d-dep2-tep4 arm; add a 3p2d-dep2-dep8 arm for conc [1024, 2048] (prefill 3xDEP2, decode 2xDEP8)"
52185218
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2338
5219+
5220+
- config-keys:
5221+
- minimaxm3-fp8-gb200-dynamo-vllm
5222+
description:
5223+
- "Bump image to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 and dynamo to 1.3.0.dev20260710; drop the minimax-m3-gb200-vllm-fixes.sh setup patch (shipped in the nightly image)"
5224+
- "Collapse every 2xDEP8 decode into a single DEP16 worker (decode_workers 2->1, gpus_per_decode 8->16, data-parallel-size 8->16); recipes renamed Xp2d-dep4-dep8 -> Xp1d-dep4-dep16 (1p2d/2p2d/3p2d/5p2d)"
5225+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2339

‎runners/launch_gb200-nv.sh‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -414,10 +414,6 @@ elif [[ $FRAMEWORK == "dynamo-vllm" && $MODEL_PREFIX == "minimaxm3" && $PRECISIO
414414
git checkout sa-submission-q2-2026 || exit 1
415415
mkdir -p recipes/vllm/minimax-m3-gb200-fp8 || exit 1
416416
cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/vllm/minimax-m3-gb200-fp8" recipes/vllm/minimax-m3-gb200-fp8 || exit 1
417-
SRTCTL_SETUP_SCRIPT="minimax-m3-gb200-vllm-fixes.sh"
418-
cp \
419-
"$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/configs/$SRTCTL_SETUP_SCRIPT" \
420-
"configs/$SRTCTL_SETUP_SCRIPT" || exit 1
421417
elif [[ $FRAMEWORK == "dynamo-vllm" && $MODEL_PREFIX == "kimik2.5" && $PRECISION == "fp4" ]]; then
422418
git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR" || exit 1
423419
cd "$SRT_REPO_DIR" || exit 1

0 commit comments

Comments
 (0)