Skip to content

Commit a63228b

Browse files
hyukjleeclaude
andcommitted
[AMD][AgentX] kimik2.7: drop --block-size=1 (incompatible with TRITON_MLA)
With VLLM_ROCM_USE_AITER_MLA=0 the only ROCm MLA backend is TRITON_MLA, which rejects block_size=1 ("No valid attention backend found ... TRITON_MLA: block_size not supported"). --block-size=1 only ever existed to force the now-disabled AITER-MLA path. Drop it and let vLLM pick a TRITON_MLA-valid default, matching AMD's reference serve command (AITER_MLA=0, no --block-size). Fixes bringup failure for every cell (pure-TP and DEP alike). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent a35a71f commit a63228b

1 file changed

Lines changed: 5 additions & 5 deletions

File tree

benchmarks/single_node/agentic/kimik2.7_fp4_mi355x.sh

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -54,10 +54,11 @@ fi
5454

5555
export VLLM_ROCM_USE_AITER=1
5656
# amd/Kimi-K2.7-Code-MXFP4 has 64 KV heads; the ROCm AITER-MLA kernel only
57-
# supports 16 or 128 heads, so it MUST be disabled for this model. Previously
58-
# left unset (inherited master=1) and only worked via vLLM's implicit backend
59-
# fallback -- make it explicit so a fallback/backend-selection change (or the
60-
# --block-size=1 AITER-MLA forcing path) can't turn this into a hard crash.
57+
# supports 16 or 128 heads, so it MUST be disabled for this model. With AITER-MLA
58+
# off, ROCm serves MLA via TRITON_MLA, which does NOT support block_size=1 -- so
59+
# this recipe passes no --block-size and lets vLLM pick a TRITON_MLA-valid
60+
# default (matches AMD's reference serve command for this model). The old
61+
# --block-size=1 only existed to force the (now-disabled) AITER-MLA path.
6162
export VLLM_ROCM_USE_AITER_MLA=0
6263
export VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4
6364
# Avoid intermittent symm_mem all-reduce rendezvous hang at engine init on
@@ -343,7 +344,6 @@ VLLM_CMD=(
343344
"${PARALLEL_ARGS[@]}"
344345
"${EP_ARGS[@]}"
345346
--gpu-memory-utilization 0.90
346-
--block-size=1
347347
--trust-remote-code
348348
--max-num-seqs "$CONC"
349349
--mm-encoder-tp-mode data

0 commit comments

Comments
 (0)