Skip to content

Commit c138338

Browse files
authored
minimaxm2.5-fp8-h200-vllm: switch 8k/1k attention backend to FLASH_ATTN (#1668)
* perf: switch minimaxm2.5-fp8-h200-vllm 8k/1k attention backend to FLASH_ATTN Switch the attention backend for the 8k/1k cell of minimaxm2.5-fp8-h200-vllm from FLASHINFER to FLASH_ATTN. ISL-conditional: the 1k/1k cell is unchanged (keeps FLASHINFER + --enable-flashinfer-autotune, byte-identical to prior behavior); only ISL=8192 triggers the swap. Appends a perf-changelog entry. * chore: set perf-changelog pr-link for minimaxm2.5-fp8-h200-vllm FA3 swap * chore: point perf-changelog pr-link to PR #1668
1 parent ea4f575 commit c138338

2 files changed

Lines changed: 17 additions & 2 deletions

File tree

benchmarks/single_node/fixed_seq_len/minimaxm2.5_fp8_h200.sh

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,14 @@ else
4242
EP=()
4343
fi
4444

45+
if [ "$ISL" = "8192" ]; then
46+
ATTN_BACKEND="FLASH_ATTN"
47+
AUTOTUNE_FLAG=()
48+
else
49+
ATTN_BACKEND="FLASHINFER"
50+
AUTOTUNE_FLAG=(--enable-flashinfer-autotune)
51+
fi
52+
4553
# Start GPU monitoring (power, temperature, clocks every second)
4654
start_gpu_monitor
4755

@@ -55,8 +63,8 @@ vllm serve "$MODEL" --port "$PORT" \
5563
--max-num-batched-tokens "$MAX_NUM_BATCHED_TOKENS" \
5664
--kv-cache-dtype fp8 \
5765
--moe-backend triton \
58-
--attention-backend FLASHINFER \
59-
--enable-flashinfer-autotune \
66+
--attention-backend "$ATTN_BACKEND" \
67+
"${AUTOTUNE_FLAG[@]}" \
6068
--compilation-config "$COMPILATION_CONFIG" \
6169
--no-enable-prefix-caching \
6270
--trust-remote-code > "$SERVER_LOG" 2>&1 &

perf-changelog.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3474,3 +3474,10 @@
34743474
- "Use scheduler-recv-interval values 2/60/30/1200/600/1920 for conc 1-4/8/16/32/64/128-256"
34753475
- "Set max-running-requests=256, chunked-prefill-size=16384, mem-fraction-static=0.8, cuda-graph-max-bs=CONC, and enable symm-mem"
34763476
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1544
3477+
3478+
- config-keys:
3479+
- minimaxm2.5-fp8-h200-vllm
3480+
description:
3481+
- "Switch attention backend from FLASHINFER to FLASH_ATTN for the 8k/1k cell of MiniMax-M2.5 FP8 H200 vLLM."
3482+
- "1k/1k cell not changed in this PR: at 1k/1k all three measured configs."
3483+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1668

0 commit comments

Comments
 (0)