Skip to content

minimaxm2.5-fp8-h200-vllm: switch 8k/1k attention backend to FLASH_ATTN #1668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
RohitNagraj merged 3 commits into from
Jun 5, 2026
+17 −2
Merged

minimaxm2.5-fp8-h200-vllm: switch 8k/1k attention backend to FLASH_ATTN #1668

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0ac4655
perf: switch minimaxm2.5-fp8-h200-vllm 8k/1k attention backend to FLA…
RohitNagraj Jun 4, 2026
424fe77
chore: set perf-changelog pr-link for minimaxm2.5-fp8-h200-vllm FA3 swap
RohitNagraj Jun 4, 2026
e5edb8c
chore: point perf-changelog pr-link to PR #1668
RohitNagraj Jun 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions benchmarks/single_node/fixed_seq_len/minimaxm2.5_fp8_h200.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,14 @@ else
EP=()
fi

if [ "$ISL" = "8192" ]; then
ATTN_BACKEND="FLASH_ATTN"
AUTOTUNE_FLAG=()
else
ATTN_BACKEND="FLASHINFER"
AUTOTUNE_FLAG=(--enable-flashinfer-autotune)
fi

# Start GPU monitoring (power, temperature, clocks every second)
start_gpu_monitor

Expand All @@ -55,8 +63,8 @@ vllm serve "$MODEL" --port "$PORT" \
--max-num-batched-tokens "$MAX_NUM_BATCHED_TOKENS" \
--kv-cache-dtype fp8 \
--moe-backend triton \
--attention-backend FLASHINFER \
--enable-flashinfer-autotune \
--attention-backend "$ATTN_BACKEND" \
"${AUTOTUNE_FLAG[@]}" \
--compilation-config "$COMPILATION_CONFIG" \
--no-enable-prefix-caching \
--trust-remote-code > "$SERVER_LOG" 2>&1 &
Expand Down
7 changes: 7 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3474,3 +3474,10 @@
- "Use scheduler-recv-interval values 2/60/30/1200/600/1920 for conc 1-4/8/16/32/64/128-256"
- "Set max-running-requests=256, chunked-prefill-size=16384, mem-fraction-static=0.8, cuda-graph-max-bs=CONC, and enable symm-mem"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1544

- config-keys:
- minimaxm2.5-fp8-h200-vllm
description:
- "Switch attention backend from FLASHINFER to FLASH_ATTN for the 8k/1k cell of MiniMax-M2.5 FP8 H200 vLLM."
- "1k/1k cell not changed in this PR: at 1k/1k all three measured configs."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1668