Skip to content

minimaxm2.5-fp8-h200-vllm: switch 8k/1k attention backend to FLASH_ATTN #1667

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
RohitNagraj wants to merge 2 commits into from
Closed

minimaxm2.5-fp8-h200-vllm: switch 8k/1k attention backend to FLASH_ATTN #1667

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0ac4655
perf: switch minimaxm2.5-fp8-h200-vllm 8k/1k attention backend to FLA…
RohitNagraj Jun 4, 2026
424fe77
chore: set perf-changelog pr-link for minimaxm2.5-fp8-h200-vllm FA3 swap
RohitNagraj Jun 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions benchmarks/single_node/fixed_seq_len/minimaxm2.5_fp8_h200.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,14 @@ else
EP=()
fi

if [ "$ISL" = "8192" ]; then
ATTN_BACKEND="FLASH_ATTN"
AUTOTUNE_FLAG=()
else
ATTN_BACKEND="FLASHINFER"
AUTOTUNE_FLAG=(--enable-flashinfer-autotune)
fi

# Start GPU monitoring (power, temperature, clocks every second)
start_gpu_monitor

Expand All @@ -55,8 +63,8 @@ vllm serve "$MODEL" --port "$PORT" \
--max-num-batched-tokens "$MAX_NUM_BATCHED_TOKENS" \
--kv-cache-dtype fp8 \
--moe-backend triton \
--attention-backend FLASHINFER \
--enable-flashinfer-autotune \
--attention-backend "$ATTN_BACKEND" \
"${AUTOTUNE_FLAG[@]}" \
--compilation-config "$COMPILATION_CONFIG" \
--no-enable-prefix-caching \
--trust-remote-code > "$SERVER_LOG" 2>&1 &
Expand Down
7 changes: 7 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3474,3 +3474,10 @@
- "Use scheduler-recv-interval values 2/60/30/1200/600/1920 for conc 1-4/8/16/32/64/128-256"
- "Set max-running-requests=256, chunked-prefill-size=16384, mem-fraction-static=0.8, cuda-graph-max-bs=CONC, and enable symm-mem"
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1544

- config-keys:
- minimaxm2.5-fp8-h200-vllm
description:
- "Switch attention backend from FLASHINFER to FLASH_ATTN for the 8k/1k cell of MiniMax-M2.5 FP8 H200 vLLM."
- "1k/1k cell not changed in this PR: at 1k/1k all three measured configs."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1667

Check failure on line 3483 in perf-changelog.yaml

View check run for this annotation

Claude / Claude Code Review

perf-changelog entry: wrong PR link and truncated description bullet 

The new perf-changelog entry at lines 3478-3483 has two documentation defects: (1) `pr-link` is set to `/pull/1667` but this is PR #1668 — every other entry in the file links to its own introducing PR (see lines 3460/3467/3476), and the latest commit `424fe77` was explicitly intended to set this link correctly; (2) the second description bullet `"1k/1k cell not changed in this PR: at 1k/1k all three measured configs."` is grammatically incomplete — the clause after the colon has no verb. Fix the