Experiment with MiniMax-M2.7 EAGLE on B200 / 在 B200 上实验 MiniMax-M2.7 EAGLE#2289
Experiment with MiniMax-M2.7 EAGLE on B200 / 在 B200 上实验 MiniMax-M2.7 EAGLE#2289cquil11 wants to merge 10 commits into
Conversation
Add pinned public NVFP4 target and full-vocabulary EAGLE3 draft staging, TP4/TP8 8K/1K and AgentX matrices, and shared SGLang launch logic with chat templates.\n\n中文：新增固定版本的公开 NVFP4 目标模型与全词表 EAGLE3 草稿模型暂存逻辑、TP4/TP8 的 8K/1K 与 AgentX 扫描矩阵，以及启用聊天模板的共享 SGLang 启动逻辑。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
Record the draft PR URL in the append-only performance changelog before starting benchmark jobs.\n\n中文：在启动基准测试任务前，将草稿 PR 链接记录到仅追加的性能变更日志中。
Partition checkpoint prefetching across tensor-parallel ranks to avoid redundant network reads during repeated B200 engine startups.\n\n中文：在张量并行 rank 之间划分检查点预取，避免 B200 推理引擎重复启动时产生冗余网络读取。
Reduce the SGLang static memory fraction from 0.90 to 0.85 after TP4 and TP8 smoke runs OOMed while capturing the 16K prefill CUDA graph.\n\n中文：TP4 和 TP8 冒烟测试在捕获 16K 预填充 CUDA Graph 时显存溢出，因此将 SGLang 静态显存比例从 0.90 降至 0.85，为 EAGLE 图捕获预留空间。
Select gpu-1 only for the MiniMax-M2.7 experiment after confirming all ten nodes are healthy and idle while gpu-2 is saturated. Other B200 benchmark families keep their existing partition.\n\n中文：确认 gpu-1 的十个节点均健康且空闲、gpu-2 已满载后，仅将 MiniMax-M2.7 实验调度到 gpu-1；其他 B200 基准测试仍使用原分区。
Keep benchmark jobs on gpu-2 after Slurm association inspection confirmed sa-shared has only gpu-2_qos; gpu-1 rejects submissions despite idle nodes.\n\n中文：检查 Slurm 关联后确认 sa-shared 仅拥有 gpu-2_qos，因此继续在 gpu-2 上运行基准测试；gpu-1 即使有空闲节点也会拒绝提交。
Use a 150-minute model-scoped Slurm limit instead of the launcher-wide eight-hour default, retaining setup, warmup, and one-hour AgentX headroom while permitting backfill.\n\n中文：为该模型使用 150 分钟的 Slurm 时限，替代启动器默认的八小时；既覆盖准备、预热和一小时 AgentX 测量，也允许调度器进行回填。
Bind the AgentX replay limit to the 196608-token SGLang server window so oversized Weka trajectory turns are excluded before they become deterministic HTTP 400 failures. 中文：将 AgentX 重放上限绑定到 SGLang 的 196608 token 上下文窗口，使超长 Weka 轨迹轮次在发送前被排除，避免确定性的 HTTP 400 失败。
Allow up to 30 minutes for AgentX requests admitted during the measurement window to finish. TP4 was healthy and decoding when the 30-second AIPerf default grace cancelled its only smoke profile request. 中文：允许在测量窗口内已接收的 AgentX 请求最多用 30 分钟完成。TP4 服务保持健康并持续解码，但 AIPerf 默认的 30 秒宽限期取消了冒烟测试中唯一的测量请求。
同步最新 main，并解决配置与性能变更日志冲突；保留 MiniMax 实验条目为追加项。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| flock -u 8 | ||
| else | ||
| hf download "$TARGET_MODEL" --revision "$TARGET_REVISION" | ||
| hf download "$DRAFT_MODEL" --revision "$DRAFT_REVISION" | ||
| DRAFT_MODEL_PATH="$DRAFT_MODEL" | ||
| fi | ||
|
|
||
| python3 - "$TARGET_MODEL_PATH" "$DRAFT_MODEL_PATH" <<'PY' | ||
| import json | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
| target_path, draft_path = map(Path, sys.argv[1:]) | ||
| target = json.loads((target_path / "config.json").read_text()) | ||
| draft = json.loads((draft_path / "config.json").read_text()) | ||
| quant = target.get("quantization_config", {}) | ||
| if quant.get("quant_algo") != "NVFP4" or quant.get("quant_method") != "modelopt": | ||
| raise SystemExit(f"Unexpected NVFP4 target configuration: {quant}") | ||
| if target.get("vocab_size") != 200064: | ||
| raise SystemExit(f"Unexpected target vocabulary: {target.get('vocab_size')}") | ||
| if draft.get("architectures") != ["LlamaForCausalLMEagle3"]: | ||
| raise SystemExit(f"Unexpected EAGLE3 architecture: {draft.get('architectures')}") | ||
| if draft.get("vocab_size") != 200064 or draft.get("num_hidden_layers") != 1: | ||
| raise SystemExit( | ||
| f"Unexpected EAGLE3 draft contract: vocab={draft.get('vocab_size')} " |
There was a problem hiding this comment.
🟡 In the hub-cache fallback branch (no --local-dir), TARGET_MODEL_PATH/DRAFT_MODEL_PATH are left as bare HF repo IDs (e.g. nvidia/MiniMax-M2.7-NVFP4) instead of resolved filesystem paths, so the Python preflight's (target_path / "config.json").read_text() raises FileNotFoundError before the server starts. This is a real defect in the documented hub-cache/local-invocation mode, but it's not exercised in CI since launch_b200-dgxc.sh always exports an absolute MODEL_PATH for this recipe (routing through the --local-dir branch instead); fix by capturing the resolved snapshot dir from hf download (or passing --local-dir) in that branch too.
Extended reasoning...
The new benchmark entrypoint minimaxm2.7_fp4_b200_sglang_mtp.sh supports two model-resolution modes: an absolute-path branch (used when MODEL_PATH is set or MODEL is already an absolute path) that passes --local-dir to hf download and correctly sets TARGET_MODEL_PATH/DRAFT_MODEL_PATH to real directories, and an else branch (lines 63-67) intended for hub-cache/local invocation where MODEL is the bare HF id and MODEL_PATH is unset.
In that else branch, hf download "$TARGET_MODEL" --revision "$TARGET_REVISION" is called without --local-dir, which places the downloaded files under the HF hub cache (e.g. ~/.cache/huggingface/hub/models--nvidia--MiniMax-M2.7-NVFP4/snapshots/<rev>/...), not in a directory literally named nvidia/MiniMax-M2.7-NVFP4 relative to $PWD. Despite this, the script sets TARGET_MODEL_PATH="$TARGET_MODEL" (i.e. the bare repo id string) and DRAFT_MODEL_PATH="$DRAFT_MODEL" (the bare draft repo id) rather than capturing the resolved snapshot path.
Those two variables are then fed straight into the inline Python preflight:
target_path, draft_path = map(Path, sys.argv[1:])
target = json.loads((target_path / "config.json").read_text())With target_path = Path("nvidia/MiniMax-M2.7-NVFP4"), this resolves to the nonexistent relative path ./nvidia/MiniMax-M2.7-NVFP4/config.json, and read_text() raises FileNotFoundError. Because the script runs under set -Eeuo pipefail, this immediately aborts the run — sglang.launch_server is never even invoked, even though sglang itself would have tolerated the bare repo id and resolved it via the HF cache fine. So the preflight validation step, not the actual model loading, is what breaks this mode.
Step-by-step proof:
- Run the script locally/manually with
MODEL=nvidia/MiniMax-M2.7-NVFP4andMODEL_PATHunset (exactly the "local validation" scenario the PR description references). TARGET_MODEL_PATHis set to the string"nvidia/MiniMax-M2.7-NVFP4"(line ~65-67), not a filesystem path.hf download nvidia/MiniMax-M2.7-NVFP4 --revision ...(no--local-dir) populates~/.cache/huggingface/hub/...; no./nvidia/MiniMax-M2.7-NVFP4directory is created in$PWD.- The Python preflight executes
Path("nvidia/MiniMax-M2.7-NVFP4") / "config.json", which does not exist relative to$PWD. read_text()raisesFileNotFoundError, the script exits nonzero underset -e, and the server never starts.
Why this doesn't block the current CI sweep: runners/launch_b200-dgxc.sh hardcodes MODEL_PATH="/lustre/fsw/gharunners/models/MiniMax-M2.7-NVFP4" for MODEL_PREFIX=minimaxm2.7/fp4 and then does export MODEL="$MODEL_PATH", so in the actual GitHub Actions sweep TARGET_MODEL_PATH is always an absolute path and the --local-dir branch is taken instead — this else branch is dead code from CI's perspective. The PR's own validation (bash -n plus matrix generation) doesn't execute the script, so it wouldn't have caught this either. All three independent verifiers reached the same conclusion and rated it nit severity for this reason.
Suggested fix: capture the resolved snapshot directory that hf download prints/returns (or, simpler, always pass --local-dir "$TARGET_MODEL_PATH"/"$DRAFT_MODEL_PATH" pointing at a real staging directory even in the hub-cache branch, mirroring the absolute-path branch's behavior) so the preflight always receives real filesystem paths.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29946677513 |
3 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29946677513 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29946677513 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29946677513 |
1 participant
Summary
Validation
bash -non the launcher and both benchmark entrypointspython -m pytest utils/matrix_logic/ -v(224 passed)中文说明
验证
bash -npython -m pytest utils/matrix_logic/ -v（224 项通过）