[AgentX] vLLM DeepSeek-V4 B300 aggregate MTP#2258
Conversation
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Signed-off-by: Jeff Ma <jeffjma@umich.edu>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LjHXfE1FRN9c5XQqgeTdez
Restore upstream lines whitespace-for-whitespace and append the B300 entry so the changelog diff contains no deletions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LjHXfE1FRN9c5XQqgeTdez
# Conflicts: # configs/nvidia-master.yaml # perf-changelog.yaml
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
… the recipe Finalize the B300 AgentX vLLM search space (dsv4-fp4-b300-vllm-agentic): - Add a TP8 GPU-resident arm at conc [1,2,4,6,8]. - Trim TP4 GPU-resident to conc [1,2,4,6,8,12,16,20]; extend TP4 SimpleCPU lazy-offload down to conc [20,24,28,32,36,40]. - Add an MTP speculative-decoding twin (num_speculative_tokens=3) for every topology (TP8/TP4 GPU-resident, TP4 SimpleCPU, DEP4, DEP8), each mirroring its non-MTP conc-list, routed via the launcher's spec-decoding=mtp suffix to dsv4_fp4_b300_vllm_mtp.sh. MTP script: NUM_SPEC_TOKENS=3 -> TOKENS_PER_SEQ=4, so FULL_DECODE_ONLY cudagraph capture sizes (num_seqs*TOKENS_PER_SEQ) scale to num_seqs*4. Sync the tail with dsv4_fp4_b300_vllm.sh to restore the EVAL_ONLY branch (added upstream by #1947) so MTP configs also run the SWE-bench Lite accuracy eval. Add the perf-changelog entry for dsv4-fp4-b300-vllm-agentic (PR #2258). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tion #1947 made single-node agentic recipes generate a SWE-bench eval row (run_eval/eval_only + agentic fields) but never widened the changelog matrix schema, so ChangelogMatrixEntry.evals (typed list[SingleNodeMatrixEntry], fixed-seq-len only) rejects every agentic eval row -- breaking check-changelog for any single-node agentic PR. Widen evals to the same Union single_node already uses, and give SingleNodeAgenticMatrixEntry optional run_eval/eval_only (None-default, so benchmark rows are unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29564785969 |
Restructure so the sweep runs only the new work, not the existing aggregate: - Revert dsv4-fp4-b300-vllm-agentic to its main (#2241) search space (TP4 [1..32], SimpleCPU [28..40], DEP4, DEP8) -- unchanged vs main, so it is not re-run. - Add dsv4-fp4-b300-vllm-agentic-mtp: MTP twins of the aggregate arms (num_speculative_tokens=3), reshaped TP4 [1..20] / SimpleCPU [20..40]. - Add dsv4-fp4-b300-vllm-agentic-tp8: new TP8 GPU-resident topology at conc [1,2,4,6,8], both non-MTP and MTP. - Point the perf-changelog entry at only the two new keys, so only they sweep. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29566371654 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29566417002 |
Inject rejection_sample_method=synthetic + synthetic_acceptance_length=2.49 into the dsv4-fp4-b300-vllm MTP speculative-config, matching the dsv4-pro golden AL (thinking_on, num_speculative_tokens=3) and the sglang gb300 MTP recipe (SGLANG_SIMULATE_ACC_LEN=2.49). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-mtp # Conflicts: # perf-changelog.yaml
TP4, DEP4, and TP8 use gpu-memory-utilization 0.95 (down from 0.96); DEP8 stays at 0.92 for its larger prefill token budget. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29614980545 |
Remove the dsv4-fp4-b300-vllm-agentic-tp8 config-key entirely; its TP8 GPU-resident MTP arm moves into dsv4-fp4-b300-vllm-agentic-mtp (the non-MTP TP8 baseline is dropped). Also drop conc 64 and 72 from the DEP4 arm (now [32,40,48,56]). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29626988314 |
Eval-only, agentic-coding-scoped entry for dsv4-fp4-b300-vllm-agentic so the PR sweep (and the on-push main sweep after merge) runs exactly one job: the DEP8 top-concurrency SWE-bench eval through the new sweep-agentic-evals dispatch, confirming the fix for the eval-only failures first seen on #2258/#2259. 中文：新增 changelog 条目以端到端验证 agentic 评估调度——针对 dsv4-fp4-b300-vllm-agentic 的 evals-only、仅 agentic-coding 场景条目，使 PR 扫描（及合并后 main 分支的 push 触发扫描）只运行一个任务：DEP8 最高并发 的 SWE-bench 评估经由新的 sweep-agentic-evals 调度执行，确认对 #2258/#2259 上首次出现的 eval-only 失败的修复。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29626988314 |
… sweep job / 修复 agentic SWE-bench 评估调度：新增 agentic_evals 独立分桶与扫描任务 (#2267) * fix: dispatch agentic SWE-bench eval rows via a dedicated bucket and sweep job Since #1947, generate_sweep_configs --evals-only marks the top-concurrency agentic arm of each config as a SWE-bench eval row, but two consumers were never taught about agentic eval rows, so any PR touching a single-node agentic-coding config failed its sweep: 1. Schema: ChangelogMatrixEntry.evals only accepted SingleNodeMatrixEntry (fixed-seq-len), so process_changelog.py's final model_validate rejected the agentic row and check-changelog failed before any GPU work. 2. Dispatch: even with the schema widened, run-sweep.yml's sweep-evals job dispatches every eval row with fixed-seq-len inputs; an agentic row gets empty ISL/OSL/MAX_MODEL_LEN and scenario-type defaults to fixed-seq-len, so the launcher runs the fixed_seq_len benchmark script, which exits on check_env_vars. Fix: route agentic eval rows into a new agentic_evals bucket (mirroring the single_node['agentic'] design) dispatched by a new sweep-agentic-evals job cloned from sweep-agentic's input block plus run-eval/eval-only, so an agentic row can never reach the fixed-seq-len eval dispatch. The agentic model accepts optional run-eval/eval-only carried by eval rows; exclude_none keeps benchmark row output byte-identical. 中文：自 #1947 起，generate_sweep_configs --evals-only 会将每个配置中并发度最高的 agentic 搜索空间条目标记为 SWE-bench 评估行，但下游两个消费方并不认识 agentic 评估行，导致任何涉及单节点 agentic-coding 配置的 PR 扫描失败：其一，schema 层 ChangelogMatrixEntry.evals 仅接受固定序列长度条目，check-changelog 直接报错； 其二，调度层 sweep-evals 以固定序列长度的输入派发评估行，ISL/OSL/MAX_MODEL_LEN 为空且 scenario-type 回落为 fixed-seq-len，启动器因此运行错误的基准测试脚本并 在 check_env_vars 处退出。本修复新增 agentic_evals 独立分桶（与 single_node['agentic'] 设计一致），由新的 sweep-agentic-evals 任务按 sweep-agentic 的输入派发并附加 run-eval/eval-only，使 agentic 评估行不可能进入 固定序列长度的评估调度路径；基准测试行的输出保持逐字节不变。 * test: trim agentic eval bucket tests to essential coverage Collapse fixtures into module constants, merge the accept/reject checks into one test, and drop redundant cases; added test lines go from 212 to 100 with the two core guards kept (agentic rows validate only in agentic_evals; realistic rows split end-to-end through the real schema). 中文：精简 agentic 评估分桶测试——fixture 合并为模块常量，接受/拒绝检查合并 为单个测试并移除冗余用例，新增测试行数从 212 降至 100，保留两项核心守卫 （agentic 行仅能通过 agentic_evals 校验；真实行形状端到端经过真实 schema 拆分）。 * chore: add changelog entry to validate agentic eval dispatch end-to-end Eval-only, agentic-coding-scoped entry for dsv4-fp4-b300-vllm-agentic so the PR sweep (and the on-push main sweep after merge) runs exactly one job: the DEP8 top-concurrency SWE-bench eval through the new sweep-agentic-evals dispatch, confirming the fix for the eval-only failures first seen on #2258/#2259. 中文：新增 changelog 条目以端到端验证 agentic 评估调度——针对 dsv4-fp4-b300-vllm-agentic 的 evals-only、仅 agentic-coding 场景条目，使 PR 扫描（及合并后 main 分支的 push 触发扫描）只运行一个任务：DEP8 最高并发 的 SWE-bench 评估经由新的 sweep-agentic-evals 调度执行，确认对 #2258/#2259 上首次出现的 eval-only 失败的修复。 * ws
…x validation" This reverts commit 5887e1a.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29638680674 |
Make the MTP --speculative-config conditional on EVAL_ONLY: throughput points pin synthetic acceptance length 2.49 (dsv4-pro golden AL), while the auto-selected SWE-bench accuracy run (EVAL_ONLY=true) uses real MTP with normal target verification. Synthetic acceptance bypasses verification and corrupts the eval output (0.0000 score). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29661297620 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29661297620 |
|
vllm recipe: vllm-project/recipes#645 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=29778042138 |
|
/reuse-sweep-run 29778042138 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have:
Additional detail section:
Signed: |
✅✅✅ Verdict: PASS ✅✅✅✅ Check 0 (CODEOWNER): PASS — signer covered via |
4 participants
Adds DeepSeek-V4 B300 vLLM AgentX MTP coverage for GPU-resident, SimpleCPU-offloaded, and DEP topologies. Uses synthetic AL 2.49 for throughput, real verification for evals, and forwards DCP/PCP from the generated config.