[NV] Update B300 DSV4 SGLang Pareto sweep#1575
Conversation
…-frontier # Conflicts: # perf-changelog.yaml
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
1 similar comment
|
Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook If it is not, please create a PR first before we can merge your single node PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. If re-running failed jobs is attempted, PR authors are responsible for ensuring it passes. See GitHub's docs on re-running failed jobs: https://docs.github.com/en/actions/how-tos/manage-workflow-runs/re-run-workflows-and-jobs#re-running-failed-jobs-in-a-workflow As a rule of thumb, generally, PR authors should request a review & get a PR approval from the respective companies' CODEOWNERS before requesting a review from core maintainers. If additional help is needed, PR authors can reach out to core maintainers over Slack. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26542138862 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - config-keys: | ||
| - dsv4-fp4-b300-sglang | ||
| description: | ||
| - "Update DeepSeek-V4-Pro FP4 B300 SGLang non-MTP sweep to the 2026-05-19 8k/1k submission frontier: TP8 no-DP-attention c1-c64 and DEP8 DP-attention c512/c768/c1024/c1536/c2048" | ||
| - "Use lmsysorg/sglang:nightly-dev-cu13-20260522-7cf193fe to pick up the merged SGLang warmup path" | ||
| - "Map dp-attn=false to TP8 flashinfer_mxfp4 with chunked-prefill 8192; map dp-attn=true to DEP8 mixed-chunk MegaMoE throughput settings" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1552 |
There was a problem hiding this comment.
🔴 The new perf-changelog.yaml entry's pr-link points to PR #1552 (the closed/superseded original), but the merging PR is #1575. This breaks the file's convention (every neighboring entry links to its own merging PR) and will trip utils/merge_with_reuse.sh: its conflict-resolution helper skips entries whose link doesn't end with /pull/<current-pr>, and the post-merge assert last["pr-link"].endswith("/$PR") would fail. Change the link to /pull/1575 (or use the XXX placeholder).
Extended reasoning...
What the bug is
The new entry appended at perf-changelog.yaml:3175-3181 sets:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1552…but the PR being merged is #1575 (the rebased successor — the PR description itself notes "Rebased copy of #1552 with origin/main merged in"). PR #1552 is closed and will not contain the merge commit for these changes.
Why this is a real issue, not just cosmetic
Every other recent entry in perf-changelog.yaml links to its own merging PR (e.g. the immediately preceding dsv4-fp4-mi355x-sglang entry → /pull/1568, others → /pull/1555, /pull/1558, /pull/1516, /pull/1354). AGENTS.md documents this convention. More importantly, the repo's canonical merge tool utils/merge_with_reuse.sh (referenced from .claude/commands/merge-prs.md) enforces it programmatically:
- At ~line 136, when resolving
perf-changelog.yamlconflicts it filters incoming entries with:if "XXX" not in link and not link.endswith(f"/pull/{pr}"): continue. Withpr=1575andlink=…/pull/1552, this entry would be silently skipped during automated conflict resolution, producing "No PR contributions found". - At ~line 172, after the merge it runs
assert last["pr-link"].endswith("/$PR"). With$PR=1575and the last entry pointing at/pull/1552, this assertion would fail.
Step-by-step proof
- Maintainer runs
utils/merge_with_reuse.sh 1575. - The script fetches
origin/main, attempts the merge, and hits a conflict inperf-changelog.yaml(likely, given how frequently this file changes). - The embedded Python helper walks the incoming entries from the PR branch. For each new entry it checks:
if "XXX" not in link and not link.endswith(f"/pull/{pr}"): continue
Withpr = "1575"andlink = "https://github.com/SemiAnalysisAI/InferenceX/pull/1552", the condition is true → the entry is dropped from the merged result. - Even if no conflict arises (so the helper isn't invoked), the post-merge sanity check runs:
assert last["pr-link"].endswith(f"/{pr}")
Since the last entry is the new one and ends with/1552, this assertion raises and the merge tool aborts. - Downstream readers using
pr-linkto find the diff/merge commit land on a closed, superseded PR instead of the one that actually merged.
Fix
Update line 3181 to:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/1575or, equivalently, use the XXX placeholder that merge_with_reuse.sh rewrites at merge time:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXThe historical reference to #1552 (which the PR description already provides) can stay in the PR description; the changelog entry should point at the PR that actually lands the change, both for convention and for tooling correctness.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26542138862 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26542138862 |
…tale mount comment
Resolve conflicts keeping PR #1575's Pareto two-line design for dsv4-fp4-b300-sglang: dev-cu13 image + conc-list search-space, with the DP_ATTENTION benchmark script relocated to fixed_seq_len/. Keep main for all other configs, the -mtp entry, and the changelog (our entry appended).
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit fbeb15a. Configure here.
| set -x | ||
| PYTHONNOUSERSITE=1 sglang serve \ | ||
| --model-path $MODEL_PATH --served-model-name $MODEL \ | ||
| --model-path $MODEL \ |
There was a problem hiding this comment.
Server ignores staged MODEL_PATH
High Severity
On B300, launch_b300-nv.sh keeps MODEL as the Hugging Face id and sets MODEL_PATH to pre-staged weights, but this script passes --model-path $MODEL and no longer uses MODEL_PATH or --served-model-name, unlike dsr1_fp4_b300.sh and dsv4_fp4_b300_sglang_mtp.sh.
Reviewed by Cursor Bugbot for commit fbeb15a. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=26987594224 |
2 participants
Summary
dsv4-fp4-b300-sglangtolmsysorg/sglang:nightly-dev-cu13-20260522-7cf193feTesting
bash -n benchmarks/single_node/dsv4_fp4_b300_sglang.shgit diff --checkuv run --python 3.13 --with pydantic --with pyyaml python utils/matrix_logic/generate_sweep_configs.py test-config --config-keys dsv4-fp4-b300-sglang --config-files .github/configs/nvidia-master.yaml --no-evalsuv run --python 3.13 --with pydantic --with pyyaml python utils/matrix_logic/generate_sweep_configs.py test-config --config-keys dsv4-fp4-b300-sglang --config-files .github/configs/nvidia-master.yaml --evals-onlyuv run --python 3.13 --with pydantic --with pyyaml python utils/process_changelog.py --base-ref origin/main --head-ref HEAD --changelog-file perf-changelog.yaml --trim-concNote
Low Risk
Benchmark matrix and shell launch tuning only; no application auth, data, or serving logic outside perf harness configs.
Overview
Aligns the DeepSeek-V4-Pro FP4 B300 SGLang (non-MTP) benchmark with the 2026-05-19 8k/1k submission frontier and a newer nightly container.
In
.github/configs/nvidia-master.yaml, bumpsdsv4-fp4-b300-sglangtolmsysorg/sglang:nightly-dev-cu13-20260604-14ed9b44and replaces sparse single-concsearch-space rows withconc-listsweeps: TP8 /dp-attn: falsefor c1–c64 and TP8 EP8 /dp-attn: truefor c512–c2048 (same pattern for 1k/1k and 8k/1k). Comments now describe selection viadp-attninstead of per-CONCscript logic.benchmarks/single_node/fixed_seq_len/dsv4_fp4_b300_sglang.shdrops the oldCONC-keyed launch profiles in favor of two recipes keyed byDP_ATTENTION: low-latency flashinfer_mxfp4 TP-only vs throughput MegaMoE / mixed-chunk DEP8. Model loading skipshf downloadwhen the runner uses a staged/data/modelspath; serving uses--model-path $MODELand fixed--max-running-requests. Adds shared SGLang JIT/opt env defaults and documents mount paths for the dev image.perf-changelog.yamldocuments the config-key change and PR link.Reviewed by Cursor Bugbot for commit fbeb15a. Bugbot is set up for automated code reviews on this repo. Configure here.