Skip to content

[DNM][AgentX][AMD] DSv4 FP4 MI355X SGLang: bump image + aiter/sglang patches #2298

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
seungrokj wants to merge 2 commits into from
+22 −3
Closed

[DNM][AgentX][AMD] DSv4 FP4 MI355X SGLang: bump image + aiter/sglang patches #2298

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7939644
[AgentX] DSv4 FP4 MI355X SGLang: bump image + aiter/sglang patches
seungrokj Jul 22, 2026
8d251c4
Update perf-changelog pr-link to #2298
seungrokj Jul 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions benchmarks/single_node/agentic/dsv4_fp4_mi355x_sglang.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,17 @@ export SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=1
METRICS_ARGS=(--enable-metrics)
SPEC_ARGS=()

# ---- Temp patch (TODO:remove) ----------------------------------------------------------

@functionstackx functionstackx Jul 22, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no patches per the new guideline from an couple weeks ago https://github.com/SemiAnalysisAI/InferenceX/blob/main/docs/PR_REVIEW_CHECKLIST.md

GIST_BASE="https://gist.githubusercontent.com/seungrokj/9911b69be4c6b0f70ebca5dc03b5fc5d/raw"
curl -LsSf "$GIST_BASE/df54b1879567807bbfc92e5ad004b74c90d030cb/dsv4_fp8fp4_tuned_fmoe.csv" \
-o /sgl-workspace/aiter/aiter/configs/model_configs/dsv4_fp8fp4_tuned_fmoe.csv
curl -LsSf "$GIST_BASE/4a79f11bca8607b6b9438e9fd37af16a2534e3fb/dsv4_fp8fp4_untuned_fmoe.csv" \
-o /sgl-workspace/aiter/aiter/configs/model_configs/dsv4_fp8fp4_untuned_fmoe.csv
curl -LsSf "$GIST_BASE/bc4ba01751f2576d6ff099ffd2167d2a76fe85b6/shuffle.py" \
-o /sgl-workspace/aiter/aiter/ops/shuffle.py
curl -LsSf "$GIST_BASE/741d6e90dceac4ed21284c8bfc6f2c25d9442649/fp8.py" \
-o /sgl-workspace/sglang/python/sglang/srt/layers/quantization/fp8.py

SGLANG_CMD=(
python3 -m sglang.launch_server
--model-path "$MODEL_PATH"
Expand Down
6 changes: 3 additions & 3 deletions configs/amd-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1536,7 +1536,7 @@ dsr1-fp4-mi355x-sglang-disagg-mtp:
- "DECODE_MTP_SIZE=1"

dsv4-fp4-mi355x-sglang-agentic-hicache:
image: lmsysorg/sglang-rocm:v0.5.14-rocm720-mi35x-20260710
image: lmsysorg/sglang-rocm:v0.5.15.post1-rocm700-mi35x-20260721
model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: cluster:mi355x-amds
Expand All @@ -1547,9 +1547,9 @@ dsv4-fp4-mi355x-sglang-agentic-hicache:
agentic-coding:
- dram-utilization: 0.80
search-space:
- { tp: 8, kv-offloading: none, conc-list: [1, 2, 4, 8] }
#- { tp: 8, kv-offloading: none, conc-list: [1, 2, 4, 8] }
- { tp: 8, dp-attn: true, kv-offloading: none, conc-list: [16, 32, 48, 64] }
- { tp: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [64] }
#- { tp: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [64] }

# MiniMax-M3 MXFP8 MI355X recipe:
# https://github.com/vllm-project/recipes/commit/2a3728ed9892debfd767a72a58ebc90b33f186e5
Expand Down
8 changes: 8 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5019,3 +5019,11 @@
- "dsv4-fp4-b200-vllm-agentic-mtp: MTP twins (--speculative-config {\"method\":\"mtp\",\"num_speculative_tokens\":3}) of the aggregate arms -- TP8 GPU-resident, TP8 SimpleCPU, DEP8 SimpleCPU, DEP8 Mooncake, each mirroring its non-MTP conc-list; routed via spec-decoding=mtp to dsv4_fp4_b200_vllm_mtp.sh."
- "MTP capture sizes are in TOKENS: --max-cudagraph-capture-size scaled to MAX_NUM_SEQS*(1+num_speculative_tokens) (= MAX_NUM_SEQS*4) so FULL_DECODE_ONLY covers the largest decode batches."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2259

- config-keys:
- dsv4-fp4-mi355x-sglang-agentic-hicache
description:
- "Bump image to lmsysorg/sglang-rocm:v0.5.15.post1-rocm700-mi35x-20260721"
- "Patch aiter tuned/untuned FMoE CSV configs, shuffle.py, and sglang fp8.py at runtime from gist"
- "Comment out non-DP-attention and HiCache sweep rows to focus on DP-attention conc 16-64"
Comment on lines +5022 to +5028

@claude claude Bot Jul 22, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 This PR's title and description are English-only, but AGENTS.md (line 7) requires every PR title/description to be bilingual (title format <English title> / <中文标题>, plus a Chinese translation/## 中文说明 section in the body). Please add the Chinese title suffix and a 中文说明 section mirroring the Summary/Test plan.

Extended reasoning...

AGENTS.md, the repo's mandatory contribution guide (which CLAUDE.md explicitly delegates all guidance to), states at line 7: "PR and GitHub-issue titles & descriptions must be bilingual — include a Simplified Chinese version in addition to English. Title format: <English title> / <中文标题>. In the PR/issue body, follow the English content with its Chinese translation (e.g. a ## 中文说明 section mirroring the summary). This applies to every PR and every issue."

This PR's title is '[AgentX] DSv4 FP4 MI355X SGLang: bump image + aiter/sglang patches' — English only, with no '/ <中文标题>' suffix. The body consists solely of an English '## Summary' and '## Test plan' section; there is no Chinese translation anywhere, and no '## 中文说明' section mirroring the summary. This same English-only content is what gets mirrored into the perf-changelog.yaml entry added by this PR (lines 5022-5028), so the changelog description bullets inherit the same English-only gap.

Nothing in the existing tooling enforces this at PR-creation time — there is no CI check or template validation blocking an English-only title/body, so the requirement relies entirely on author discipline. That it's easy to miss is corroborated by looking at the two immediately preceding merged PRs in this same repo, e.g. commit 01e9e73 ('... / 将 LMCache 分支并入官方 DSV4 FP4 B200 vLLM AgentX 扫描') and 9ecc56d ('... / 在调优后的 DSV4 FP4 B300 vLLM AgentX 配方上新增 LMCache 分支'), both of which do follow the bilingual convention — showing it is an actively-observed norm in this repo that this PR simply didn't follow.

Step-by-step verification:

  1. Open AGENTS.md at the repo root and read line 7 — the bilingual requirement is stated unconditionally ('applies to every PR and every issue'), not as a suggestion.
  2. Compare against this PR's actual title: '[AgentX] DSv4 FP4 MI355X SGLang: bump image + aiter/sglang patches'. There is no '/' followed by a Chinese-character title segment.
  3. Read the PR body: it contains '## Summary' (3 English bullets) and '## Test plan' (2 English checklist items), and the auto-generated 'Generated with Claude Code' footer — no '## 中文说明' or any Chinese text appears anywhere.
  4. Cross-check the requirement is actively followed elsewhere in the repo's history (01e9e73, 9ecc56d) to confirm this isn't dead/unenforced documentation.
  5. Conclude: the mandatory bilingual format is unmet for both the title and the body of this PR.

Fix: the PR author should retitle to append ' / <中文标题>' (a Chinese translation of the current English title), and add a '## 中文说明' section to the body that mirrors the Summary bullets (image bump, aiter/sglang gist patches, and the sweep-row comment-outs) and Test plan in Simplified Chinese.

This is a process/documentation compliance issue only — it does not affect the benchmark's runtime correctness, the sglang server launch, or the perf-changelog data itself, so it doesn't warrant blocking the merge. It is, however, an explicit and unambiguous repo-mandated rule (not subjective style), so per review policy it should still be surfaced even though its default severity is nit.

pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2298
Loading