Skip to content

Commit 7cd3b30

Browse files
committed
fix(agentic): capture GB300 SGLang worker metrics
Layer topology-aware worker endpoint discovery onto the pinned srt-slurm release and opt all GB300 DeepSeek-V4 AgentX recipes into AIPerf server metrics collection. Add the missing GB300 hardware metadata so the DRAM-offload matrix generates. 中文:在固定版本的 srt-slurm 上叠加拓扑感知的 worker 指标端点发现逻辑,并为全部 GB300 DeepSeek-V4 AgentX 配方启用 AIPerf 服务端指标采集。同时补充 GB300 硬件元数据,确保 DRAM KV 卸载矩阵可正常生成。
1 parent a141bfc commit 7cd3b30

9 files changed

Lines changed: 24 additions & 1 deletion

benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/agentic/agg-gb300-tp4-mtp-kvoffload.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,7 @@ srun_options:
127127

128128
benchmark:
129129
type: custom
130+
aiperf_server_metrics: true
130131
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
131132
env:
132133
INFMAX_CONTAINER_WORKSPACE: /infmax-workspace

benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/agentic/disagg-gb300-12p4d-dep8-dep16-c1024-mtp-kvoffload.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,7 @@ srun_options:
200200

201201
benchmark:
202202
type: custom
203+
aiperf_server_metrics: true
203204
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
204205
env:
205206
INFMAX_CONTAINER_WORKSPACE: /infmax-workspace

benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/agentic/disagg-gb300-2p1d-dep8-tp4-c8-mtp-kvoffload.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,7 @@ srun_options:
199199

200200
benchmark:
201201
type: custom
202+
aiperf_server_metrics: true
202203
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
203204
env:
204205
INFMAX_CONTAINER_WORKSPACE: /infmax-workspace

benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/agentic/disagg-gb300-2p4d-dep8-dep16-c256-mtp-kvoffload.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,7 @@ srun_options:
200200

201201
benchmark:
202202
type: custom
203+
aiperf_server_metrics: true
203204
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
204205
env:
205206
INFMAX_CONTAINER_WORKSPACE: /infmax-workspace

benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/agentic/disagg-gb300-4p4d-dep8-dep16-c256-mtp-kvoffload.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,7 @@ srun_options:
200200

201201
benchmark:
202202
type: custom
203+
aiperf_server_metrics: true
203204
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
204205
env:
205206
INFMAX_CONTAINER_WORKSPACE: /infmax-workspace

benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/agentic/disagg-gb300-6p4d-dep8-dep16-c512-mtp-kvoffload.yaml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,7 @@ srun_options:
200200

201201
benchmark:
202202
type: custom
203+
aiperf_server_metrics: true
203204
command: bash /infmax-workspace/benchmarks/multi_node/agentic_srt.sh
204205
env:
205206
INFMAX_CONTAINER_WORKSPACE: /infmax-workspace

configs/runners.yaml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -315,6 +315,9 @@ hardware:
315315
cluster:gb200-nv:
316316
available-cpu-dram-mib: 860_160
317317
gpus-per-node: 4
318+
cluster:gb300-nv:
319+
available-cpu-dram-mib: 860_160
320+
gpus-per-node: 4
318321
cluster:mi300x-amds:
319322
available-cpu-dram-mib: 2_321_924
320323
gpus-per-node: 8

perf-changelog.yaml

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4974,3 +4974,11 @@
49744974
- "Run 29651235293 showed the 1M-context corpus working set outgrowing the HBM KV pool past conc 8 (TP8) / conc 64 (DP8): gpu_kv_cache_usage pinned at 1.0 and the radix hit rate collapsed from a ~0.97 theoretical ceiling to 0.04-0.06, so every post-knee turn re-prefilled its full history and throughput fell together with interactivity"
49754975
- "HiCache spills evicted prefixes to host DRAM and restores them at C2C bandwidth instead of recomputing; sizing follows the qwen3.5-fp8-b300-sglang-agentic-hicache recipe (GLM-5.2 is plain GQA: one host pool per rank, GB-based --hicache-size)"
49764976
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2280
4977+
4978+
- config-keys:
4979+
- dsv4-fp4-gb300-dynamo-sglang-agentic-agg
4980+
- dsv4-fp4-gb300-dynamo-sglang-agentic-disagg
4981+
description:
4982+
- "Correctly capture AIPerf Prometheus metrics from the Dynamo frontend and every logical SGLang worker endpoint"
4983+
- "Register GB300 hardware metadata required to generate AgentX DRAM-offload configs"
4984+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX

runners/launch_gb300-nv.sh

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,10 +142,16 @@ if [[ "$IS_AGENTIC" == "1" && $FRAMEWORK == "dynamo-sglang" && $MODEL_PREFIX ==
142142
# DSv4 GB300 sglang agentic: NVIDIA/srt-slurm v1.0.10 has the nginx
143143
# client_max_body_size fix (>1 MiB agentic warmup bodies), the
144144
# session-affinity frontend, and the BenchmarkType.CUSTOM / extra_mount
145-
# schema these recipes need.
145+
# schema these recipes need. Layer the pinned custom-AIPerf metrics patch
146+
# onto that release so srt-slurm injects one topology-ordered Prometheus
147+
# URL per logical SGLang worker leader. This keeps hostname/port discovery
148+
# in the host-side orchestrator, which owns the resolved process topology.
146149
git clone https://github.com/NVIDIA/srt-slurm.git "$SRT_REPO_DIR"
147150
cd "$SRT_REPO_DIR"
148151
git checkout v1.0.10
152+
git fetch https://github.com/cquil11/srt-slurm-nv.git \
153+
de59739b172e507e15ebf145bfe305f606e82fbf
154+
git cherry-pick --no-commit FETCH_HEAD
149155
mkdir -p recipes/sglang/deepseek-v4/agentic
150156
cp -rT "$GITHUB_WORKSPACE/benchmarks/multi_node/srt-slurm-recipes/sglang/deepseek-v4/agentic" \
151157
recipes/sglang/deepseek-v4/agentic

0 commit comments

Comments
 (0)