[NV] GB300 DeepSeek-V4-Pro AgentX Dynamo-SGLang session-routing upgrade / [NV] GB300 DeepSeek-V4-Pro AgentX Dynamo-SGLang 会话路由升级 - #2319
Conversation
…ynamo-Session-ID) Update the six DeepSeek-V4-Pro GB300 Dynamo-SGLang AgentX sweep points: - Dynamo 1.3.0.dev1 -> 1.3.0.dev20260718 (router queueing off by default, targeting the high-concurrency regression seen with queueing enabled). - Migrate AgentX session affinity from nvext.session_control to X-Dynamo-Session-ID headers: recipes set AIPERF_HTTP_X_DYNAMO_SESSION_ID_FROM_CORRELATION_ID=true; nginx affinity + Dynamo router use X-Dynamo-Session-ID with a 3600s TTL. benchmark_lib retains the legacy --use-dynamo-conv-aware-routing path when the header opt-in is absent. - Pin the aiperf submodule to 6c84373b (X-Dynamo-Session-ID header support). - Remove DYN_ROUTER_TEMPERATURE=10000000, restoring Dynamo's deterministic 0.0 default. - Add SGLANG_DISABLE_DRAFT_EXTEND_CUDA_GRAPH=1 to the agg + 2p1d recipes to avoid the pathological DSV4 EAGLE draft-extend CUDA-graph capture. - SGLang nightly-dev-cu13-20260711-7de33ce8 -> nightly-dev-cu13-20260719-99f5a6f4. - Add gb300-nv hardware metadata to runners.yaml (available-cpu-dram-mib, gpus-per-node), required by the agentic DRAM-offload matrix logic.
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
1 similar comment
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30051155312 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30051155312 |
|
/reuse-sweep-run |
…19) Replace the frozen pin to the aiperf PR #17 merge (6c84373b, which the integration branch reverted in #18) with the current cquil11/aiperf-agentx-v1.0 tip (655792405), where the X-Dynamo-Session-ID session-routing support was re-landed via aiperf #19. This tracks the branch InferenceX's aiperf submodule targets instead of a stranded pre-revert commit.
…-routing # Conflicts: # perf-changelog.yaml
Ankur-singh
changed the title
Resolve conflicts in benchmark_lib.sh, perf-changelog.yaml, and the AIPerf submodule pointer. Preserve the X-Dynamo-Session-ID header path alongside main's explicit legacy-routing opt-out, append the PR entry after every current main changelog entry, and use AIPerf f5ae8134, which descends from the PR's session-header commit. 中文：merge：将 PR #2319 与 main 同步。解决 benchmark_lib.sh、perf-changelog.yaml 和 AIPerf 子模块指针的冲突；在保留 X-Dynamo-Session-ID 请求头路径的同时兼容 main 中显式关闭旧版路由的配置，将本 PR 的记录追加到当前 main 变更日志末尾，并采用包含本 PR 会话请求头提交的后续 AIPerf 提交 f5ae8134。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30315392821 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30315392821 |
|
@csahithi the AgentX/AIPerf harness has been updated, please merge origin/main into your branch and refresh your submission. Additional tuning may be necessary depending on the config. I apologize for any inconvenience. This is an automated message. |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30586991521 |
4 participants
Summary
Update six DeepSeek-V4-Pro FP4 GB300 Dynamo-SGLang AgentX MTP recipe topologies covering nine concurrency points:
1.3.0.dev1to1.3.0.dev20260718; remove theDYN_ROUTER_TEMPERATURE=10000000override and use the deterministic default.nvext.session_controlactions toX-Dynamo-Session-IDheaders. The recipes setAIPERF_HTTP_X_DYNAMO_SESSION_ID_FROM_CORRELATION_ID=true; nginx affinity and the Dynamo router use the same header with a 3600-second TTL. Existing Dynamo recipes retain the legacy CLI path when the header opt-in is absent.655792405980c5211722bc45a5f8401f3bad304a, the merge commit of SemiAnalysisAI/aiperf#19, which re-landed the session-header support oncquil11/aiperf-agentx-v1.0.nightly-dev-cu13-20260711-7de33ce8to upstreamlmsysorg/sglang:nightly-dev-cu13-20260719-99f5a6f4.cluster:gb300-nvCPU-DRAM and GPU-per-node metadata required by the AgentX DRAM-offload matrix logic.Scope and validation
agentic-codingreplay only; AGG and DISAGG; MTP with synthetic AL2.49; trace-derived ISL/OSL rather than fixed-sequence 1k1k or 8k1k.41407f30b2886e2501f12f671d63fd90925a324d; all nine applicable multi-node AgentX jobs passed with the current SGLang image and Dynamo version.6c84373b1d638a7a9ac077ee8b90490b9e93e7d9and current pin655792405980c5211722bc45a5f8401f3bad304ahave identical file trees; the intervening commits are the revert and re-land sequence.中文说明
更新 DeepSeek-V4-Pro FP4 GB300 Dynamo-SGLang AgentX MTP 的 6 个配方拓扑，共覆盖 9 个并发点：
1.3.0.dev1升级到1.3.0.dev20260718；移除DYN_ROUTER_TEMPERATURE=10000000覆盖项，恢复确定性的默认行为。nvext.session_control操作迁移到X-Dynamo-Session-ID请求头。配方设置AIPERF_HTTP_X_DYNAMO_SESSION_ID_FROM_CORRELATION_ID=true，nginx 亲和性与 Dynamo 路由器统一使用该请求头，并将 TTL 设为 3600 秒。未启用该请求头的既有 Dynamo 配方仍保留旧版 CLI 路径。655792405980c5211722bc45a5f8401f3bad304a，即已合并的 SemiAnalysisAI/aiperf#19 的合并提交；该 PR 在cquil11/aiperf-agentx-v1.0上重新引入了会话请求头支持。nightly-dev-cu13-20260711-7de33ce8升级到上游镜像lmsysorg/sglang:nightly-dev-cu13-20260719-99f5a6f4。cluster:gb300-nv的 CPU DRAM 容量与单节点 GPU 数量元数据，供 AgentX DRAM 卸载矩阵逻辑使用。范围与验证
agentic-coding回放；同时覆盖聚合与分离式推理；使用 MTP 与合成接受长度(AL)2.49；ISL/OSL 由轨迹数据决定，不属于固定序列长度的 1k1k 或 8k1k。41407f30b2886e2501f12f671d63fd90925a324d上成功完成；9 个适用的多节点 AgentX 任务全部通过，并使用当前 SGLang 镜像与 Dynamo 版本。6c84373b1d638a7a9ac077ee8b90490b9e93e7d9与当前提交655792405980c5211722bc45a5f8401f3bad304a的文件树完全一致；中间提交仅对应回滚与重新引入流程。