feat: DEP (EP8) for glm5.2 b300 sglang high-throughput arm, cap conc at 64 / 为 glm5.2 b300 sglang 高吞吐分支启用 DEP(EP8)并将并发上限设为 64

Add ep: 8 to the TP8/DP8 attention-DP arm of glm5.2-fp4-b300-sglang-agentic
so the MoE layers run expert-parallel across all 8 ranks (--ep-size 8)
instead of TP-sharded (the arm previously left ep unset -> EP_SIZE=1).
Attention stays data-parallel behind sglang-router consistent hashing.
The recipe already passes --ep-size EP_SIZE unconditionally, so only the
master-config search space changes; no recipe logic change is needed.
Cap the DEP arm at conc 64 (conc-list [48, 64], was up to 512). The
low-latency TP8 arm (conc 1-32, fp8 KV + cutedsl GEMM) is unchanged. DEP
mirrors the proven dsv4-fp4-b300-sglang tp8/ep8/dp-attn arm.

中文：为 glm5.2-fp4-b300-sglang-agentic 的 TP8/DP8 注意力数据并行(attention-DP)
分支添加 ep: 8，使 混合专家(MoE) 层在全部 8 个 rank 上按 专家并行(EP) 运行
(--ep-size 8)，而非此前的 TP 切分(未设置 ep -> EP_SIZE=1)。注意力仍为数据
并行，并置于 sglang-router 一致性哈希之后以保持会话亲和性。该 recipe 脚本本就
无条件传入 --ep-size EP_SIZE，故仅需修改 master 配置的搜索空间，无需改动脚本
逻辑。将 DEP 分支并发上限设为 64(conc-list [48, 64]，原先最高 512)。低延迟
TP8 分支(并发 1-32，fp8 KV + cutedsl GEMM)保持不变。此 DEP 配置对齐已验证的
dsv4-fp4-b300-sglang tp8/ep8/dp-attn 分支。

Author: Oseltamivir

benchmarks/single_node/agentic/glm5.2_fp4_b300_sglang.sh

@@ -8,7 +8,8 @@ set -x
 # (https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.2), STP only:
 # the cookbook's EAGLE MTP variants are intentionally not wired up yet.
 #   DP_ATTENTION=false -> low-latency arm (TP8, fp8 KV, cutedsl bf16 GEMM)
-#   DP_ATTENTION=true  -> high-throughput arm (TP8 + DP8 attention-DP)
+#   DP_ATTENTION=true  -> high-throughput DEP arm (TP8 + DP8 attention-DP +
+#                         EP_SIZE expert-parallel MoE via --ep-size)
 #
 # Required env vars:
 #   MODEL, TP, CONC, KV_OFFLOADING, TOTAL_CPU_DRAM_GB, RESULT_DIR, DURATION,

configs/nvidia-master.yaml

@@ -7858,9 +7858,10 @@ qwen3.5-fp8-gb300-dynamo-sglang:
 # single-node recipes (https://docs.sglang.io/cookbook/autoregressive/GLM/GLM-5.2),
 # STP only (the cookbook's EAGLE MTP variants are deferred). The TP8 arm is the
 # cookbook low-latency recipe (fp8 KV, cutedsl bf16 GEMM) and covers the
-# interactivity end; the TP8/DP8 attention-DP arm is the cookbook
-# high-throughput recipe behind sglang-router consistent hashing for session
-# affinity. Conc lists are disjoint between arms so exp-names stay unique.
+# interactivity end; the TP8/DP8/EP8 DEP arm (attention-DP + expert-parallel
+# MoE) is the cookbook high-throughput recipe behind sglang-router consistent
+# hashing for session affinity. Conc lists are disjoint between arms so
+# exp-names stay unique.
 glm5.2-fp4-b300-sglang-agentic:
   image: lmsysorg/sglang:v0.5.15.post1-cu130
   model: nvidia/GLM-5.2-NVFP4
@@ -7874,4 +7875,5 @@ glm5.2-fp4-b300-sglang-agentic:
     - dram-utilization: 0.80
       search-space:
       - { tp: 8, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [1, 2, 4, 8, 16, 32] }
-      - { tp: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [48, 64, 96, 128, 192, 256, 512], router: { name: sglang-router, version: "0.3.2" } }
+      # DEP (attention-DP + EP8 expert-parallel MoE) throughput arm, capped at conc 64.
+      - { tp: 8, ep: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [48, 64], router: { name: sglang-router, version: "0.3.2" } }

perf-changelog.yaml

@@ -4966,3 +4966,11 @@
     - "Run 29651235293 showed the 1M-context corpus working set outgrowing the HBM KV pool past conc 8 (TP8) / conc 64 (DP8): gpu_kv_cache_usage pinned at 1.0 and the radix hit rate collapsed from a ~0.97 theoretical ceiling to 0.04-0.06, so every post-knee turn re-prefilled its full history and throughput fell together with interactivity"
     - "HiCache spills evicted prefixes to host DRAM and restores them at C2C bandwidth instead of recomputing; sizing follows the qwen3.5-fp8-b300-sglang-agentic-hicache recipe (GLM-5.2 is plain GQA: one host pool per rank, GB-based --hicache-size)"
   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2280
+
+- config-keys:
+    - glm5.2-fp4-b300-sglang-agentic
+  description:
+    - "Convert the high-throughput arm to DEP: add ep: 8 to the TP8/DP8 attention-DP arm so the MoE layers run expert-parallel across all 8 ranks (--ep-size 8) instead of TP-sharded (the arm previously left ep unset -> EP_SIZE=1). Attention stays data-parallel behind sglang-router consistent hashing; the recipe already passes --ep-size EP_SIZE unconditionally, so only the master-config search space changes"
+    - "Cap the DEP arm concurrency at 64: conc-list [48, 64] (was [48, 64, 96, 128, 192, 256, 512])"
+    - "Low-latency TP8 arm (conc [1, 2, 4, 8, 16, 32], fp8 KV + cutedsl bf16 GEMM) unchanged; DEP mirrors the proven dsv4-fp4-b300-sglang tp8/ep8/dp-attn DEP arm"
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/PENDING