fix: recover PR 2279 ingest via sweep reuse (#2280)

* feat: DRAM KV offloading via HiCache for all glm5.2 b300 agentic points

Run 29651235293 showed the 1M-context corpus working set outgrowing the
HBM KV pool past conc 8 (TP8) / conc 64 (DP8): gpu_kv_cache_usage pinned
at 1.0 while the radix hit rate collapsed from a ~0.97 theoretical
ceiling to 0.04-0.06, so every post-knee turn re-prefilled its whole
history and throughput fell together with interactivity, leaving the
frontier a single dominated cliff. Spill evicted prefixes to a per-rank
pinned host pool (0.80 x node DRAM / TP) instead of recomputing them,
following the qwen3.5 b300 hicache recipe; GLM-5.2 is plain GQA so there
is one host pool per rank and GB-based --hicache-size works (the DSv4
ratio-based sizing is a DSv4-only workaround).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: fill perf-changelog pr-link for glm5.2 b300 hicache entry

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: ratio-based HiCache sizing (0.75) for DSA-family GLM-5.2

Run 29678598595 OOM-killed both arms at scheduler init: GLM-5.2 runs
attention_backend=dsa (DeepSeek-sparse family; the 169.98 GB device KV
pool is replicated on every rank), so the GB-based --hicache-size of
TOTAL_CPU_DRAM_GB/TP pinned the whole 0.80-DRAM budget (8 x 299 GB =
2.4 TB) on top of 465 GB of weights on a 3 TB node. Control host
capacity through the host/device token-capacity ratio like the DSv4
recipe — but DSv4's ratio=2 default also OOMs here (2 x 170 GB x 8 =
2.7 TB; GLM-5.2's device pool is far larger than DSv4's), so use
fractional ratio 0.75 = ~128 GB/rank = ~1.0 TB total, matching the
cluster's proven host-pool envelope. Adopt DSv4's write_back / direct /
page_first_direct transfer flags. Validated on-node (b300-019,
2026-07-19): both sizing regressions reproduced, ratio 0.75 boots in
~290 s and survives a 4.2M-token overflow bench that forces eviction
through the DSA KV+INDEXER host pools.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore: prepare PR 2279 ingest recovery

* fix: recover PR 2279 ingest

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

Author: Oseltamivir

perf-changelog.yaml

@@ -4958,3 +4958,11 @@
     - "Run 29651235293 showed the 1M-context corpus working set outgrowing the HBM KV pool past conc 8 (TP8) / conc 64 (DP8): gpu_kv_cache_usage pinned at 1.0 and the radix hit rate collapsed from a ~0.97 theoretical ceiling to 0.04-0.06, so every post-knee turn re-prefilled its full history and throughput fell together with interactivity"
     - "HiCache spills evicted prefixes to host DRAM and restores them at C2C bandwidth instead of recomputing; sizing follows the qwen3.5-fp8-b300-sglang-agentic-hicache recipe (GLM-5.2 is plain GQA: one host pool per rank, GB-based --hicache-size)"
   pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2279
+
+- config-keys:
+    - glm5.2-fp4-b300-sglang-agentic
+  description:
+    - "Enable DRAM KV offloading (SGLang HiCache) on all points of both arms: DSv4-style host/device token-capacity ratio, tuned to 0.75 for GLM-5.2 (DSA/MLA-family: the 169.98 GB device pool is replicated on all 8 ranks, so ratio 0.75 = ~1.0 TB pinned host pools; ratio 2 and GB-based sizing both OOM-killed 3 TB nodes), io-backend direct, mem-layout page_first_direct, write policy write_back; validated on-node with an overflow bench forcing eviction through the DSA KV+INDEXER host pools"
+    - "Run 29651235293 showed the 1M-context corpus working set outgrowing the HBM KV pool past conc 8 (TP8) / conc 64 (DP8): gpu_kv_cache_usage pinned at 1.0 and the radix hit rate collapsed from a ~0.97 theoretical ceiling to 0.04-0.06, so every post-knee turn re-prefilled its full history and throughput fell together with interactivity"
+    - "HiCache spills evicted prefixes to host DRAM and restores them at C2C bandwidth instead of recomputing; sizing follows the qwen3.5-fp8-b300-sglang-agentic-hicache recipe (GLM-5.2 is plain GQA: one host pool per rank, GB-based --hicache-size)"
+  pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2280