Skip to content

Commit 8970c9a

Browse files
committed
agentx: restore perf-changelog with kimik2.7 entry
Previous commit accidentally dropped perf-changelog.yaml (blob too large for inline arg). Restore it with the kimik2.7-fp4-mi355x-vllm-agentic entry appended.
1 parent 4075e6c commit 8970c9a

1 file changed

Lines changed: 17 additions & 1 deletion

File tree

perf-changelog.yaml

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4992,7 +4992,6 @@
49924992
- "LMCache MP server (lmcache_driven transfer mode) + LMCacheMPConnector per PR #2153/#2231; L1 pool derated to 75% of TOTAL_CPU_DRAM_GB; PYTORCH_ALLOC_CONF=expandable_segments:True dropped on the lmcache arms only (VMM allocations cannot be CUDA-IPC-exported to the LMCache server, same failure mode as cuMem on B200); gpu-memory-utilization 0.92 on lmcache DEP8 (matches the official DEP8 derate) and 0.94 on lmcache TP4/DEP4 (at 0.96 the PR #2232 bring-up sweeps hit torch-pool, DeepGEMM-JIT, and cuBLAS-workspace OOMs; 23 lmcache points validated green in runs 29463061871/29535333851 after the derate)"
49934993
- "LMCache ladders offset the official arms: TP4 +4 conc [32, 36, 40, 44] vs SimpleCPU [28, 32, 36, 40]; DEP4 +4 conc [36, 44, 52, 60, 68, 76] vs SimpleCPU [32, 40, 48, 56, 64, 72]; DEP8 +8 conc capped at 208 [72, 104, 120, 136, 152, 168, 184, 200] vs GPU-resident [64, 96, 112, 128, 144, 160, 176, 192, 224]"
49944994
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2232
4995-
49964995

49974996
- config-keys:
49984997
- dsv4-fp4-b200-vllm-agentic
@@ -5003,3 +5002,20 @@
50035002
- "LMCache MP server (lmcache_driven transfer mode) + LMCacheMPConnector per PR #2153; L1 pool derated to 75% of TOTAL_CPU_DRAM_GB; --enable-cumem-allocator dropped on the lmcache arm only (cuMem/VMM allocations cannot be CUDA-IPC-exported to the LMCache server)"
50045003
- "LMCache TP8 points mirror the vllm-simple ladder (conc [8, 12, 16]); LMCache DEP8 points mirror the Mooncake ladder (conc [12, 20, 28, 36, 44, 52, 60, 68, 76]) for a like-for-like backend comparison; the PR #2231 bring-up sweep validated the recipe across DEP8 conc 8-80 (peak ~25.7k total tok/s/GPU at conc 72, 96-98% cache hit)"
50055004
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2231
5005+
5006+
- config-keys:
5007+
- kimik2.7-fp4-mi355x-vllm-agentic
5008+
description:
5009+
- "Add Kimi-K2.7-Code-MXFP4 agentic-coding config on MI355X (runner: cluster:mi355x-amds) via vLLM v0.24.0, TP4"
5010+
- "Sweep KV-offloading none (conc [1,4,8]) vs dram/LMCache (conc [16,32]); LMCache MP server + LMCacheMPConnector, dram-utilization 0.80"
5011+
- "Recipe benchmarks/single_node/agentic/kimik2.7_fp4_mi355x.sh clones LMCache to container-local /opt/lmcache-src (pinned) so CI checkout never trips over root-owned build artifacts"
5012+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2126
5013+
5014+
- config-keys:
5015+
- kimik2.7-fp4-mi355x-vllm-agentic
5016+
description:
5017+
- "Move the agentic-coding sweep from TP4 to TP8 and compare three KV tiers on the agentic trace: GPU-only (none) at conc [4,8], and dram/LMCache + dram/Mooncake at conc [16,32,48] (3600s trace replay)"
5018+
- "TP4 was GPU-KV-capacity bound: the ~121 GiB pool held only ~12 resident ~90k-token contexts, so throughput plateaued (~160 tok/s) and requests preempted/thrashed from conc16 up. GPU-only still collapses under load at TP8, so it is swept only at low conc; the dram offload backends carry the high-conc range"
5019+
- "Add a Mooncake KV-offload backend to the recipe (embedded MooncakeStoreConnector store). Built from source for ROCm via Mooncake's build_wheel.sh and pip-installed so the mooncake package lands in site-packages and is importable from the vLLM worker subprocesses (a bare make install left workers unable to import it)"
5020+
- "Adds optional per-scenario 'duration' override to the agentic-coding config schema (defaults to 3600s when unset)"
5021+
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2126

0 commit comments

Comments
 (0)