Skip to content

Commit 2846e86

Browse files
committed
agentx: reshape kimik2.7 sweep to Pareto frontier + address review
- Frontier-only sweep: none [1,4,8] (low conc, fits GPU HBM) + lmcache [16,32] (above the KV cliff). Drops dominated/cliff points per review. - Fix recipe header: 'Variant of kimik2.5_fp4_mi355x.sh' (was self-referential) - perf-changelog: correct runner (cluster:mi355x-amds) and conc lists
1 parent 5b20d87 commit 2846e86

3 files changed

Lines changed: 5 additions & 5 deletions

File tree

benchmarks/single_node/agentic/kimik2.7_fp4_mi355x.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ set -x
44

55
# Agentic trace replay benchmark for Kimi-K2.7 FP4 on MI355X using vLLM.
66
#
7-
# Variant of kimik2.7_fp4_mi355x.sh that supports TWO KV configs:
7+
# Variant of kimik2.5_fp4_mi355x.sh that supports TWO KV configs:
88
# KV_OFFLOADING=none -> GPU KV only
99
# KV_OFFLOADING=dram KV_OFFLOAD_BACKEND=lmcache -> LMCache MP server + connector
1010
#

configs/amd-master.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -560,8 +560,8 @@ kimik2.7-fp4-mi355x-vllm-agentic:
560560
agentic-coding:
561561
- dram-utilization: 0.80
562562
search-space:
563-
- { tp: 4, kv-offloading: none, conc-list: [1, 4, 8, 16, 32, 42] }
564-
- { tp: 4, kv-offloading: dram, kv-offload-backend: lmcache, conc-list: [1, 4, 8, 16, 32, 42] }
563+
- { tp: 4, kv-offloading: none, conc-list: [1, 4, 8] }
564+
- { tp: 4, kv-offloading: dram, kv-offload-backend: lmcache, conc-list: [16, 32] }
565565

566566

567567
kimik2.5-fp4-mi355x-atom:

perf-changelog.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5006,8 +5006,8 @@
50065006
- config-keys:
50075007
- kimik2.7-fp4-mi355x-vllm-agentic
50085008
description:
5009-
- "Add Kimi-K2.7-Code-MXFP4 agentic-coding config on MI355X (cluster:gbt350docker->mi355x-amds) via vLLM v0.24.0, TP4"
5010-
- "Sweep KV-offloading none vs dram/LMCache at conc [1,4,8,16,32,42]; LMCache MP server + LMCacheMPConnector, dram-utilization 0.80"
5009+
- "Add Kimi-K2.7-Code-MXFP4 agentic-coding config on MI355X (runner: cluster:mi355x-amds) via vLLM v0.24.0, TP4"
5010+
- "Sweep KV-offloading none (conc [1,4,8]) vs dram/LMCache (conc [16,32]); LMCache MP server + LMCacheMPConnector, dram-utilization 0.80"
50115011
- "Recipe benchmarks/single_node/agentic/kimik2.7_fp4_mi355x.sh clones LMCache to container-local /opt/lmcache-src (pinned) so CI checkout never trips over root-owned build artifacts"
50125012
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2126
50135013

0 commit comments

Comments
 (0)