Skip to content

Commit bb205f8

Browse files
hyukjleeclaude
andcommitted
[AMD][AgentX] kimik2.7 TP8 3600s: none c[4,8], lmcache+mooncake c[16,32,48]; fix Mooncake worker import
Reshape the TP8 agentic sweep for the real 3600s comparison run: - none (GPU-only) swept only at low conc [4,8] (it collapses under load) - LMCache and Mooncake carry the high-conc [16,32,48] range - duration 3600s (the prior 900s smoke was warmup-dominated for the ~90k-token agentic inputs, so per-request metrics were unreliable) Fix Mooncake so the vLLM worker subprocesses can import it: the previous bare `make install` only laid down C++ libs, so the launcher imported `mooncake` but the workers hit "Please install mooncake ..." at KV-cache init. Build the wheel via Mooncake's scripts/build_wheel.sh (auditwheel bundles every .so) and pip install it into site-packages, and build with -DWITH_STORE=ON so store.so (MooncakeStoreConnector) is present. Drop the temporary configs/smoke-kimik2.7-mooncake.yaml overlay; all three backends now run from one canonical config. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 9fa02ce commit bb205f8

3 files changed

Lines changed: 22 additions & 26 deletions

File tree

benchmarks/single_node/agentic/kimik2.7_fp4_mi355x.sh

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -211,18 +211,31 @@ case "$OFFLOAD_MODE" in
211211
# if the store module isn't importable. Clone to a container-local dir
212212
# (NOT bind-mounted /workspace) so the next job's checkout `clean: true`
213213
# won't trip over root-owned build artifacts.
214+
#
215+
# A bare `make install` only lays down the C++ libs, so the launcher can
216+
# import `mooncake` but the vLLM worker SUBPROCESSES cannot (fresh
217+
# site-packages) -> "Please install mooncake ..." at KV-cache init. Build
218+
# the wheel via Mooncake's own scripts/build_wheel.sh (auditwheel bundles
219+
# every .so into a self-contained package) and pip install it so the
220+
# `mooncake` package lands in site-packages and is importable everywhere.
214221
if ! python3 -c "from mooncake.store import MooncakeDistributedStore" >/dev/null 2>&1; then
215222
MOONCAKE_SRC_DIR="${MOONCAKE_SRC_DIR:-/opt/mooncake-src}"
216223
MOONCAKE_GIT_REF="${MOONCAKE_GIT_REF:-main}"
224+
MOONCAKE_PYVER="$(python3 -c 'import sys;print(f"{sys.version_info.major}.{sys.version_info.minor}")')"
225+
pip install --quiet build auditwheel patchelf
217226
rm -rf "$MOONCAKE_SRC_DIR"
218227
git clone https://github.com/kvcache-ai/Mooncake.git "$MOONCAKE_SRC_DIR"
219228
( cd "$MOONCAKE_SRC_DIR"
220229
git checkout "$MOONCAKE_GIT_REF"
221230
bash dependencies.sh
222231
mkdir -p build && cd build
223-
cmake ..
232+
cmake .. -DWITH_STORE=ON
224233
make -j
225-
make install )
234+
make install
235+
cd ..
236+
# Produces a self-contained wheel under mooncake-wheel/dist/.
237+
bash scripts/build_wheel.sh "$MOONCAKE_PYVER"
238+
pip install --force-reinstall mooncake-wheel/dist/*.whl )
226239
python3 -c "from mooncake.store import MooncakeDistributedStore" >/dev/null
227240
fi
228241

configs/amd-master.yaml

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -558,14 +558,15 @@ kimik2.7-fp4-mi355x-vllm-agentic:
558558
multinode: false
559559
scenarios:
560560
agentic-coding:
561-
# TP8 900s smoke to break the TP4 GPU-KV-capacity wall at higher concurrency.
562-
# At TP4 the ~121 GiB KV pool held only ~12 resident ~90k-token contexts, so
563-
# throughput plateaued (~160 tok/s) and preemption thrashed from conc16 up.
564-
# TP8 doubles GPUs -> ~2x KV pool + faster per-token decode.
561+
# TP8, 3600s. Compares three KV tiers on the agentic trace. At TP8 the
562+
# GPU-only (none) path collapses under load (KV-capacity thrash), so it is
563+
# only swept at low conc [4,8] where it is viable; the dram offload backends
564+
# (LMCache, Mooncake) carry the high-conc [16,32,48] range where they hold
565+
# throughput by keeping the GPU KV pool from thrashing.
565566
- dram-utilization: 0.80
566-
duration: 900
567+
duration: 3600
567568
search-space:
568-
- { tp: 8, kv-offloading: none, conc-list: [16, 32, 48] }
569+
- { tp: 8, kv-offloading: none, conc-list: [4, 8] }
569570
- { tp: 8, kv-offloading: dram, kv-offload-backend: { name: lmcache, version: "aaf7c0d3" }, conc-list: [16, 32, 48] }
570571
- { tp: 8, kv-offloading: dram, kv-offload-backend: { name: mooncake, version: "source-main" }, conc-list: [16, 32, 48] }
571572

configs/smoke-kimik2.7-mooncake.yaml

Lines changed: 0 additions & 18 deletions
This file was deleted.

0 commit comments

Comments
 (0)