Skip to content

Commit 87abb7e

Browse files
committed
agentx: keep prefix caching ON in kimik2.7 none cell
The none case previously passed --no-enable-prefix-caching, which disables the on-GPU prefix cache entirely, not just DRAM offload. That crippled the no-offload baseline (no reuse even within HBM) and made lmcache look artificially good. Leave prefix caching at vLLM's default (ON) so none is an honest GPU-only-KV baseline, apples-to-apples vs lmcache. Matches the kimik2.5 / dsv4 agentic recipes.
1 parent de4cff8 commit 87abb7e

1 file changed

Lines changed: 6 additions & 1 deletion

File tree

benchmarks/single_node/agentic/kimik2.7_fp4_mi355x.sh

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -130,7 +130,12 @@ fi
130130

131131
case "$OFFLOAD_MODE" in
132132
none)
133-
OFFLOAD_ARGS=(--no-enable-prefix-caching)
133+
# GPU-only KV, no DRAM offload. Leave prefix caching at vLLM's
134+
# default (ON) so this is an honest no-offload baseline that still
135+
# reuses shared prefixes on-GPU — apples-to-apples vs the lmcache
136+
# cell, which extends that same reuse into DRAM. (Matches the
137+
# kimik2.5 / dsv4 agentic recipes.)
138+
PREFIX_CACHE_ARGS=(--enable-prefix-caching)
134139
;;
135140
lmcache)
136141
unset VLLM_USE_SIMPLE_KV_OFFLOAD

0 commit comments

Comments
 (0)