Skip to content

Commit 1f57f08

Browse files
seungrokjclaude
andcommitted
fix(agentic-mtp): revert mem_fraction_static override, narrow sweep to c48
Disable the 0.75 mem_fraction_static override that caused a 6x KV cache reduction (5.1M → 873K tokens), falling back to the default 0.90. Narrow the sweep to DPA c48 only for a focused retest. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 38f33b2 commit 1f57f08

2 files changed

Lines changed: 5 additions & 3 deletions

File tree

benchmarks/single_node/agentic/dsv4_fp4_mi355x_sglang_mtp.sh

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,8 @@ if [ "$DP_ATTENTION" = "true" ]; then
8282
USE_SGLANG_ROUTER=true
8383
# DPA + MTP needs additional runtime headroom for speculative decode and
8484
# communication buffers beyond SGLang's static KV pool.
85-
MEM_FRACTION_STATIC=0.75
85+
# 0719
86+
#MEM_FRACTION_STATIC=0.75
8687
export AIPERF_HTTP_X_SMG_ROUTING_KEY_FROM_CORRELATION_ID=true
8788
SGLANG_BACKEND_PORT=$((PORT + 1))
8889
SGLANG_ROUTER_METRICS_PORT=$((PORT + 10000))

configs/amd-master.yaml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1563,8 +1563,9 @@ dsv4-fp4-mi355x-sglang-agentic-mtp:
15631563
agentic-coding:
15641564
- dram-utilization: 0.80
15651565
search-space:
1566-
- { tp: 8, kv-offloading: none, conc-list: [4, 8], spec-decoding: mtp }
1567-
- { tp: 8, dp-attn: true, kv-offloading: none, conc-list: [16, 32, 48, 52], spec-decoding: mtp }
1566+
#- { tp: 8, kv-offloading: none, conc-list: [4, 8], spec-decoding: mtp }
1567+
#- { tp: 8, dp-attn: true, kv-offloading: none, conc-list: [16, 32, 48, 52], spec-decoding: mtp }
1568+
- { tp: 8, dp-attn: true, kv-offloading: none, conc-list: [48], spec-decoding: mtp }
15681569

15691570
# MiniMax-M3 MXFP8 MI355X recipe:
15701571
# https://github.com/vllm-project/recipes/commit/2a3728ed9892debfd767a72a58ebc90b33f186e5

0 commit comments

Comments
 (0)