Skip to content

Commit c33e79f

Browse files
seungrokjclaude
andcommitted
fix(agentic-mtp): revert mem_fraction_static override, narrow sweep to c48
Disable the 0.75 mem_fraction_static override that caused a 6x KV cache reduction (5.1M → 873K tokens), falling back to the default 0.90. Narrow the sweep to DPA c48 only for a focused retest. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 6b8619f commit c33e79f

2 files changed

Lines changed: 5 additions & 3 deletions

File tree

benchmarks/single_node/agentic/dsv4_fp4_mi355x_sglang_mtp.sh

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,8 @@ if [ "$DP_ATTENTION" = "true" ]; then
8282
USE_SGLANG_ROUTER=true
8383
# DPA + MTP needs additional runtime headroom for speculative decode and
8484
# communication buffers beyond SGLang's static KV pool.
85-
MEM_FRACTION_STATIC=0.75
85+
# 0719
86+
#MEM_FRACTION_STATIC=0.75
8687
export AIPERF_HTTP_X_SMG_ROUTING_KEY_FROM_CORRELATION_ID=true
8788
SGLANG_BACKEND_PORT=$((PORT + 1))
8889
SGLANG_ROUTER_METRICS_PORT=$((PORT + 10000))

configs/amd-master.yaml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1713,8 +1713,9 @@ dsv4-fp4-mi355x-sglang-agentic-mtp:
17131713
agentic-coding:
17141714
- dram-utilization: 0.80
17151715
search-space:
1716-
- { tp: 8, kv-offloading: none, conc-list: [4, 8], spec-decoding: mtp }
1717-
- { tp: 8, dp-attn: true, kv-offloading: none, conc-list: [16, 32, 48, 52], spec-decoding: mtp }
1716+
#- { tp: 8, kv-offloading: none, conc-list: [4, 8], spec-decoding: mtp }
1717+
#- { tp: 8, dp-attn: true, kv-offloading: none, conc-list: [16, 32, 48, 52], spec-decoding: mtp }
1718+
- { tp: 8, dp-attn: true, kv-offloading: none, conc-list: [48], spec-decoding: mtp }
17181719

17191720
# MiniMax-M3 MXFP8 MI355X recipe:
17201721
# https://github.com/vllm-project/recipes/commit/2a3728ed9892debfd767a72a58ebc90b33f186e5

0 commit comments

Comments
 (0)