Skip to content

Commit 5baf0d5

Browse files
seungrokjclaude
andcommitted
fix(agentic-mtp): stop dividing CUDA_GRAPH_MAX_BS by TP for DPA
Remove the per-DP-rank division of CUDA_GRAPH_MAX_BS — keep it at CONC so decode batches up to the concurrency level stay on the CUDA graph fast path (was dropping from 52 to 7 with TP8). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c33e79f commit 5baf0d5

1 file changed

Lines changed: 1 addition & 3 deletions

File tree

benchmarks/single_node/agentic/dsv4_fp4_mi355x_sglang_mtp.sh

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -113,10 +113,8 @@ fi
113113
# internally. CUDA graph capture is per scheduler, so only its batch size is
114114
# divided across DP ranks.
115115
MAX_RUNNING_REQUESTS=$CONC
116+
# 0719
116117
CUDA_GRAPH_MAX_BS=$CONC
117-
if [ "$DP_ATTENTION" = "true" ]; then
118-
CUDA_GRAPH_MAX_BS=$(( (CUDA_GRAPH_MAX_BS + TP - 1) / TP ))
119-
fi
120118
[ "$CUDA_GRAPH_MAX_BS" -gt 128 ] && CUDA_GRAPH_MAX_BS=128
121119

122120
# Simulated acceptance-length (AL) settings.

0 commit comments

Comments
 (0)