Skip to content

Commit 303d1ef

Browse files
committed
fix: tp oom
1 parent 870df10 commit 303d1ef

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

benchmarks/multi_node/srt-slurm-recipes/vllm/deepseek-v4/agentic/agg-gb300-tp4-agentic.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,7 @@ backend:
109109
attention-config: '{"backend":"FLASHINFER_MLA_SPARSE_DSV4","use_prefill_query_quantization":true,"use_fp4_indexer_cache":true}'
110110
max-model-len: 1048576
111111
max-num-seqs: 32
112-
max-num-batched-tokens: 32768
112+
max-num-batched-tokens: 16384
113113
trust-remote-code: true
114114
no-enable-flashinfer-autotune: true
115115
block-size: 256

0 commit comments

Comments
 (0)