Skip to content

Commit 172ad96

Browse files
committed
fix(gb300): reserve TP4 aggregate memory headroom
Lower gpu-memory-utilization from 0.95 to 0.94 after TP4 aggregate c1 and c8 both ran out of memory during FP4 MoE execution. 中文:将 gpu-memory-utilization 从 0.95 降至 0.94,为 GB300 TP4 聚合式 worker 预留显存余量;此前 c1 和 c8 均在执行 FP4 MoE 时发生显存不足。
1 parent 303d1ef commit 172ad96

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

benchmarks/multi_node/srt-slurm-recipes/vllm/deepseek-v4/agentic/agg-gb300-tp4-agentic.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -115,7 +115,7 @@ backend:
115115
block-size: 256
116116
compilation-config: '{"cudagraph_mode":"FULL_DECODE_ONLY","mode":0}'
117117
max-cudagraph-capture-size: 32
118-
gpu-memory-utilization: 0.95
118+
gpu-memory-utilization: 0.94
119119
stream-interval: 10
120120
no-disable-hybrid-kv-cache-manager: true
121121
tokenizer-mode: deepseek_v4

0 commit comments

Comments
 (0)