Skip to content

Commit 1f98965

Browse files
authored
Update dsr1_fp4_mi355x_docker.sh
1 parent 44aec22 commit 1f98965

1 file changed

Lines changed: 3 additions & 1 deletion

File tree

benchmarks/dsr1_fp4_mi355x_docker.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,9 @@ python3 -m sglang.launch_server --model-path=$MODEL --trust-remote-code \
3030
--disable-radix-cache \
3131
--num-continuous-decode-steps=4 \
3232
--max-prefill-tokens=$PREFILL_SIZE \
33-
--cuda-graph-max-bs=128 > $SERVER_LOG 2>&1 &
33+
--cuda-graph-max-bs=128 \
34+
--attention-backend aiter \
35+
--kv-cache-dtype fp8_e4m3 > $SERVER_LOG 2>&1 &
3436

3537
SERVER_PID=$!
3638

0 commit comments

Comments
 (0)