Skip to content

Commit d93b9c3

Browse files
committed
[AMD] improve dsr1 fp4 disagg
> Co-authored-by: billishyahao <bill.he@amd.com> > Co-authored-by: Duyi-Wang <duyi.wang@amd.com>
1 parent c9798a7 commit d93b9c3

1 file changed

Lines changed: 23 additions & 4 deletions

File tree

.github/configs/amd-master.yaml

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1862,7 +1862,7 @@ dsr1-fp4-mi355x-sglang-disagg:
18621862
- "DECODE_MTP_SIZE=0"
18631863

18641864
dsr1-fp4-mi355x-sglang-disagg-mtp:
1865-
image: lmsysorg/sglang-rocm:v0.5.12-rocm720-mi35x-20260519
1865+
image: rocm/sgl-dev:v0.5.12.post1-rocm720-mi35x-20260526-sync
18661866
model: amd/DeepSeek-R1-0528-MXFP4-v2
18671867
model-prefix: dsr1
18681868
runner: mi355x-disagg
@@ -2030,7 +2030,7 @@ dsr1-fp4-mi355x-sglang-disagg-mtp:
20302030
dp-attn: false
20312031
additional-settings:
20322032
- "DECODE_NODES=2"
2033-
- "DECODE_MTP_SIZE=2"
2033+
- "DECODE_MTP_SIZE=3"
20342034

20352035
# 1*DEP8 + 1*DEP8
20362036
- spec-decoding: "mtp"
@@ -2049,7 +2049,7 @@ dsr1-fp4-mi355x-sglang-disagg-mtp:
20492049
dp-attn: true
20502050
additional-settings:
20512051
- "DECODE_NODES=1"
2052-
- "DECODE_MTP_SIZE=1"
2052+
- "DECODE_MTP_SIZE=3"
20532053

20542054
# 1*DEP8 + 1*DEP8
20552055
- spec-decoding: "mtp"
@@ -2068,7 +2068,26 @@ dsr1-fp4-mi355x-sglang-disagg-mtp:
20682068
dp-attn: true
20692069
additional-settings:
20702070
- "DECODE_NODES=1"
2071-
- "DECODE_MTP_SIZE=1"
2071+
- "DECODE_MTP_SIZE=3"
2072+
2073+
# 1*DEP8 + 1*DEP8
2074+
- spec-decoding: "mtp"
2075+
conc-list: [ 64, 128 ]
2076+
prefill:
2077+
num-worker: 1
2078+
tp: 8
2079+
ep: 8
2080+
dp-attn: true
2081+
additional-settings:
2082+
- "PREFILL_NODES=1"
2083+
decode:
2084+
num-worker: 1
2085+
tp: 8
2086+
ep: 8
2087+
dp-attn: true
2088+
additional-settings:
2089+
- "DECODE_NODES=1"
2090+
- "DECODE_MTP_SIZE=3"
20722091

20732092
# 2*DEP8 + 1*DEP8
20742093
- spec-decoding: "mtp"

0 commit comments

Comments
 (0)