Skip to content

Commit 189ae51

Browse files
authored
[NVIDIA] Set EP_SIZE back to 1 for B200 dsr1 fp8 (#242)
* Set ep = 1 for b200 * Re-enable EP for fp4
1 parent 0ba7aa3 commit 189ae51

1 file changed

Lines changed: 3 additions & 3 deletions

File tree

.github/configs/nvidia-master.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -83,15 +83,15 @@ dsr1-fp8-b200-sglang:
8383
- isl: 1024
8484
osl: 1024
8585
search-space:
86-
- { tp: 8, ep: 8, conc-start: 4, conc-end: 64 }
86+
- { tp: 8, ep: 1, conc-start: 4, conc-end: 64 }
8787
- isl: 1024
8888
osl: 8192
8989
search-space:
90-
- { tp: 8, ep: 8, conc-start: 4, conc-end: 64 }
90+
- { tp: 8, ep: 1, conc-start: 4, conc-end: 64 }
9191
- isl: 8192
9292
osl: 1024
9393
search-space:
94-
- { tp: 8, ep: 8, conc-start: 4, conc-end: 64 }
94+
- { tp: 8, ep: 1, conc-start: 4, conc-end: 64 }
9595

9696
dsr1-fp8-b200-trt:
9797
image: nvcr.io#nvidia/tensorrt-llm/release:1.1.0rc2.post2

0 commit comments

Comments
 (0)