Skip to content

Commit b26b54d

Browse files
csg-pr-botDev Agent
andauthored
feat(inference): add data-parallel-size config for vllm (#975)
Co-authored-by: Dev Agent <dev-agent@example.com>
1 parent 2c3fe8e commit b26b54d

1 file changed

Lines changed: 10 additions & 0 deletions

File tree

configs/inference/vllm.json

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -235,6 +235,11 @@
235235
"value": "0",
236236
"format": "--cpu-offload-gb %s"
237237
},
238+
{
239+
"name": "data-parallel-size",
240+
"value": "1",
241+
"format": "--data-parallel-size %s"
242+
},
238243
{
239244
"name": "pipeline-parallel-size",
240245
"value": "1",
@@ -285,6 +290,11 @@
285290
"value": "disable",
286291
"format": "--enable-auto-tool-choice"
287292
},
293+
{
294+
"name": "enable-expert-parallel",
295+
"value": "disable",
296+
"format": "--enable-expert-parallel"
297+
},
288298
{
289299
"name": "limit-mm-per-prompt",
290300
"value": "image=5,video=5",

0 commit comments

Comments
 (0)