Skip to content

Commit ee7606f

Browse files
committed
simplify installation
1 parent 64afd71 commit ee7606f

1 file changed

Lines changed: 12 additions & 4 deletions

File tree

.github/workflows/vllm.yml

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -73,15 +73,23 @@ jobs:
7373
- name: Install Hatch
7474
run: pip install hatch
7575

76-
- name: Install vLLM and start server
76+
- name: Install vLLM (CPU)
7777
run: |
78-
pip install vllm
79-
78+
export VLLM_VERSION=$(curl -s https://api.github.com/repos/vllm-project/vllm/releases/latest | jq -r .tag_name | sed 's/^v//')
79+
echo "Installing vLLM ${VLLM_VERSION} (CPU)"
80+
hatch shell
81+
uvpip install "vllm==${VLLM_VERSION}" --extra-index-url https://wheels.vllm.ai/releases/${VLLM_VERSION}/cpu/
82+
83+
- name: Start vLLM server
84+
env:
85+
VLLM_CPU_KVCACHE_SPACE: "4"
86+
run: |
87+
hatch shell
8088
nohup vllm serve ${{ env.VLLM_MODEL }} \
8189
--reasoning-parser qwen3 \
8290
--max-model-len 1024 \
8391
--enforce-eager \
84-
--dtype half \
92+
--dtype bfloat16 \
8593
--enable-auto-tool-choice \
8694
--tool-call-parser hermes \
8795
> vllm.log 2>&1 &

0 commit comments

Comments
 (0)