Skip to content

Commit 3f9c7ac

Browse files
davanstrienclaude
andcommitted
Use lighter server-cuda image now that llama ships there (ggml-org/llama.cpp#25035)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 95025b4 commit 3f9c7ac

1 file changed

Lines changed: 2 additions & 2 deletions

File tree

docs/hub/jobs-serving.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ The same pattern works with any server that speaks HTTP. llama.cpp's `llama serv
6868

6969
```bash
7070
>>> hf jobs run --detach --expose 8080 --flavor a10g-small -s HF_TOKEN \
71-
... ghcr.io/ggml-org/llama.cpp:full-cuda -- \
71+
... ghcr.io/ggml-org/llama.cpp:server-cuda -- \
7272
... /app/llama serve -hf ggml-org/gemma-4-E4B-it-GGUF \
7373
... --host 0.0.0.0 --port 8080 -ngl 99 \
7474
... --temp 1.0 --top-p 0.95 --top-k 64
@@ -82,7 +82,7 @@ The `--` separates the job's command from `hf jobs run`'s own options — needed
8282
> ```bash
8383
> >>> hf jobs run --detach --expose 8080 --flavor a10g-small -s HF_TOKEN \
8484
> ... -v hf://ggml-org/gemma-4-E4B-it-GGUF:/model:ro \
85-
> ... ghcr.io/ggml-org/llama.cpp:full-cuda -- \
85+
> ... ghcr.io/ggml-org/llama.cpp:server-cuda -- \
8686
> ... /app/llama serve --model /model/gemma-4-E4B-it-Q4_K_M.gguf \
8787
> ... --host 0.0.0.0 --port 8080 -ngl 99
8888
> ```

0 commit comments

Comments
 (0)