File tree Expand file tree Collapse file tree
Expand file tree Collapse file tree Original file line number Diff line number Diff line change @@ -68,7 +68,7 @@ The same pattern works with any server that speaks HTTP. llama.cpp's `llama serv
6868
6969``` bash
7070>>> hf jobs run --detach --expose 8080 --flavor a10g-small -s HF_TOKEN \
71- ... ghcr.io/ggml-org/llama.cpp:full -cuda -- \
71+ ... ghcr.io/ggml-org/llama.cpp:server -cuda -- \
7272... /app/llama serve -hf ggml-org/gemma-4-E4B-it-GGUF \
7373... --host 0.0.0.0 --port 8080 -ngl 99 \
7474... --temp 1.0 --top-p 0.95 --top-k 64
@@ -82,7 +82,7 @@ The `--` separates the job's command from `hf jobs run`'s own options — needed
8282> ``` bash
8383> >>> hf jobs run --detach --expose 8080 --flavor a10g-small -s HF_TOKEN \
8484> ... -v hf://ggml-org/gemma-4-E4B-it-GGUF:/model:ro \
85- > ... ghcr.io/ggml-org/llama.cpp:full -cuda -- \
85+ > ... ghcr.io/ggml-org/llama.cpp:server -cuda -- \
8686> ... /app/llama serve --model /model/gemma-4-E4B-it-Q4_K_M.gguf \
8787> ... --host 0.0.0.0 --port 8080 -ngl 99
8888> ` ` `
You can’t perform that action at this time.
0 commit comments