Commit cad45fc
Use
* Use `llama serve` in Jobs serving guide (llama.cpp unified CLI)
Switches the llama.cpp section of the Jobs serving guide from the
standalone `llama-server` binary to the unified `llama serve` command,
per review feedback on #2554.
`llama serve` and `llama-server` are the same server (the unified
`llama` CLI, ggml-org/llama.cpp#23875). The unified binary ships in the
`full-cuda` image; the `server-cuda` image only contains
`/app/llama-server`, so the image changes too.
Tested on live Jobs (full-cuda + a10g-small): both the `-hf` and the
volume-mounted `--model` forms reached ready and returned completions
through the exposed-port proxy.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Use lighter server-cuda image now that llama ships there (ggml-org/llama.cpp#25035)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>llama serve in Jobs serving guide (llama.cpp unified CLI) (#2593)1 parent 8a8282a commit cad45fc
1 file changed
Lines changed: 5 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
64 | 64 | | |
65 | 65 | | |
66 | 66 | | |
67 | | - | |
| 67 | + | |
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
71 | 71 | | |
72 | | - | |
| 72 | + | |
73 | 73 | | |
74 | 74 | | |
75 | 75 | | |
76 | 76 | | |
77 | | - | |
| 77 | + | |
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
| |||
83 | 83 | | |
84 | 84 | | |
85 | 85 | | |
86 | | - | |
| 86 | + | |
87 | 87 | | |
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
91 | 91 | | |
92 | 92 | | |
93 | | - | |
| 93 | + | |
94 | 94 | | |
95 | 95 | | |
96 | 96 | | |
| |||
0 commit comments