Skip to content

Commit 50b5e6d

Browse files
authored
feat(serve): detach from an attached serve session with Ctrl-D (#85)
* feat(serve): detach from an attached serve session with Ctrl-D `rocm serve --verbose`/`--foreground` ran the engine in-process, so the only way out was Ctrl-C, which killed the model. The attached path now spawns the same detached managed child the background path uses and tails its log, so the server outlives the session: Ctrl-D detaches and leaves it running (manage it later with `rocm services`), while Ctrl-C stops it. Split the engine spawn out of start_managed_service so the attached path streams startup logs live instead of blocking on the readiness wait. Signed-off-by: Eugene Volen <Eugene.Volen@amd.com> * fix(serve): supervise and harden the attached detach path Address review on the attached serve path: - Ensure the supervisor daemon is running after spawning the attached child, so a Ctrl-D-detached server gets the same health-check, auto-recovery, record reconciliation, and dashboard metrics a background-managed server gets. - Enter raw mode before spawning the key reader thread, closing a window where a terminal Ctrl-C would kill the CLI with no detach/stop message. - Discover AppPaths once and thread it into spawn_managed_engine_child instead of rediscovering it per call. Signed-off-by: Eugene Volen <Eugene.Volen@amd.com> --------- Signed-off-by: Eugene Volen <Eugene.Volen@amd.com>
1 parent 800785e commit 50b5e6d

3 files changed

Lines changed: 374 additions & 67 deletions

File tree

README.md

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -190,10 +190,13 @@ quick smoke test (time to first token and approximate tokens/sec). Control
190190
returns to your shell with the server still running; manage it later with `rocm
191191
services` (below).
192192

193-
`--verbose` (or `--foreground`) instead attaches the server to the current
193+
`--verbose` (or `--foreground`) instead attaches to the server in the current
194194
terminal and streams every engine log line — use it to debug a startup problem.
195-
`--managed` is the explicit form of the default background behavior.
196-
`--no-smoke-test` skips the post-startup inference probe.
195+
The server still runs as a managed background process, so while streaming you can
196+
press **Ctrl-D to detach** — the log stream stops, your shell comes back, and the
197+
server keeps running (manage it afterward with `rocm services`). Press **Ctrl-C**
198+
to stop the server instead. `--managed` is the explicit form of the default
199+
background behavior. `--no-smoke-test` skips the post-startup inference probe.
197200

198201
Use full HuggingFace model IDs (e.g., `Qwen/Qwen2.5-1.5B-Instruct`) for
199202
reliable cross-engine compatibility. Short aliases from `rocm model` may not

0 commit comments

Comments
 (0)