Motivation
rocm serve --engine lemonade <owner/repo:variant> serves a Hugging Face GGUF under its canonical name. Today a variant is required — a bare owner/repo, or a partial/ambiguous variant like :Q4 that matches several quantizations, is rejected with an actionable error rather than guessed (to avoid silently serving the wrong quantization). See PR #80.
A nicer UX would be to enumerate the available variants and let the user pick when the variant is missing or ambiguous, instead of erroring.
Proposed approach
- Enumerate a repo's GGUF variants without downloading (Lemonade exposes a variants endpoint used by its own
pull flow).
- In the foreground/interactive serve path (which has a TTY, during the serve-plan phase in
apps/rocm), present the variants and let the user choose, then proceed with the chosen owner/repo:variant.
- Keep the current non-interactive behavior as the fallback: with no TTY (managed/background serves, scripts, CI), list the available variants in the error message so the user can specify one.
Notes / constraints
- Interactive selection only applies to the foreground path; managed/background serves must keep the clear error.
- Belongs in the
apps/rocm foreground layer (TTY + serve plan), not the engine's detached serve_http.
- Invariant to preserve: never silently pick a variant.
Motivation
rocm serve --engine lemonade <owner/repo:variant>serves a Hugging Face GGUF under its canonical name. Today a variant is required — a bareowner/repo, or a partial/ambiguous variant like:Q4that matches several quantizations, is rejected with an actionable error rather than guessed (to avoid silently serving the wrong quantization). See PR #80.A nicer UX would be to enumerate the available variants and let the user pick when the variant is missing or ambiguous, instead of erroring.
Proposed approach
pullflow).apps/rocm), present the variants and let the user choose, then proceed with the chosenowner/repo:variant.Notes / constraints
apps/rocmforeground layer (TTY + serve plan), not the engine's detachedserve_http.