Skip to content

Commit 21640b6

Browse files
rominfjuhovainio
andauthored
feat(model): curate the built-in catalog by hardware target (#84)
* feat(model): curate the built-in catalog by hardware target `rocm model` listed every built-in recipe flat, including tiny smoke/test paths, and did not say what the list was for. Turn it into a short curated list of current popular open-weight models, each sized to a quant that fits a single GPU (multi-GPU serving is not supported). - Author the built-in list as an embedded JSON file using the same schema as a signed recipe index, so the offline default and hosted indexes share one format and the catalog is easy to curate. A unit test parses and validates it so a malformed catalog cannot ship. - Feature current popular models under canonical Hugging Face ids with the quant that fits one GPU: Qwen3.6-35B-A3B and Gemma 4 26B-A4B as Q4_K_M GGUF on Strix Halo (served via Lemonade's owner/repo:variant form), and Qwen3.6-27B and Gemma 4 31B as BF16 on one MI300X via vLLM. - `rocm model` shows only the featured list for the built-in catalog, grouped by hardware target, with a note that any compatible Hugging Face model can be served. A configured recipe index is shown in full and named in the header. The default assistant, smoke/test, and superseded recipes stay resolvable for `rocm serve` and appear under `--verbose`. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com> * feat(model): move featured/platforms to catalog JSON - Add `featured` boolean field to ModelRecipeRecord; mark the four curated models in model_catalog.json. model_recipe_featured() now reads the field directly, eliminating the hardcoded FEATURED_MODEL_IDS Rust constant that had to be kept in sync by hand. - Add `platforms` array to ModelRecipeIndexDocument with label, engines, and gfx_families per platform. MODEL_CATALOG_PLATFORMS constant and the engine-name match arm are removed; model_recipe_target_platform_label() looks up the recipe's engine against the catalog's platform list instead. Signed indexes that omit platforms fall back to the built-in definitions. - rocm model highlights the matching platform with "← your GPU" by probing detect_host_gfx_target() + normalize_therock_family() and matching against each platform's gfx_families list. - Footer "These are recommendations..." is now only emitted for the built-in catalog, not for operator-configured SignedIndex registries. - Remove dangling gpt-oss-120b manual_alternative entries (no recipe in the catalog resolves that id). - Fix docstring: catalog validation is test-time, not build-time. Signed-off-by: Juho Vainio <juho.vainio@amd.com> * fix(model): clippy and fmt fixes Signed-off-by: Juho Vainio <juho.vainio@amd.com> * fix(model): fix remaining clippy warnings in main.rs Signed-off-by: Juho Vainio <juho.vainio@amd.com> * feat(model): expand platform list to Ryzen AI, Radeon, and Instinct Add three hardware platform entries to model_catalog.json replacing the previous two (Strix Halo / MI300X): - AMD Ryzen AI — Strix Halo (Lemonade / llama.cpp): gfx1151 - AMD Radeon — RX 7000/9000, Radeon PRO (Lemonade / llama.cpp): gfx110X-all, gfx120X-all - AMD Instinct — MI300X, MI350X, MI355X (vLLM): gfx906, gfx908, gfx90a, gfx94X-dcgpu, gfx950-dcgpu Lemonade recipes now appear under both Strix Halo and Radeon by matching per-platform engine lists directly rather than assigning each recipe to only the first matching platform. Signed-off-by: Juho Vainio <juho.vainio@amd.com> --------- Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com> Signed-off-by: Juho Vainio <juho.vainio@amd.com> Co-authored-by: Juho Vainio <juho.vainio@amd.com>
1 parent 9841b96 commit 21640b6

4 files changed

Lines changed: 807 additions & 305 deletions

File tree

README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -325,6 +325,20 @@ Show recommended models and hardware compatibility:
325325
rocm model [--verbose]
326326
```
327327

328+
`rocm model` prints a curated catalog of popular open-weight models grouped by
329+
the hardware path they target — Strix Halo (Lemonade / llama.cpp) and MI300X
330+
(vLLM) — using canonical Hugging Face ids, and shows for each the quantization
331+
that fits a single GPU. Strix Halo entries use the `owner/repo:variant` GGUF form
332+
(e.g. `unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M`) that `rocm serve` needs; MI300X
333+
entries serve at BF16. This catalog ships inside the binary, so it is available
334+
offline; when a recipe index is configured instead, the header names it.
335+
`--verbose` also lists the other recipes (the default assistant and smoke/test
336+
paths) that `rocm serve` can still resolve but that are hidden from the list.
337+
338+
The catalog is only a starting point: you can serve any compatible Hugging Face
339+
model by passing its id to `rocm serve``owner/repo` for vLLM, or
340+
`owner/repo:<quant>` for a Lemonade GGUF.
341+
328342
Manage background servers started with `--managed`:
329343

330344
```

apps/rocm/src/main.rs

Lines changed: 176 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,13 @@ use rocm_core::{
2929
PERMISSIONS_MODE_FULL_ACCESS, RocmCliConfig, TELEMETRY_MODE_LOCAL, TELEMETRY_MODE_OFF,
3030
WatcherMode, append_audit_event, builtin_model_recipes, builtin_watcher, builtin_watchers,
3131
connect_tcp_stream, daemon_binary_path, default_engine_for_platform,
32-
default_interactive_shell_program, detect_host_gpu_summary, engine_binary_path,
33-
engine_plugin_dirs, format_host_port, format_http_base_url, generate_service_id,
34-
interactive_terminal, load_model_recipe_registry, load_recent_audit_events,
35-
load_recent_automation_events, load_recent_automation_proposals, managed_pip_cache_dir,
36-
managed_service_endpoint_model_ready, model_artifact_cache_status,
32+
default_interactive_shell_program, detect_host_gfx_target, detect_host_gpu_summary,
33+
engine_binary_path, engine_plugin_dirs, format_host_port, format_http_base_url,
34+
generate_service_id, interactive_terminal, load_model_recipe_registry,
35+
load_recent_audit_events, load_recent_automation_events, load_recent_automation_proposals,
36+
managed_pip_cache_dir, managed_service_endpoint_model_ready, model_artifact_cache_status,
37+
model_catalog_platforms, model_recipe_featured, model_recipe_target_platform_label,
38+
normalize_therock_family, platform_matches_gfx_family,
3739
preferred_serve_engine_for_host_gpu_summary, prepend_runtime_path, process_is_running,
3840
read_tcp_stream_to_string, resolve_builtin_model_recipe, resolve_model_recipe,
3941
runtime_install_root_is_protected, runtime_path_is_same_or_inside,
@@ -10678,27 +10680,82 @@ pub(crate) fn render_model_registry_text_with_context_and_host(
1067810680
_host_ram_gib: Option<f64>,
1067910681
) -> String {
1068010682
let mut output = String::new();
10681-
let _ = writeln!(output, "Local models");
1068210683
let registry = match load_model_recipe_registry() {
1068310684
Ok(registry) => registry,
1068410685
Err(error) => {
10685-
let _ = writeln!(output, " Model list is unavailable: {error}");
10686+
let _ = writeln!(output, "Recommended models unavailable: {error}");
1068610687
return output;
1068710688
}
1068810689
};
10689-
for recipe in &registry.recipes {
10690+
let _ = writeln!(output, "{}", model_catalog_header(&registry));
10691+
// Curated view: for the built-in catalog show only the featured short list;
10692+
// a configured index is already curated, so show all of it. Group either way
10693+
// by the hardware path each model targets.
10694+
let show_all = !matches!(registry.source, ModelRecipeRegistrySource::BuiltIn);
10695+
let visible = registry
10696+
.recipes
10697+
.iter()
10698+
.filter(|recipe| show_all || model_recipe_featured(recipe))
10699+
.collect::<Vec<_>>();
10700+
let platforms = model_catalog_platforms(&registry);
10701+
let host_gfx_family = detect_host_gfx_target()
10702+
.as_deref()
10703+
.and_then(normalize_therock_family);
10704+
for platform in &platforms {
10705+
let mut group = visible
10706+
.iter()
10707+
.filter(|recipe| {
10708+
recipe.preferred_engines.first().is_some_and(|engine| {
10709+
platform
10710+
.engines
10711+
.iter()
10712+
.any(|e| e.eq_ignore_ascii_case(engine))
10713+
})
10714+
})
10715+
.peekable();
10716+
if group.peek().is_none() {
10717+
continue;
10718+
}
10719+
let your_gpu = host_gfx_family
10720+
.as_deref()
10721+
.is_some_and(|family| platform_matches_gfx_family(platform, family));
10722+
let heading = if your_gpu {
10723+
format!("{} \u{2190} your GPU", platform.label)
10724+
} else {
10725+
platform.label.clone()
10726+
};
10727+
let _ = writeln!(output, "\n{heading}");
10728+
for recipe in group {
10729+
// Show the canonical Hugging Face id (the reliable serve target) and
10730+
// the quant that fits this hardware. Append a fit verdict only when the
10731+
// host GPU VRAM is known — otherwise every row would read "GPU fit
10732+
// unknown", which is noise.
10733+
let detail = recipe
10734+
.quantization
10735+
.clone()
10736+
.unwrap_or_else(|| model_recipe_memory_label(recipe));
10737+
let fit = if aggregate_gpu_vram_gib.is_some() {
10738+
format!(
10739+
" {}",
10740+
model_recipe_gpu_fit_label(recipe, aggregate_gpu_vram_gib)
10741+
)
10742+
} else {
10743+
String::new()
10744+
};
10745+
let _ = writeln!(output, " {} {}{}", recipe.canonical_model_id, detail, fit);
10746+
}
10747+
}
10748+
if matches!(registry.source, ModelRecipeRegistrySource::BuiltIn) {
1069010749
let _ = writeln!(
1069110750
output,
10692-
" {} {} {}",
10693-
recipe_display_ref(recipe),
10694-
model_recipe_memory_label(recipe),
10695-
model_recipe_gpu_fit_label(recipe, aggregate_gpu_vram_gib)
10751+
"\nThese are recommendations — you can serve any compatible Hugging Face model:\n \
10752+
rocm serve <owner/repo> # vLLM, e.g. Qwen/Qwen3.6-27B\n \
10753+
rocm serve <owner/repo>:<quant> # Lemonade GGUF, e.g. unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M\n\
10754+
\nUse `rocm model --verbose` for details."
1069610755
);
10756+
} else {
10757+
let _ = writeln!(output, "\nUse `rocm model --verbose` for details.");
1069710758
}
10698-
let _ = writeln!(
10699-
output,
10700-
"\nUse `rocm serve <model>` to start one. Use `rocm model --verbose` for details."
10701-
);
1070210759
output
1070310760
}
1070410761

@@ -10765,6 +10822,21 @@ pub(crate) fn render_model_registry_verbose_text_with_context_and_host(
1076510822
memory,
1076610823
engines
1076710824
);
10825+
let hidden_from_builtin = matches!(registry.source, ModelRecipeRegistrySource::BuiltIn)
10826+
&& !model_recipe_featured(recipe);
10827+
if hidden_from_builtin {
10828+
let _ = writeln!(
10829+
output,
10830+
" catalog: hidden (resolvable via rocm serve, not in the curated list)"
10831+
);
10832+
} else {
10833+
let platforms = model_catalog_platforms(&registry);
10834+
let _ = writeln!(
10835+
output,
10836+
" catalog: {}",
10837+
model_recipe_target_platform_label(recipe, &platforms)
10838+
);
10839+
}
1076810840
append_model_recipe_metadata_lines(&mut output, recipe, paths);
1076910841
append_model_host_ram_fit_lines(&mut output, recipe, host_ram_gib);
1077010842
append_model_fit_lines(&mut output, recipe, aggregate_gpu_vram_gib);
@@ -10780,6 +10852,23 @@ pub(crate) fn render_model_registry_verbose_text_with_context_and_host(
1078010852
output
1078110853
}
1078210854

10855+
/// The `rocm model` header. For the default built-in list it just names the
10856+
/// action; a configured recipe index instead advertises its provenance (the
10857+
/// only case where the source differs from the default and is worth surfacing).
10858+
fn model_catalog_header(registry: &ModelRecipeRegistry) -> String {
10859+
match &registry.source {
10860+
ModelRecipeRegistrySource::BuiltIn => {
10861+
"Recommended models — run one with `rocm serve <model>`".to_owned()
10862+
}
10863+
ModelRecipeRegistrySource::SignedIndex { index_path, .. } => {
10864+
format!(
10865+
"Recommended models — from recipe index {}",
10866+
index_path.display()
10867+
)
10868+
}
10869+
}
10870+
}
10871+
1078310872
#[allow(dead_code)]
1078410873
fn append_model_recipe_registry_source(output: &mut String, registry: &ModelRecipeRegistry) {
1078510874
match &registry.source {
@@ -21717,16 +21806,78 @@ VERSION_ID="41"
2171721806
fn render_model_registry_text_lists_builtin_recipes() {
2171821807
let rendered = render_model_registry_text_with_context_and_host(None, None, None);
2171921808

21720-
assert!(rendered.contains("Local models"));
21721-
assert!(rendered.contains("qwen3.5"));
21722-
assert!(rendered.contains("GPU fit unknown"));
21723-
assert!(rendered.contains("Use `rocm serve <model>`"));
21809+
// Clean header: what the list is + the one action, no implementation jargon.
21810+
assert!(rendered.contains("Recommended models — run one with `rocm serve <model>`"));
21811+
// Featured current models are grouped under their hardware target, each
21812+
// with the quant that fits a single GPU.
21813+
assert!(rendered.contains("AMD Ryzen AI — Strix Halo (Lemonade / llama.cpp)"));
21814+
assert!(rendered.contains("AMD Instinct — MI300X, MI350X, MI355X (vLLM)"));
21815+
// Strix Halo GGUF entries carry the servable owner/repo:variant ref.
21816+
assert!(rendered.contains("unsloth/Qwen3.6-35B-A3B-GGUF:Q4_K_M"));
21817+
assert!(rendered.contains("Q4_K_M GGUF"));
21818+
assert!(rendered.contains("Qwen/Qwen3.6-27B"));
21819+
assert!(rendered.contains("BF16"));
21820+
// Multi-GPU-only models are not featured (single-GPU serving only).
21821+
assert!(!rendered.contains("GLM-5.2"));
21822+
assert!(!rendered.contains("DeepSeek-V4-Flash"));
21823+
// Without a known host GPU, no misleading per-row fit verdict is shown.
21824+
assert!(!rendered.contains("GPU fit unknown"));
21825+
// Superseded / smoke / assistant recipes are hidden from the curated list.
21826+
assert!(!rendered.contains("tiny-gpt2"));
21827+
assert!(!rendered.contains("Qwen/Qwen3.5-4B"));
21828+
assert!(!rendered.contains("Qwen3-4B-Instruct-2507-GGUF"));
21829+
// Disclaimer: the list is a starting point; any compatible HF model works.
21830+
assert!(rendered.contains("you can serve any compatible Hugging Face model"));
21831+
assert!(rendered.contains("rocm serve <owner/repo>:<quant>"));
2172421832
assert!(rendered.contains("rocm model --verbose"));
2172521833
assert!(!rendered.contains("recommended_system_ram:"));
2172621834
assert!(!rendered.contains("engine_support:"));
2172721835
assert!(!rendered.contains("artifact_check:"));
2172821836
}
2172921837

21838+
#[test]
21839+
fn render_model_registry_text_shows_fit_when_gpu_known() {
21840+
// With a known aggregate VRAM, each row gains a concrete fit verdict.
21841+
// 60 GiB fits Qwen3.6-27B (~54) but not Gemma-4-31B (~62).
21842+
let rendered = render_model_registry_text_with_context_and_host(None, Some(60.0), None);
21843+
21844+
assert!(rendered.contains("fits this GPU"));
21845+
assert!(rendered.contains("needs a larger GPU"));
21846+
assert!(!rendered.contains("GPU fit unknown"));
21847+
}
21848+
21849+
#[test]
21850+
fn model_catalog_header_only_names_source_for_configured_index() {
21851+
// Default built-in list: no implementation jargon, just the action.
21852+
let builtin = ModelRecipeRegistry {
21853+
recipes: Vec::new(),
21854+
platforms: Vec::new(),
21855+
source: ModelRecipeRegistrySource::BuiltIn,
21856+
};
21857+
let header = model_catalog_header(&builtin);
21858+
assert_eq!(
21859+
header,
21860+
"Recommended models — run one with `rocm serve <model>`"
21861+
);
21862+
assert!(!header.contains("built-in"));
21863+
assert!(!header.contains("fallback"));
21864+
21865+
// A configured index is the only case worth advertising provenance for.
21866+
let configured = ModelRecipeRegistry {
21867+
recipes: Vec::new(),
21868+
platforms: Vec::new(),
21869+
source: ModelRecipeRegistrySource::SignedIndex {
21870+
index_path: PathBuf::from("/etc/rocm/recipes.json"),
21871+
signature_path: PathBuf::from("/etc/rocm/recipes.json.sig"),
21872+
public_key_path: PathBuf::from("/etc/rocm/recipes.pub"),
21873+
},
21874+
};
21875+
assert_eq!(
21876+
model_catalog_header(&configured),
21877+
"Recommended models — from recipe index /etc/rocm/recipes.json"
21878+
);
21879+
}
21880+
2173021881
#[test]
2173121882
fn render_model_registry_verbose_text_keeps_diagnostics() {
2173221883
let rendered = render_model_registry_verbose_text_with_context_and_host(None, None, None);
@@ -21739,6 +21890,11 @@ VERSION_ID="41"
2173921890
assert!(rendered.contains("engine_support:"));
2174021891
assert!(rendered.contains("engine_action: use /engine install <engine>"));
2174121892
assert!(rendered.contains("source: built-in recipe registry"));
21893+
// Verbose keeps every recipe (including hidden ones) and annotates each
21894+
// with its curated catalog placement.
21895+
assert!(rendered.contains("catalog: AMD Instinct — MI300X, MI350X, MI355X (vLLM)"));
21896+
assert!(rendered.contains("catalog: hidden (resolvable via rocm serve"));
21897+
assert!(rendered.contains("sshleifer/tiny-gpt2"));
2174221898
}
2174321899

2174421900
#[test]

0 commit comments

Comments
 (0)