feat(gallery): expose model variants for selection over API, CLI and MCP

A gallery entry may carry `variants:`, alternative builds of the same model.
Selection already worked at install time, but nothing could see what an entry
offered or ask for a specific build, so the feature was undrivable.

Listing: `GET /api/models` now reports `variants` and `auto_variant` for the
entries that declare variants. Each variant carries its resolved backend, its
measured size and whether it fits this host. `auto_variant` is what installing
without a choice would pick right now.

The new gallery.DescribeVariants runs the same variantOptions + SelectVariant
pass the installer runs, so the reported default cannot drift from what
installing actually does, and HostResolveEnv is extracted so both derive the
host and share pkg/vram's probe cache from one place.

Performance: an entry that declares no variants returns early without touching
the probe, so the ~1280 ordinary entries cost exactly what they cost before.

Selection: `variant` is accepted on POST /models/apply, as a query param on
POST /api/models/install/:id, on the gallery apply file/string request, as
`local-ai models install --variant`, and as a parameter on the install_model
MCP tool (both the httpapi and inproc clients). Empty means auto-select.

An unknown variant name now fails the install naming what was requested. This
closes a real hole: an entry declaring no variants short-circuits before
selection runs, so a requested variant was previously dropped silently and the
install reported success.

startup.InstallModels ends in a variadic model list, so install options could
not be appended to it; InstallModelsWithOptions is added alongside and
InstallModels delegates to it. No caller signature changed.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

Author: mudler

.agents/adding-gallery-models.md

@@ -91,6 +91,46 @@ To add a variant (e.g., different quantization), use YAML merge:
       uri: huggingface://<gguf-org>/<gguf-repo>/<filename>-Q8_0.gguf
 ```
 
+## Offering several builds of one model (`variants`)
+
+When the same model is published in more than one quantization, or is also
+servable by another engine, add each build as its own ordinary gallery entry and
+then point one of them at the others with `variants`:
+
+```yaml
+- !!merge <<: *chatml
+  name: "nanbeige4.1-3b-q4"
+  # ... the usual urls / overrides / files for the Q4 build ...
+  variants:
+    - model: nanbeige4.1-3b-q8
+```
+
+Rules:
+
+- The declaring entry is a **complete, normal entry**. It keeps its own
+  `files`/`overrides` and stays installable on every host and by every older
+  LocalAI release, which simply ignore `variants`.
+- A variant references another gallery entry **by name**. That entry must exist
+  and must not declare `variants` of its own.
+- **Order carries no meaning.** Do not try to encode a preference; write the
+  list in whatever order reads best.
+- **Do not describe hardware.** At install time LocalAI drops variants whose
+  backend cannot run on the host, drops those that do not fit available memory,
+  and installs the largest survivor, falling back to the declaring entry's own
+  build. Sizes are measured live from the weights and cached, so nothing has to
+  be written down.
+- `min_memory` on a single variant (e.g. `min_memory: 20GiB`) overrides the
+  measured size and suppresses the measurement for that variant. Use it only
+  when you have measured a real load and know the estimate is wrong; most
+  variants should not carry it.
+
+Users can override the automatic choice with `variant` on `POST /models/apply`,
+`local-ai models install --variant`, or the `install_model` MCP tool. See
+`docs/content/features/model-gallery.md`.
+
+The gallery lint specs live in `core/gallery`, so run that suite after adding a
+`variants` list.
+
 ## Available template configs
 
 Look at existing `.yaml` files in `gallery/` to find the right prompt template for your model architecture:

core/cli/models.go

@@ -39,6 +39,7 @@ type ModelsInstall struct {
 	RequireBackendIntegrity  bool     `env:"LOCALAI_REQUIRE_BACKEND_INTEGRITY,REQUIRE_BACKEND_INTEGRITY" help:"If true, reject backend installs without a configured signature verification policy (OCI URIs) or SHA256 (tarball/HTTP URIs)." group:"hardening" default:"false"`
 	AutoloadBackendGalleries bool     `env:"LOCALAI_AUTOLOAD_BACKEND_GALLERIES" help:"If true, automatically loads backend galleries" group:"backends" default:"true"`
 	ModelArgs                []string `arg:"" optional:"" name:"models" help:"Model configuration URLs to load"`
+	Variant                  string   `name:"variant" help:"Install a specific variant of a gallery entry that declares them, by the variant's model name. Leave unset to let LocalAI auto-select the largest build this machine can run." group:"models"`
 
 	ModelsCMDFlags `embed:""`
 }
@@ -147,7 +148,11 @@ func (mi *ModelsInstall) Run(ctx *cliContext.Context) error {
 		}
 
 		modelLoader := model.NewModelLoader(systemState)
-		err = startup.InstallModels(context.Background(), galleryService, galleries, backendGalleries, systemState, modelLoader, !mi.DisablePredownloadScan, mi.AutoloadBackendGalleries, mi.RequireBackendIntegrity, progressCallback, modelName)
+		var installOptions []gallery.InstallOption
+		if mi.Variant != "" {
+			installOptions = append(installOptions, gallery.WithVariant(mi.Variant))
+		}
+		err = startup.InstallModelsWithOptions(context.Background(), galleryService, galleries, backendGalleries, systemState, modelLoader, !mi.DisablePredownloadScan, mi.AutoloadBackendGalleries, mi.RequireBackendIntegrity, progressCallback, installOptions, modelName)
 		if err != nil {
 			return err
 		}

core/gallery/describe_variants.go

@@ -0,0 +1,85 @@
+package gallery
+
+// VariantView is one selectable build of an entry, as a client sees it.
+//
+// It is the read-only mirror of what selection decides at install time, so a
+// picker can show the user the same facts the installer acts on rather than a
+// second, independently-derived opinion that could disagree with it.
+type VariantView struct {
+	// Model is the name to send back as the install request's `variant`.
+	Model string `json:"model"`
+	// Backend is the engine the referenced entry resolves to. A client renders
+	// it, and it is also the reason Fits may be false on a host whose memory
+	// would be ample.
+	Backend string `json:"backend"`
+	// MemoryBytes is the measured footprint, or 0 when it could not be
+	// determined. Zero is unknown, never "needs nothing", so a client must
+	// render it as unknown rather than as a free option.
+	MemoryBytes uint64 `json:"memory_bytes,omitempty"`
+	// Fits reports whether auto-selection would consider this variant on this
+	// host: its backend can run here and its known footprint is within budget.
+	// An unknown footprint counts as fitting, exactly as selection treats it.
+	Fits bool `json:"fits"`
+	// IsBase marks the entry's own payload. It is always installable and is
+	// what auto-selection falls back to, which is why it is listed alongside
+	// the declared variants rather than hidden.
+	IsBase bool `json:"is_base"`
+}
+
+// EntryVariants is the variant surface of a single gallery entry.
+type EntryVariants struct {
+	Variants []VariantView `json:"variants"`
+	// AutoSelected is the variant auto-selection would install on this host
+	// right now. Clients show it as the default choice.
+	AutoSelected string `json:"auto_selected"`
+}
+
+// DescribeVariants reports an entry's selectable builds and which one
+// auto-selection would currently pick.
+//
+// It runs the SAME variantOptions + SelectVariant pass the installer runs, so
+// the reported auto-selection cannot drift from what installing would actually
+// do. The env carries the same probe seam too, so the size shown here and the
+// size the installer compares against come from one cache and one round trip.
+//
+// An entry that declares no variants returns nil, nil WITHOUT touching the
+// probe. That is load-bearing: the gallery listing walks entries by the
+// thousand and the overwhelming majority declare nothing, so the no-variant
+// case must cost nothing at all.
+func DescribeVariants(models []*GalleryModel, entry *GalleryModel, env ResolveEnv) (*EntryVariants, error) {
+	if entry == nil || !entry.HasVariants() {
+		return nil, nil
+	}
+
+	options, err := variantOptions(models, entry, env)
+	if err != nil {
+		return nil, err
+	}
+
+	selection, err := SelectVariant(options, env, "")
+	if err != nil {
+		return nil, err
+	}
+
+	views := make([]VariantView, 0, len(options))
+	for _, o := range options {
+		memory, known, err := o.EffectiveMemory()
+		if err != nil {
+			return nil, err
+		}
+		view := VariantView{
+			Model:   o.Variant.Model,
+			Backend: o.Backend,
+			// The base is exempt from every filter and always installs, so
+			// reporting it as anything but fitting would misdescribe it.
+			Fits:   o.IsBase || (env.backendRuns(o.Backend) && (!known || memory <= env.AvailableMemory)),
+			IsBase: o.IsBase,
+		}
+		if known {
+			view.MemoryBytes = memory
+		}
+		views = append(views, view)
+	}
+
+	return &EntryVariants{Variants: views, AutoSelected: selection.Option.Variant.Model}, nil
+}