fix(gallery): rank the entry's own build against its variants

Variant selection pulled the declaring entry's own payload, the base, out
of the candidate set and consulted it only once every declared variant had
been rejected. Two real failures followed.

A variant whose size the probe cannot determine deliberately survives the
memory filter, because nothing proves it does not fit. As the only survivor
it then won outright on any host, however small: a 2GiB machine installed an
unmeasured variant in preference to the 4GiB build the entry itself ships,
with no warning. 241 of the 1280 current index entries carry no files and no
size, which is exactly that shape.

"Largest wins" also broke whenever the base was the largest. An author
writing a Q8 entry that offers a Q4 downgrade for small hosts, a natural
shape that nothing in the lint, schema or docs discourages, had the Q4
installed on every large host instead.

Make the base an ordinary participant. It is still exempt from both filters,
so selection always terminates on something installable, but it is now
ranked against the variants: a proven fit first and largest, then the base,
then any variant whose size nothing could measure. Both failures disappear
together. The base is probed for its size accordingly, which it was not
before, because an unsized base would lose every contest to an unmeasurable
variant.

FellBackToBase is kept but narrowed to "no declared variant survived",
rather than "the base was chosen", since the base now also wins on merit and
that is not worth warning about.

A recalled variant pin also became a permanent install failure. A pin the
caller supplies on this request must stay fatal, but one recalled from
._gallery_<name>.yaml can be invalidated by any later gallery edit, and
failing on it turned one rename into a model that could never be reinstalled
or upgraded again short of deleting a dotfile the user has never heard of.
A stale recalled pin is now dropped with a warning naming it, and selection
runs as if it had never been recorded.

Also drop the last textual reference to two abandoned designs from the
DetectedCapability comment, correct the documented variants JSON example,
which showed a memory_bytes of 0 that omitempty makes impossible, and remove
an em dash from the install skill.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>

Author: mudler

.agents/adding-gallery-models.md

@@ -114,11 +114,15 @@ Rules:
   and must not declare `variants` of its own.
 - **Order carries no meaning.** Do not try to encode a preference; write the
   list in whatever order reads best.
+- **A variant may be smaller than the declaring entry.** Offering a downgrade
+  for small hosts is a normal shape: the declaring entry's own build competes on
+  size like every other candidate, so a large host keeps the large build.
 - **Do not describe hardware.** At install time LocalAI drops variants whose
   backend cannot run on the host, drops those that do not fit available memory,
-  and installs the largest survivor, falling back to the declaring entry's own
-  build. Sizes are measured live from the weights and cached, so nothing has to
-  be written down.
+  and installs the largest of what is left, the declaring entry's own build
+  included. That build is never dropped, so selection always terminates on
+  something installable. Sizes are measured live from the weights and cached, so
+  nothing has to be written down.
 - A variant is nothing but a name; there is no per-variant memory field. When
   the measured size for a build is wrong, correct it on the referenced entry by
   setting that entry's own `size:` (e.g. `size: "20GiB"`). The estimator prefers

core/gallery/describe_variants.go

@@ -12,9 +12,10 @@ type VariantView struct {
 	// it, and it is also the reason Fits may be false on a host whose memory
 	// would be ample.
 	Backend string `json:"backend"`
-	// MemoryBytes is the measured footprint, or 0 when it could not be
-	// determined. Zero is unknown, never "needs nothing", so a client must
-	// render it as unknown rather than as a free option.
+	// MemoryBytes is the measured footprint. It is omitted from the JSON
+	// entirely when the size could not be determined, rather than serialized as
+	// a zero that a client would have to know to read as unknown. An absent key
+	// never means "needs nothing".
 	MemoryBytes uint64 `json:"memory_bytes,omitempty"`
 	// Fits reports whether auto-selection would consider this variant on this
 	// host: its backend can run here and its known footprint is within budget.

core/gallery/models.go

@@ -111,13 +111,18 @@ func variantOptions(models []*GalleryModel, entry *GalleryModel, env ResolveEnv)
 		options = append(options, option)
 	}
 
-	// The base is never filtered nor ranked, so its size would change nothing
-	// and probing it would spend a round trip on a discarded answer.
-	return append(options, VariantOption{
+	// The base is probed like any other candidate. It is never filtered out, but
+	// it IS ranked against the variants, and an unsized base would lose every
+	// contest to a variant whose size nothing could measure.
+	base := VariantOption{
 		Variant: Variant{Model: entry.Name},
 		Backend: entry.Backend,
 		IsBase:  true,
-	}), nil
+	}
+	if env.ProbeMemory != nil {
+		base.ProbedMemory = env.ProbeMemory(entry)
+	}
+	return append(options, base), nil
 }
 
 // probeContextLength is the context size the footprint estimate is taken at.
@@ -180,9 +185,10 @@ func probeEntryMemory(ctx context.Context, entry *GalleryModel) uint64 {
 //
 // The base entry always resolves, whatever the host has. It is a complete entry
 // that every older LocalAI release installs unconditionally, so refusing it here
-// would make the gallery behave worse the newer the client is. That is also why
-// the base declares no memory requirement of its own: a floor that can only
-// warn cannot change any outcome.
+// would make the gallery behave worse the newer the client is. Being exempt from
+// the filters does not make it exempt from ranking: it is measured and compared
+// like every declared variant, and wins by default only once they have all been
+// ruled out.
 func ResolveVariant(models []*GalleryModel, entry *GalleryModel, env ResolveEnv, pin string) (*GalleryModel, Variant, error) {
 	options, err := variantOptions(models, entry, env)
 	if err != nil {
@@ -462,19 +468,40 @@ func InstallModelFromGallery(
 	// back under the entry's own name would miss the record for every custom-named
 	// install and silently re-resolve a deliberately pinned model onto a
 	// different variant, possibly swapping its backend.
+	//
+	// recalledPin tracks where the pin came from, because the two sources must
+	// fail differently when the name no longer resolves. See below.
+	recalledPin := ""
 	if pin == "" {
 		installName := model.Name
 		if req.Name != "" {
 			installName = req.Name
 		}
 		if previous, err := GetLocalModelConfiguration(systemState.Model.ModelsPath, installName); err == nil && previous != nil {
-			pin = previous.PinnedVariant
+			recalledPin = previous.PinnedVariant
+			pin = recalledPin
 		}
 	}
 
 	env := HostResolveEnv(ctx, systemState)
 
 	resolved, variant, err := ResolveVariant(models, model, env, pin)
+	// A pin the caller supplied on this request must stay fatal: they named
+	// something this entry cannot give, and installing anything else would
+	// report success for a request that was not honored.
+	//
+	// A pin recalled from disk is different. The gallery can rename or withdraw
+	// a variant long after it was pinned, and the user is not asking for it
+	// again on this call. Failing here would turn one gallery edit into a
+	// permanently unrepairable model whose only remedy is deleting a dotfile
+	// they have never heard of, so the stale pin is dropped and selection runs
+	// as if it had never been recorded.
+	if err != nil && recalledPin != "" && errors.Is(err, ErrPinNotFound) {
+		xlog.Warn("The recorded variant pin for this model no longer exists in the gallery; re-selecting automatically",
+			"model", model.Name, "dropped_pin", recalledPin)
+		pin = ""
+		resolved, variant, err = ResolveVariant(models, model, env, "")
+	}
 	if err != nil {
 		return err
 	}

core/gallery/resolve_variant.go

@@ -27,8 +27,10 @@ type VariantOption struct {
 	// precisely why a gallery author never has to describe hardware.
 	Backend string
 	// IsBase marks the declaring entry's own payload. It is exempt from every
-	// filter and is the answer when nothing else survives, because the entry
-	// must stay installable on every host and for every client.
+	// filter, because the entry must stay installable on every host and for
+	// every client, but it is otherwise an ordinary candidate: it is ranked
+	// against the declared variants rather than consulted only once they have
+	// all been rejected.
 	IsBase bool
 	// ProbedMemory is the footprint measured live from the referenced entry's
 	// weights, in bytes. It is the only source of a variant's size. An author
@@ -87,9 +89,14 @@ func (e ResolveEnv) backendRuns(backend string) bool {
 // VariantSelection is the outcome of a selection pass.
 type VariantSelection struct {
 	Option VariantOption
-	// FellBackToBase reports that no declared variant survived and the entry's
-	// own payload was chosen instead. Callers log this, because a host quietly
-	// taking the base when upgrades were on offer is worth being able to see.
+	// FellBackToBase reports that no declared variant survived the filters and
+	// the entry's own payload was all that remained. Callers log this, because a
+	// host quietly taking the base when upgrades were on offer is worth being
+	// able to see.
+	//
+	// It is deliberately narrower than "the base was selected": the base also
+	// wins on merit whenever it outranks every surviving variant, and that is an
+	// ordinary, uninteresting outcome rather than something to warn about.
 	FellBackToBase bool
 	// Reasons explains, one line per rejected variant, why it was dropped.
 	Reasons []string
@@ -107,11 +114,13 @@ type VariantSelection struct {
 //     dropped. A variant with an UNKNOWN requirement survives, because nothing
 //     proves it does not fit and refusing on a size the probe could not read
 //     would let a network hiccup silently downgrade what gets installed.
-//  4. The largest survivor wins. A bigger footprint is a higher quality
-//     quantization of the same model, so among things that fit, more is better.
-//     Unknown requirements rank last, so a proven fit always beats a guess.
-//  5. With no survivor the base option wins. The base always installs; this
-//     never refuses.
+//  4. The base survives both filters unconditionally, and then competes. It is
+//     a candidate like any other, not a last resort: an entry whose own build is
+//     the largest thing that fits must win against a smaller variant, and a base
+//     of known size must win against a variant whose size nothing could measure.
+//  5. The survivors are ranked and the best one wins, by rankOf below.
+//  6. With no survivor at all, which can only happen when the caller supplied no
+//     base, there is nothing to install and this reports ErrNoVariantMatch.
 func SelectVariant(options []VariantOption, env ResolveEnv, pin string) (VariantSelection, error) {
 	if pin != "" {
 		for _, o := range options {
@@ -125,54 +134,88 @@ func SelectVariant(options []VariantOption, env ResolveEnv, pin string) (Variant
 	type ranked struct {
 		option VariantOption
 		memory uint64
-		known  bool
+		rank   int
 	}
 
-	var base *VariantOption
 	survivors := make([]ranked, 0, len(options))
 	reasons := make([]string, 0, len(options))
+	survivingVariants := 0
 
 	for i := range options {
 		o := options[i]
-		if o.IsBase {
-			base = &options[i]
-			continue
-		}
-
-		if !env.backendRuns(o.Backend) {
-			reasons = append(reasons, fmt.Sprintf("%s needs backend %q, which cannot run on this system", o.Variant.Model, o.Backend))
-			continue
-		}
-
 		memory, known := o.EffectiveMemory()
-		if known && memory > env.AvailableMemory {
-			reasons = append(reasons, fmt.Sprintf("%s needs %s of memory", o.Variant.Model, humanBytes(memory)))
-			continue
+
+		// The base skips both gates. There is nothing below it, so refusing it
+		// would make an entry every older LocalAI installs fine uninstallable on
+		// newer ones.
+		if !o.IsBase {
+			if !env.backendRuns(o.Backend) {
+				reasons = append(reasons, fmt.Sprintf("%s needs backend %q, which cannot run on this system", o.Variant.Model, o.Backend))
+				continue
+			}
+			if known && memory > env.AvailableMemory {
+				reasons = append(reasons, fmt.Sprintf("%s needs %s of memory", o.Variant.Model, humanBytes(memory)))
+				continue
+			}
+			survivingVariants++
 		}
 
-		survivors = append(survivors, ranked{option: o, memory: memory, known: known})
+		survivors = append(survivors, ranked{option: o, memory: memory, rank: rankOf(o, env)})
 	}
 
-	if len(survivors) > 0 {
-		// Stable so that variants with identical requirements keep their
-		// authored order, which is the only thing order still decides.
-		sort.SliceStable(survivors, func(i, j int) bool {
-			if survivors[i].known != survivors[j].known {
-				return survivors[i].known
-			}
-			return survivors[i].memory > survivors[j].memory
-		})
-		return VariantSelection{Option: survivors[0].option, Reasons: reasons}, nil
+	if len(survivors) == 0 {
+		return VariantSelection{}, fmt.Errorf(
+			"%w: %s of memory available; variants: %s",
+			ErrNoVariantMatch, humanBytes(env.AvailableMemory), strings.Join(reasons, "; "),
+		)
 	}
 
-	if base != nil {
-		return VariantSelection{Option: *base, FellBackToBase: true, Reasons: reasons}, nil
-	}
+	// Stable so that options within one rank and of identical size keep their
+	// authored order, which is the only thing order still decides.
+	sort.SliceStable(survivors, func(i, j int) bool {
+		if survivors[i].rank != survivors[j].rank {
+			return survivors[i].rank < survivors[j].rank
+		}
+		return survivors[i].memory > survivors[j].memory
+	})
+
+	winner := survivors[0].option
+	return VariantSelection{
+		Option: winner,
+		// Only a base that won by default is worth reporting. A base that
+		// outranked live competition is an ordinary selection.
+		FellBackToBase: winner.IsBase && survivingVariants == 0,
+		Reasons:        reasons,
+	}, nil
+}
 
-	return VariantSelection{}, fmt.Errorf(
-		"%w: %s of memory available; variants: %s",
-		ErrNoVariantMatch, humanBytes(env.AvailableMemory), strings.Join(reasons, "; "),
-	)
+// Ranks, best first. Within a rank the larger footprint wins, because a bigger
+// build is a higher quality quantization of the same model.
+const (
+	// rankProvenFit is a measured size that the host is measured to satisfy.
+	rankProvenFit = iota
+	// rankBase is the entry's own build when it is not a proven fit: either it
+	// needs more memory than the host reports, or its size could not be
+	// measured either. It still outranks any unsized variant, because it is the
+	// payload the entry is guaranteed to be able to install and a variant of
+	// unmeasurable size is a guess. Taking the guess on a host that cannot be
+	// shown to accommodate it is how an unreachable network silently changes
+	// what gets installed.
+	rankBase
+	// rankUnknownFit is a variant whose size nothing could measure. Nothing
+	// proves it does not fit, so it is not dropped, but nothing proves it does
+	// either, so it ranks last.
+	rankUnknownFit
+)
+
+func rankOf(o VariantOption, env ResolveEnv) int {
+	if memory, known := o.EffectiveMemory(); known && memory <= env.AvailableMemory {
+		return rankProvenFit
+	}
+	if o.IsBase {
+		return rankBase
+	}
+	return rankUnknownFit
 }
 
 func humanBytes(b uint64) string {

core/gallery/resolve_variant_test.go

@@ -47,7 +47,8 @@ var _ = Describe("SelectVariant", func() {
 	}
 
 	// The base is exempt from every filter, which the fallback specs below pin
-	// down; its size is carried only so ranking has nothing special to do.
+	// down, but it is ranked against the variants like any other candidate, so
+	// its size is load-bearing.
 	base := func(model string, probed uint64) gallery.VariantOption {
 		o := option(model, "llama-cpp", probed)
 		o.IsBase = true
@@ -159,8 +160,62 @@ var _ = Describe("SelectVariant", func() {
 
 			selection, err := gallery.SelectVariant(options, gallery.ResolveEnv{AvailableMemory: gib(4)}, "")
 			Expect(err).ToNot(HaveOccurred())
-			Expect(selection.Option.Variant.Model).To(Equal("m-unknown"))
+			// Surviving is observable through the rejection reasons: a dropped
+			// variant is always accounted for there, and this one is not.
+			Expect(selection.Reasons).ToNot(ContainElement(ContainSubstring("m-unknown")))
+			// It survives, but it does not win: the base is a sized, guaranteed
+			// payload and an unmeasurable variant is a guess.
+			Expect(selection.Option.Variant.Model).To(Equal("m-base"))
+			Expect(selection.FellBackToBase).To(BeFalse())
+		})
+
+		It("installs the base rather than an unsized variant on a host too small for either", func() {
+			// The exact shape 241 of the current index entries have: a referenced
+			// entry with no files and no size, whose probe can only answer
+			// "unknown". Ranking it above the base would install an unmeasured
+			// download on a machine with 2GiB, and would do so silently.
+			options := []gallery.VariantOption{
+				option("m-unknown", "llama-cpp", 0),
+				base("m-base-q4", gib(4)),
+			}
+
+			selection, err := gallery.SelectVariant(options, gallery.ResolveEnv{AvailableMemory: gib(2)}, "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(selection.Option.Variant.Model).To(Equal("m-base-q4"))
+			Expect(selection.Option.IsBase).To(BeTrue())
+		})
+
+		It("selects the base when the base is the largest option that fits", func() {
+			// A Q8 base offering a Q4 downgrade for small hosts is a natural
+			// authoring shape. Treating the base as a last resort would install
+			// the Q4 on every host large enough for the Q8 and permanently
+			// downgrade the user.
+			options := []gallery.VariantOption{
+				option("m-q4", "llama-cpp", gib(4)),
+				base("m-base-q8", gib(8)),
+			}
+
+			selection, err := gallery.SelectVariant(options, gallery.ResolveEnv{AvailableMemory: gib(16)}, "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(selection.Option.Variant.Model).To(Equal("m-base-q8"))
+			// The Q4 survived every filter, so this is the base winning on rank
+			// and not the base being fallen back to.
 			Expect(selection.FellBackToBase).To(BeFalse())
+			Expect(selection.Reasons).To(BeEmpty())
+		})
+
+		It("selects a smaller variant when the base does not fit but the variant does", func() {
+			// The mirror of the spec above: the base competes, it does not win by
+			// default, so a host that cannot hold it must still take the downgrade
+			// the entry offers for exactly that case.
+			options := []gallery.VariantOption{
+				option("m-q4", "llama-cpp", gib(4)),
+				base("m-base-q8", gib(8)),
+			}
+
+			selection, err := gallery.SelectVariant(options, gallery.ResolveEnv{AvailableMemory: gib(6)}, "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(selection.Option.Variant.Model).To(Equal("m-q4"))
 		})
 
 		It("reports why a probed size that does not fit was rejected", func() {
@@ -213,6 +268,21 @@ var _ = Describe("SelectVariant", func() {
 			selection, err := gallery.SelectVariant(options, gallery.ResolveEnv{AvailableMemory: 0}, "")
 			Expect(err).ToNot(HaveOccurred())
 			Expect(selection.Option.Variant.Model).To(Equal("m-base"))
+			Expect(selection.Option.IsBase).To(BeTrue())
+			Expect(selection.FellBackToBase).To(BeTrue())
+		})
+
+		It("prefers the base to an unsized variant even when the base itself is unsized", func() {
+			// Neither can be shown to fit, so nothing separates them on size. The
+			// base is still the payload the entry is guaranteed to install.
+			options := []gallery.VariantOption{
+				option("m-unknown", "llama-cpp", 0),
+				base("m-base", 0),
+			}
+
+			selection, err := gallery.SelectVariant(options, gallery.ResolveEnv{AvailableMemory: gib(8)}, "")
+			Expect(err).ToNot(HaveOccurred())
+			Expect(selection.Option.Variant.Model).To(Equal("m-base"))
 		})
 
 		It("explains why each variant was rejected", func() {