refactor: remove "next" key from compat manifest, add DATABRICKS_FORCE_EMBEDDED_COMPAT

The manifest is now purely range-based: each versioned entry defines a
range floor that applies to that CLI version and all above it. The "next"
key was redundant since we always know the CLI version when updating the
manifest. Dev builds now resolve to the highest versioned entry.

Also adds DATABRICKS_FORCE_EMBEDDED_COMPAT=true env var to skip remote
fetch and use only the embedded manifest, useful for local development.

Co-authored-by: Isaac

Author: pkosiec

.agent/skills/bump-cli-compat/SKILL.md

@@ -68,7 +68,7 @@ Read `internal/build/cli-compat.json`. Note the current versions and the list of
 
 ### Step 4: Update the manifest
 
-Update **all entries** (both `next` and all versioned CLI entries) to the new appkit and skills versions. This is the "no template changes" scenario — a simple search & replace.
+Update the **highest versioned entry** to the new appkit and skills versions. Do NOT add new versioned keys — this is the default "no breaking changes" flow. The manifest is range-based: updating the highest entry automatically covers all CLI versions in that range.
 
 Write the updated `internal/build/cli-compat.json`.
 

internal/build/README.md

@@ -6,22 +6,27 @@
 
 ```json
 {
-  "next": { "appkit": "0.24.0", "skills": "0.1.5" },
-  "0.300.0": { "appkit": "0.24.0", "skills": "0.1.5" }
+  "0.296.0": { "appkit": "0.27.0", "skills": "0.1.5" },
+  "0.290.0": { "appkit": "0.24.0", "skills": "0.1.4" },
+  "0.280.0": { "appkit": "0.20.0", "skills": "0.1.0" }
 }
 ```
 
-- Each key is a CLI version (`X.Y.Z`) or `"next"`.
-- Each value specifies the compatible `appkit` and `skills` versions.
-- `"next"` is used for dev builds (`0.0.0-dev*`). For production CLI versions newer than all listed entries, the highest versioned entry is used.
+Each key is a CLI version in semver format. Each entry defines a **range floor**: it applies to that CLI version and all versions above it, up to (but not including) the next entry. The manifest should be **sparse** — not every CLI version needs its own entry. Only add a new entry when a compatibility boundary changes.
+
+For example, with the manifest above:
+- CLI `0.285.0` → uses `0.280.0` entry (appkit `0.20.0`)
+- CLI `0.293.0` → uses `0.290.0` entry (appkit `0.24.0`)
+- CLI `0.300.0` → uses `0.296.0` entry (appkit `0.27.0`, highest versioned)
+- CLI `0.0.0-dev+abc` → uses `0.296.0` entry (dev builds use the highest versioned entry)
 
 ## How the CLI resolves versions
 
-1. **Exact match** on CLI version → use that entry.
-2. **No exact match**, between two entries → use the nearest lower version's entry.
-3. **Newer than all entries** → use the highest versioned entry.
-4. **Older than all entries** → use the lowest (oldest) entry.
-5. **Dev builds** (`0.0.0-dev*`) → use `"next"`.
+1. **Dev builds** (`0.0.0-dev*`) → use the highest versioned entry.
+2. **Exact match** on CLI version → use that entry.
+3. **No exact match**, between two entries → use the nearest lower version's entry.
+4. **Newer than all entries** → use the highest versioned entry.
+5. **Older than all entries** → use the lowest (oldest) entry.
 
 ## Manifest sources (fallback chain)
 
@@ -32,15 +37,16 @@ At runtime, the CLI resolves the manifest from four sources:
 3. **Stale local cache** — if remote fetch fails but a previously cached file exists (even if expired), it is used as-is.
 4. **Embedded manifest** — compiled into the binary via `go:embed`. Used as last resort when both remote and local cache fail.
 
+Set `DATABRICKS_FORCE_EMBEDDED_COMPAT=true` to skip all tiers and use only the embedded manifest. This is useful for local development when testing with a locally compiled binary that has a modified `cli-compat.json`.
+
 ## When to update
 
-After each AppKit or Agent Skills release:
+The goal is to **keep the manifest sparse** — only add entries at compatibility boundaries. After each AppKit or Agent Skills release:
 
 1. **Run evals** on the new AppKit version. If there is no regression, proceed.
 2. **Open a PR** to update `cli-compat.json`. The change depends on the type of release:
-   - **No template changes** (just an AppKit/skills version bump): search & replace all version occurrences in the manifest and update `next`.
-   - **Template changes that don't require new CLI features**: test the last 3 CLI versions with the new template and update matching entries.
-   - **Template changes that require new CLI features**: add a new entry for the minimum CLI version that supports them; older entries keep pointing to the previous template version.
+   - **No breaking changes** (the new AppKit/skills version works with all existing CLI versions): update the existing highest versioned entry's appkit/skills values in-place. Do NOT add a new versioned key. All CLI versions in that range automatically pick up the new versions.
+   - **Breaking changes** (the new AppKit templates require specific `apps init` features, or the new skills version requires CLI commands that older CLIs lack): add a new entry keyed to the **minimum CLI version** that supports the new features. Older entries keep their previous appkit/skills values so older CLI binaries stay compatible.
 
 This process is manual for now but can be automated as part of the release workflow in the future. Use the `/bump-cli-compat` Claude Code skill to automate the update and PR creation.
 
@@ -52,7 +58,7 @@ The manifest is validated by Go tests in `libs/clicompat/`:
 go test ./libs/clicompat/... -run TestEmbeddedManifest -v
 ```
 
-This checks: valid JSON, `"next"` key present, at least one versioned entry, valid semver keys, valid semver entry values, `next` versions >= all entries, and ascending key order.
+This checks: valid JSON, at least one entry, valid semver keys, valid semver entry values, and ascending key order.
 
 ## Pruning policy
 

internal/build/cli-compat.json

@@ -1,4 +1,3 @@
 {
-  "next": { "appkit": "0.24.0", "skills": "0.1.5" },
   "0.300.0": { "appkit": "0.24.0", "skills": "0.1.5" }
 }

libs/clicompat/clicompat.go

@@ -41,16 +41,15 @@ const (
 	// fetchTimeout is the HTTP timeout for fetching the manifest at runtime.
 	fetchTimeout = 3 * time.Second
 
-	// nextKey is the special manifest key for CLI versions newer than any entry.
-	nextKey = "next"
-
 	// cacheTTL is how long a locally cached manifest is considered fresh.
 	cacheTTL = 1 * time.Hour
 
 	// localManifestFile is the filename for the locally cached manifest.
 	localManifestFile = "compat-manifest.json"
 
-	// devVersionPrefix identifies dev builds that always use the "next" entry.
+	// devVersionPrefix identifies dev builds whose semver (0.0.0) is lower than
+	// all real CLI versions. These are treated as bleeding-edge and resolve to
+	// the highest versioned entry.
 	devVersionPrefix = "0.0.0-dev"
 
 	maxFetchAttempts  = 3
@@ -88,9 +87,18 @@ var httpClient = &http.Client{Timeout: fetchTimeout}
 //  3. Stale local file (if remote fails but a previously cached file exists)
 //  4. Embedded manifest compiled into the binary
 //
+// Set DATABRICKS_FORCE_EMBEDDED_COMPAT=true to skip all tiers and use only
+// the embedded manifest. Useful for local development when testing with a
+// locally compiled binary.
+//
 // Set DATABRICKS_CACHE_ENABLED=false to bypass the local cache (tiers 1 and 3a),
 // which is useful to recover from a bad cached manifest.
 func FetchManifest(ctx context.Context) (Manifest, error) {
+	if force, ok := env.GetBool(ctx, "DATABRICKS_FORCE_EMBEDDED_COMPAT"); ok && force {
+		log.Debugf(ctx, "Using embedded manifest (DATABRICKS_FORCE_EMBEDDED_COMPAT=true)")
+		return parseEmbeddedManifest()
+	}
+
 	cacheEnabled := true
 	if enabled, ok := env.GetBool(ctx, "DATABRICKS_CACHE_ENABLED"); ok {
 		cacheEnabled = enabled
@@ -197,8 +205,11 @@ func IsNotFoundError(err error) bool {
 
 // Resolve returns the manifest entry for the given CLI version.
 //
+// Each versioned entry defines a range floor: it applies to that CLI version
+// and all versions above it, up to (but not including) the next entry.
+//
 // Resolution order:
-//  1. Dev builds (version starts with "0.0.0-dev") use the "next" entry.
+//  1. Dev builds (version starts with "0.0.0-dev") use the highest versioned entry.
 //  2. Exact match on CLI version.
 //  3. Nearest lower version (semver-sorted). This also handles CLI versions
 //     newer than all entries, returning the highest known entry.
@@ -208,34 +219,24 @@ func Resolve(m Manifest, cliVersion string) (Entry, error) {
 		return Entry{}, errors.New("empty compatibility manifest")
 	}
 
-	next, ok := m[nextKey]
-	if !ok {
-		return Entry{}, fmt.Errorf("compatibility manifest missing %q key", nextKey)
+	// Collect and sort versioned keys descending.
+	versions := sortedVersions(m)
+	if len(versions) == 0 {
+		return Entry{}, errors.New("compatibility manifest has no versioned entries")
 	}
 
-	// Dev builds always use "next".
+	// Dev builds (0.0.0-dev*) have semver lower than all real CLI versions,
+	// so they would incorrectly resolve to the lowest entry. Use the highest
+	// versioned entry instead, since dev builds represent the bleeding edge.
 	if strings.HasPrefix(cliVersion, devVersionPrefix) {
-		return next, nil
+		return m[versions[0]], nil
 	}
 
 	// Exact match.
 	if entry, ok := m[cliVersion]; ok {
 		return entry, nil
 	}
 
-	// Collect and sort versioned keys (exclude "next").
-	var versions []string
-	for k := range m {
-		if k != nextKey {
-			versions = append(versions, k)
-		}
-	}
-
-	// Sort descending by semver. The semver package requires a "v" prefix.
-	slices.SortFunc(versions, func(a, b string) int {
-		return semver.Compare("v"+b, "v"+a)
-	})
-
 	// Find the nearest lower version.
 	vCLI := "v" + cliVersion
 	for _, v := range versions {
@@ -244,14 +245,22 @@ func Resolve(m Manifest, cliVersion string) (Entry, error) {
 		}
 	}
 
-	// CLI is older than all entries — use the lowest (oldest) entry as closest match.
-	// If there are no versioned entries (only "next"), fall back to "next".
-	if len(versions) == 0 {
-		return next, nil
-	}
+	// CLI is older than all entries — use the lowest (oldest) entry.
 	return m[versions[len(versions)-1]], nil
 }
 
+// sortedVersions returns manifest keys sorted descending by semver.
+func sortedVersions(m Manifest) []string {
+	versions := make([]string, 0, len(m))
+	for k := range m {
+		versions = append(versions, k)
+	}
+	slices.SortFunc(versions, func(a, b string) int {
+		return semver.Compare("v"+b, "v"+a)
+	})
+	return versions
+}
+
 // resolveEntry fetches the manifest, resolves for the given CLI version.
 func resolveEntry(ctx context.Context, cliVersion string) (Entry, error) {
 	m, err := FetchManifest(ctx)
@@ -418,11 +427,8 @@ func parseManifest(data []byte) (Manifest, error) {
 	if len(m) == 0 {
 		return nil, errors.New("empty compatibility manifest")
 	}
-	if _, ok := m[nextKey]; !ok {
-		return nil, fmt.Errorf("compatibility manifest missing %q key", nextKey)
-	}
 	for k := range m {
-		if k != nextKey && !semver.IsValid("v"+k) {
+		if !semver.IsValid("v" + k) {
 			return nil, fmt.Errorf("invalid semver key %q in compatibility manifest", k)
 		}
 	}