Resolve AppKit and Agent Skills versions from compatibility manifest (#5139)

## Summary

Introduces a CLI compatibility manifest
(`internal/build/cli-compat.json`) that maps CLI versions to compatible
AppKit template and Agent Skills versions. This enables template updates
to reach users without CLI releases.

### Manifest format

The manifest is purely range-based — each versioned entry defines a
**range floor** that applies to that CLI version and all versions above
it, up to the next entry. The manifest should be sparse: only add a new
entry when a compatibility boundary changes (e.g., new AppKit templates
require specific CLI features).

### Resolution

- **Exact match** → use that entry
- **Between entries** → nearest lower version
- **Newer than all** → highest versioned entry
- **Dev builds** (`0.0.0-dev*`) → highest versioned entry

### Manifest sources (fallback chain)

1. Fresh local cache (< 1h)
2. Remote fetch from GitHub (with retry)
3. Stale local cache or embedded manifest fallback

Set `DATABRICKS_FORCE_EMBEDDED_COMPAT=true` to skip remote fetch and use
only the embedded manifest (useful for local development).

### Companion PRs

- [databricks/appkit#333](databricks/appkit#333)
-
[databricks/databricks-agent-skills#64](databricks/databricks-agent-skills#64)

## Screenshot

<img width="1046" height="570" alt="image"
src="https://github.com/user-attachments/assets/1b51cd90-bcaa-4b0e-bcbd-a5370da572ef"
/>

---------

Signed-off-by: Pawel Kosiec <pawel.kosiec@databricks.com>

Author: pkosiec

.agent/skills/bump-cli-compat/SKILL.md

@@ -0,0 +1,153 @@
+---
+name: bump-cli-compat
+description: "Bump cli-compat.json with new AppKit and Agent Skills versions, then create a PR. Use when the user says 'bump cli-compat', 'update cli-compat', 'bump compatibility manifest', 'new appkit release cli-compat', or wants to update the CLI compatibility manifest after an AppKit or Agent Skills release."
+user-invocable: true
+allowed-tools: Read, Edit, Write, Bash, Glob, Grep, AskUserQuestion
+---
+
+# Bump CLI Compatibility Manifest
+
+Updates `internal/build/cli-compat.json` with new AppKit and Agent Skills versions, validates the result, and creates a PR.
+
+## Arguments
+
+Parse the user's input for optional named flags:
+
+- `--appkit <version>` → AppKit version (e.g. `0.28.0`)
+- `--skills <version>` → Agent Skills version (e.g. `0.1.6`)
+- No args → auto-detect latest versions from GitHub tags
+
+Versions should be provided **without** the `v` prefix (e.g. `0.28.0`, not `v0.28.0`). If provided with the prefix, strip it.
+
+## Workflow
+
+### Step 1: Resolve versions
+
+If both `--appkit` and `--skills` versions were provided, skip to Step 2.
+
+For any missing version, fetch the latest tag from GitHub:
+
+```bash
+# Latest appkit version (strip leading 'v')
+gh api repos/databricks/appkit/tags --jq '.[0].name' | sed 's/^v//'
+
+# Latest skills version (strip leading 'v')
+gh api repos/databricks/databricks-agent-skills/tags --jq '.[0].name' | sed 's/^v//'
+```
+
+Show the resolved versions to the user and ask:
+
+> The latest versions are:
+> - AppKit: `{appkit_version}`
+> - Agent Skills: `{skills_version}`
+>
+> Have these versions been evaluated (evals passed with no regressions)?
+
+**Do NOT proceed until the user confirms.** If the user says no or wants different versions, ask them to provide the correct versions.
+
+### Step 2: Validate tags exist
+
+Verify that the corresponding Git tags exist on GitHub. For AppKit, also validate the `template-v` tag (used by `apps init`):
+
+```bash
+# AppKit release tag
+gh api repos/databricks/appkit/git/ref/tags/v{appkit_version} --jq '.ref'
+
+# AppKit template tag (used by apps init)
+gh api repos/databricks/appkit/git/ref/tags/template-v{appkit_version} --jq '.ref'
+
+# Agent Skills tag
+gh api repos/databricks/databricks-agent-skills/git/ref/tags/v{skills_version} --jq '.ref'
+```
+
+If any tag doesn't exist, report the error and stop.
+
+### Step 3: Read current manifest
+
+Read `internal/build/cli-compat.json`. Note the current versions and the list of versioned entries.
+
+### Step 4: Determine update type
+
+Ask the user:
+
+> Do any of these apply?
+> - **AppKit**: The new templates require new CLI logic in `apps init` (e.g. new flags, prompts, or template handling that older CLIs don't have)
+> - **Skills**: The new skills version uses CLI commands that older CLIs don't support
+>
+> If **neither** applies, this is a non-breaking bump (default).
+
+- **No breaking changes** (default): proceed to Step 4a.
+- **Breaking changes**: proceed to Step 4b.
+
+### Step 4a: No breaking changes (update in-place)
+
+Update the **highest versioned entry** to the new appkit and skills versions. Do NOT add new versioned keys. The manifest is range-based: updating the highest entry automatically covers all CLI versions in that range.
+
+Write the updated `internal/build/cli-compat.json`.
+
+### Step 4b: Breaking changes (add new entry)
+
+Ask the user for the **minimum CLI version** that supports the new features.
+
+Add a new entry keyed to that CLI version with the new appkit and skills versions. Keep older entries unchanged so older CLI binaries stay compatible.
+
+Write the updated `internal/build/cli-compat.json`.
+
+### Step 5: Validate
+
+Run the Go tests to ensure the manifest is well-formed:
+
+```bash
+go test ./libs/clicompat/... -run TestEmbeddedManifest -v
+```
+
+If validation fails, show the errors and fix them before proceeding.
+
+### Step 6: Create branch, commit, and PR
+
+```bash
+# Create a new branch from the current branch (or main)
+git checkout -b bump-cli-compat-appkit-{appkit_version}-skills-{skills_version}
+
+# Stage and commit
+git add internal/build/cli-compat.json
+git commit -s -m "Bump cli-compat to appkit {appkit_version}, skills {skills_version}"
+
+# Push and create PR
+git push -u origin HEAD
+gh pr create \
+  --title "Bump cli-compat to appkit {appkit_version}, skills {skills_version}" \
+  --body "$(cat <<'EOF'
+## Summary
+Bump `cli-compat.json` to use:
+- AppKit `{appkit_version}`
+- Agent Skills `{skills_version}`
+
+## Checklist
+- [ ] Evals passed with no regressions
+- [ ] `go test ./libs/clicompat/... -run TestEmbeddedManifest` passes
+EOF
+)"
+```
+
+Show the PR URL to the user when done.
+
+## Examples
+
+### Example: With explicit versions
+```
+/bump-cli-compat --appkit 0.28.0 --skills 0.1.6
+```
+Validates tags exist (including `template-v0.28.0`), updates manifest, creates PR.
+
+### Example: Auto-detect latest
+```
+/bump-cli-compat
+```
+Fetches latest tags, asks for eval confirmation, then updates and creates PR.
+
+### Example: Only bump AppKit
+```
+/bump-cli-compat --appkit 0.28.0
+```
+Auto-detects latest skills version, asks for confirmation, then updates both.

.github/OWNERS

@@ -59,5 +59,9 @@
 # Internal
 /internal/                  team:platform
 
+# CLI compatibility manifest
+/internal/build/cli-compat.json  team:eng-apps-devex team:platform
+/libs/clicompat/                 team:eng-apps-devex team:platform
+
 # Experimental
 /experimental/aitools/      team:eng-apps-devex @lennartkats-db

cmd/apps/init.go

@@ -23,6 +23,7 @@ import (
 	"github.com/databricks/cli/libs/apps/initializer"
 	"github.com/databricks/cli/libs/apps/manifest"
 	"github.com/databricks/cli/libs/apps/prompt"
+	"github.com/databricks/cli/libs/clicompat"
 	"github.com/databricks/cli/libs/cmdctx"
 	"github.com/databricks/cli/libs/cmdio"
 	"github.com/databricks/cli/libs/env"
@@ -38,7 +39,6 @@ const (
 	appkitTemplateDir    = "template"
 	appkitDefaultBranch  = "main"
 	appkitTemplateTagPfx = "template-v"
-	appkitDefaultVersion = "template-v0.24.0"
 	defaultProfile       = "DEFAULT"
 )
 
@@ -169,7 +169,7 @@ Environment variables:
 
 	cmd.Flags().StringVar(&templatePath, "template", "", "Template path (local directory or GitHub URL)")
 	cmd.Flags().StringVar(&branch, "branch", "", "Git branch or tag (for GitHub templates, mutually exclusive with --version)")
-	cmd.Flags().StringVar(&version, "version", "", fmt.Sprintf("AppKit version to use (default: %s, use 'latest' for main branch)", appkitDefaultVersion))
+	cmd.Flags().StringVar(&version, "version", "", "AppKit version to use (default: auto-detected, use 'latest' for main branch)")
 	cmd.Flags().StringVar(&name, "name", "", "Project name (prompts if not provided)")
 	cmd.Flags().StringVar(&warehouseID, "warehouse-id", "", "SQL warehouse ID")
 	_ = cmd.Flags().MarkDeprecated("warehouse-id", "use --set <plugin>.sql-warehouse.id=<value> instead")
@@ -805,8 +805,12 @@ func runCreate(ctx context.Context, opts createOptions) error {
 		case opts.version != "":
 			gitRef = normalizeVersion(opts.version)
 		default:
-			// Default: use pinned version
-			gitRef = appkitDefaultVersion
+			appkitVersion, err := clicompat.ResolveAppKitVersion(ctx)
+			if err != nil {
+				return fmt.Errorf("could not resolve AppKit template version: %w; use --version to specify a version manually", err)
+			}
+			gitRef = normalizeVersion(appkitVersion)
+			cmdio.LogString(ctx, "Using AppKit template version "+appkitVersion)
 		}
 		templateSrc = appkitRepoURL
 	}
@@ -859,6 +863,20 @@ func runCreate(ctx context.Context, opts createOptions) error {
 
 	// Step 2: Wait for template (may already be done if the user took time typing the name)
 	resolvedPath, cleanup, err := awaitTemplate(ctx, templateCh)
+	// Only fall back to the embedded version when the version was auto-resolved
+	// from the manifest, not when the user explicitly passed --version or --branch.
+	versionAutoResolved := opts.version == "" && opts.branch == ""
+	if err != nil && usingDefaultTemplate && versionAutoResolved && clicompat.IsNotFoundError(err) {
+		fallbackVersion, fbErr := clicompat.ResolveEmbeddedAppKitVersion()
+		if fbErr == nil && fallbackVersion != "" && normalizeVersion(fallbackVersion) != branchForClone {
+			log.Warnf(ctx, "Template version not found, falling back to embedded version %s", fallbackVersion)
+			fallbackRef := normalizeVersion(fallbackVersion)
+			templateCh = resolveTemplateAsync(ctx, templateSrc, fallbackRef, appkitTemplateDir)
+			resolvedPath, cleanup, err = awaitTemplate(ctx, templateCh)
+		} else if fbErr != nil {
+			log.Warnf(ctx, "Could not resolve embedded AppKit version: %v", fbErr)
+		}
+	}
 	if err != nil {
 		return err
 	}

cmd/apps/init_test.go

@@ -443,7 +443,7 @@ func TestNormalizeVersion(t *testing.T) {
 		{"", ""},
 		{"main", "main"},
 		{"feat/something", "feat/something"},
-		{appkitDefaultVersion, appkitDefaultVersion},
+		{"template-v0.24.0", "template-v0.24.0"},
 	}
 
 	for _, tt := range tests {

cmd/apps/manifest.go

@@ -9,7 +9,9 @@ import (
 	"path/filepath"
 
 	"github.com/databricks/cli/libs/apps/manifest"
+	"github.com/databricks/cli/libs/clicompat"
 	"github.com/databricks/cli/libs/env"
+	"github.com/databricks/cli/libs/log"
 	"github.com/spf13/cobra"
 )
 
@@ -27,7 +29,11 @@ func runManifestOnly(ctx context.Context, templatePath, branch, version string)
 		case version != "":
 			gitRef = normalizeVersion(version)
 		default:
-			gitRef = appkitDefaultVersion
+			appkitVersion, err := clicompat.ResolveAppKitVersion(ctx)
+			if err != nil {
+				return fmt.Errorf("could not resolve AppKit template version: %w; use --version to specify a version manually", err)
+			}
+			gitRef = normalizeVersion(appkitVersion)
 		}
 		templateSrc = appkitRepoURL
 	}
@@ -39,6 +45,17 @@ func runManifestOnly(ctx context.Context, templatePath, branch, version string)
 		subdirForClone = appkitTemplateDir
 	}
 	resolvedPath, cleanup, err := resolveTemplate(ctx, templateSrc, branchForClone, subdirForClone)
+	versionAutoResolved := version == "" && branch == ""
+	if err != nil && usingDefaultTemplate && versionAutoResolved && clicompat.IsNotFoundError(err) {
+		fallbackVersion, fbErr := clicompat.ResolveEmbeddedAppKitVersion()
+		if fbErr == nil && fallbackVersion != "" && normalizeVersion(fallbackVersion) != gitRef {
+			log.Warnf(ctx, "Template version not found, falling back to embedded version %s", fallbackVersion)
+			fallbackRef := normalizeVersion(fallbackVersion)
+			resolvedPath, cleanup, err = resolveTemplate(ctx, templateSrc, fallbackRef, appkitTemplateDir)
+		} else if fbErr != nil {
+			log.Warnf(ctx, "Could not resolve embedded AppKit version: %v", fbErr)
+		}
+	}
 	if err != nil {
 		return err
 	}

experimental/aitools/cmd/list.go

@@ -48,10 +48,13 @@ func newListCmd() *cobra.Command {
 func defaultListSkills(cmd *cobra.Command, scope string) error {
 	ctx := cmd.Context()
 
-	ref := installer.GetSkillsRef(ctx)
+	ref, explicit, err := installer.GetSkillsRef(ctx)
+	if err != nil {
+		return err
+	}
 
 	src := &installer.GitHubManifestSource{}
-	manifest, err := src.FetchManifest(ctx, ref)
+	manifest, ref, err := installer.FetchSkillsManifestWithFallback(ctx, src, ref, !explicit)
 	if err != nil {
 		return fmt.Errorf("failed to fetch manifest: %w", err)
 	}

experimental/aitools/cmd/version.go

@@ -7,6 +7,7 @@ import (
 
 	"github.com/databricks/cli/experimental/aitools/lib/installer"
 	"github.com/databricks/cli/libs/cmdio"
+	"github.com/databricks/cli/libs/log"
 	"github.com/spf13/cobra"
 )
 
@@ -44,7 +45,10 @@ func newVersionCmd() *cobra.Command {
 				return nil
 			}
 
-			latestRef := installer.GetSkillsRef(ctx)
+			latestRef, _, err := installer.GetSkillsRef(ctx)
+			if err != nil {
+				log.Debugf(ctx, "could not resolve skills version: %v", err)
+			}
 			bothScopes := globalState != nil && projectState != nil
 
 			cmdio.LogString(ctx, "Databricks AI Tools:")
@@ -80,6 +84,12 @@ func printVersionLine(ctx context.Context, label string, state *installer.Instal
 		skillNoun = "skill"
 	}
 
+	if latestRef == "" {
+		cmdio.LogString(ctx, fmt.Sprintf("  %s: v%s (%d %s)", label, version, len(state.Skills), skillNoun))
+		cmdio.LogString(ctx, "  Last updated: "+state.LastUpdated.Format("2006-01-02"))
+		return
+	}
+
 	if latestRef == state.Release {
 		cmdio.LogString(ctx, fmt.Sprintf("  %s: v%s (%d %s, up to date)", label, version, len(state.Skills), skillNoun))
 		cmdio.LogString(ctx, "  Last updated: "+state.LastUpdated.Format("2006-01-02"))