aitools: parse experimental_skills manifest section (#5243)

## Summary

The skills manifest in `databricks/databricks-agent-skills` is gaining
experimental skills sourced from a new `experimental/` directory in the
repo (see paired [d-a-s PR
#73](databricks/databricks-agent-skills#73),
which imports the ai-dev-kit skill catalog into `experimental/`).

This wires the parsing through the aitools installer:

- `Manifest.Skills` is a **single map** holding both stable and
experimental entries; the per-skill `repo_dir` field ("skills" or
"experimental") is the source of truth for whether a skill is
experimental. `SkillMeta.IsExperimental()` derives state from `RepoDir`.
- Experimental skills get a `-experimental` suffix on their install-side
key during `normalizeManifest`; `SourceName` preserves the unsuffixed
name for fetch URLs.
- The existing `--experimental` flag (already wired in `cmd/skills.go`)
now has experimental skills to install; without it, `resolveSkills`
filters them out as before.

## UX

```
# default — only stable skills
databricks experimental aitools skills install

# all experimental skills, plus stable
databricks experimental aitools skills install --experimental

# one experimental skill by name (--experimental still required by resolveSkills)
databricks experimental aitools skills install databricks-iceberg-experimental --experimental
```

## TODOs / caveats for iteration

1. ~~**`DATABRICKS_SKILLS_REF` pin.**~~ **Partially resolved.** The
default ref is still the latest stable release tag (sourced from
`experimental/aitools/lib/installer/SKILLS_VERSION`); experimental
entries won't exist there until d-a-s cuts a release with [PR
#73](databricks/databricks-agent-skills#73)
merged. The default ref bump is a follow-up automated by the
SKILLS_VERSION file. **UX fix shipped in this PR**: if `--experimental`
is passed but the manifest at the resolved ref exposes no experimental
skills, a warning is logged pointing users at
`DATABRICKS_SKILLS_REF=main`.
2. ~~**Collision handling is naive.**~~ **Resolved.** Every experimental
skill gets a `-experimental` suffix on its install-side key during
`normalizeManifest`. The manifest key + install dir both carry the
suffix; the `SourceName` field on `SkillMeta` preserves the upstream
repo dir name for fetch URLs. Users see at a glance which installed
skills are experimental.

Also handled: **experimental↔stable transitions**. If a skill flips its
experimental status upstream (the same logical skill changes manifest
key), `install` removes the stale variant on disk + state before
installing the new one, and `uninstall` accepts either variant name (and
removes both if both are present). Helper: `alternateVariantKey()`.
Covered by tests `TestInstallReplacesAlternateVariant`,
`TestUninstallByEitherVariantRemovesBoth`,
`TestUninstallByAlternateNameWhenOnlyOneVariantInstalled`.
3. ~~**`list` UX.**~~ **Resolved.** `aitools skills list` shows
experimental skills with an `[experimental]` tag in the NAME column
(driven by `meta.IsExperimental()`). Combined with the TODO #2
resolution (`-experimental` suffix in the manifest key), every
experimental row reads e.g. `databricks-iceberg-experimental
[experimental]` — slightly redundant but a clear visual anchor.
Hide-by-default was considered but rejected: users running `list` are
usually looking for what's available, and silently omitting experimental
skills makes them un-discoverable.
4. ~~**State tracking.**~~ **Resolved — kept additive semantics.**
`InstallState.IncludeExperimental` records what was last requested but
is not used to drive retroactive removal. Running `install` without
`--experimental` leaves previously-installed experimental skills in
place. Rationale: (a) users running `install` are typically
adding/updating, not declaring set membership; (b) silently uninstalling
things the user previously asked for is surprising; (c) the transition
cleanup shipped under TODO #2 handles the actual drift case (skill's
experimental status flipping upstream). Removal is what `uninstall` is
for.
5. ~~**No acceptance test yet.**~~ **Resolved.** Added acceptance tests
under `acceptance/experimental/aitools/skills/install*/` covering the
install flow against a mocked manifest server:
   - Stable-only install (no flag) → 1 skill installed
- `--experimental` install adds the experimental skill (with
`-experimental` suffix per the install-path model) → 2 skills total
   - Re-running `--experimental` is idempotent
- Specific-skill install (`install --skills <name>`) for both stable and
experimental
- `--experimental` against a manifest with no experimental entries logs
a nudge

To make these reachable, exposed a new env-var override
`DATABRICKS_SKILLS_BASE_URL` that overrides the hard-coded
`raw.githubusercontent.com` base URL used by
`GitHubManifestSource.FetchManifest` and `fetchSkillFile`. Defaults to
the canonical URL when unset, so no production behavior change. Updated
`Taskfile.yml`'s `test-exp-aitools` task to include
`acceptance/experimental/aitools/**`.

Variants left as follow-up acceptance tests (the structure is now in
place):
- Variant transition cleanup (stable → experimental, experimental →
stable)
   - Uninstall flow (with both variants installed)
6. ~~**`--experimental` flag scope.**~~ **Resolved — kept current
scope.** Each command has internally consistent behavior:
- `install --experimental` → explicit opt-in (required to install
experimental skills).
- `update` → state-driven (honors `InstallState.IncludeExperimental`
from the last `install`). If you opted in once, future updates refresh
experimentals; otherwise they're skipped.
- `list` → shows all skills with an `[experimental]` tag (no filtering —
discovery first, opt-in to install).

Adding `--experimental` / `--no-experimental` to `update` for one-off
overrides was considered but rejected: the natural workflow is to re-run
`install --experimental` (or just `install`), which already sets the
desired state. Follow-up if real users hit a use case for the override.
7. ~~**Manifest shape.**~~ **Resolved.** Replaced the original two-map
design (`skills` + `experimental_skills` + a per-skill `experimental`
bool) with a single `skills` map where each entry's `repo_dir`
(`"skills"` or `"experimental"`) is the source of truth. The cli derives
experimental state from `RepoDir` via `SkillMeta.IsExperimental()`.
Collisions between stable and experimental skills with the same repo dir
name must be resolved upstream in d-a-s (which they already are — d-a-s
PR #73's TODO #1a merged the only known collision into stable). The
d-a-s manifest generator should be updated to emit `repo_dir` per skill;
until then `normalizeManifest` defaults a missing `RepoDir` to
`"skills"` so older manifests still parse.

## Test plan

- [x] `go build ./...` passes.
- [x] `go test ./experimental/aitools/...` passes (`source_test.go`
covers the normalize/IsExperimental cases).
- [x] `go test ./acceptance -run TestAccept/experimental/aitools` passes
(a pre-existing flake intermittently surfaces an `lstat` warning during
copyDir, ~10% of multi-test runs; unrelated to this refactor).
- [ ] Run `./task lint` and `./task fmt` before merge.
- [ ] Manual: against a d-a-s ref containing experimental entries with
`repo_dir`, verify the four UX cases above behave correctly.

This pull request and its description were written by Claude.

---------

Co-authored-by: simon <4305831+simonfaltum@users.noreply.github.com>
Co-authored-by: simon <simon.faltum@databricks.com>

Author: jamesbroadhead

Taskfile.yml

@@ -615,6 +615,7 @@ tasks:
       - libs/aitools/**
       - experimental/aitools/**
       - acceptance/apps/**
+      - acceptance/experimental/aitools/**
       - "{{.EMBED_SOURCES}}"
     cmds:
       - |
@@ -628,7 +629,7 @@ tasks:
           --format ${GOTESTSUM_FORMAT:-pkgname-and-test-fails} \
           --no-summary=skipped \
           --packages ./acceptance/... \
-          -- -timeout=${LOCAL_TIMEOUT:-30m} -run "TestAccept/apps"
+          -- -timeout=${LOCAL_TIMEOUT:-30m} -run "TestAccept/(apps|experimental/aitools)"
 
   test-exp-ssh:
     desc: Run experimental SSH unit and acceptance tests

acceptance/experimental/aitools/skills/install-experimental-empty/out.test.toml

@@ -0,0 +1,9 @@
+
+=== --experimental against a manifest with no experimental skills logs a nudge
+>>> [CLI] experimental aitools install --global --experimental
+Command "install" is deprecated, use "databricks aitools install" instead.
+Flag --global has been deprecated, use --scope=global
+Installing Databricks AI skills for Claude Code...
+Using skills version test-ref
+Warn: --experimental was set but the manifest at test-ref exposes no experimental skills. Set DATABRICKS_SKILLS_REF to a release that includes them (or =main for the latest).
+Installed 1 skill.

acceptance/experimental/aitools/skills/install-experimental-empty/output.txt

@@ -0,0 +1,5 @@
+# Agent detection needs ~/.claude; prefer USERPROFILE on Windows.
+mkdir -p "${USERPROFILE:-$HOME}/.claude"
+
+title "--experimental against a manifest with no experimental skills logs a nudge"
+trace $CLI experimental aitools install --global --experimental

acceptance/experimental/aitools/skills/install-experimental-empty/script

@@ -0,0 +1,31 @@
+Env.DATABRICKS_SKILLS_BASE_URL = "$DATABRICKS_HOST"
+Env.DATABRICKS_SKILLS_REF = "test-ref"
+
+Ignore = [
+    ".databricks/aitools/skills/.state.json",
+]
+
+[EnvMatrix]
+DATABRICKS_BUNDLE_ENGINE = []
+
+# Stable-only manifest (no repo_dir=experimental).
+[[Server]]
+Pattern = "GET /test-ref/manifest.json"
+Response.Body = '''
+{
+  "version": "2",
+  "updated_at": "2026-01-01T00:00:00Z",
+  "skills": {
+    "test-stable": {"version": "1.0.0", "files": ["SKILL.md"]}
+  }
+}
+'''
+
+[[Server]]
+Pattern = "GET /test-ref/skills/test-stable/SKILL.md"
+Response.Body = '''---
+name: test-stable
+---
+
+# Stable
+'''

acceptance/experimental/aitools/skills/install-experimental-empty/test.toml

@@ -0,0 +1,26 @@
+
+=== install only one specific stable skill via --skills
+>>> [CLI] experimental aitools install --global --skills test-stable-a
+Command "install" is deprecated, use "databricks aitools install" instead.
+Flag --global has been deprecated, use --scope=global
+Installing Databricks AI skills for Claude Code...
+Using skills version test-ref
+Installed 1 skill.
+
+=== install a specific experimental skill
+>>> [CLI] experimental aitools install --global --skills test-exp --experimental
+Command "install" is deprecated, use "databricks aitools install" instead.
+Flag --global has been deprecated, use --scope=global
+Installing Databricks AI skills for Claude Code...
+Using skills version test-ref
+Installed 1 skill.
+
+=== asking for an experimental skill without --experimental flag errors out
+>>> [CLI] experimental aitools install --global --skills test-exp
+Command "install" is deprecated, use "databricks aitools install" instead.
+Flag --global has been deprecated, use --scope=global
+Installing Databricks AI skills for Claude Code...
+Using skills version test-ref
+Error: skill "test-exp" is experimental; use --experimental to install
+
+Exit code: 1

acceptance/experimental/aitools/skills/install-specific/out.test.toml

@@ -0,0 +1,11 @@
+# Agent detection needs ~/.claude; prefer USERPROFILE on Windows.
+mkdir -p "${USERPROFILE:-$HOME}/.claude"
+
+title "install only one specific stable skill via --skills"
+trace $CLI experimental aitools install --global --skills test-stable-a
+
+title "install a specific experimental skill"
+trace $CLI experimental aitools install --global --skills test-exp --experimental
+
+title "asking for an experimental skill without --experimental flag errors out"
+errcode trace $CLI experimental aitools install --global --skills test-exp

acceptance/experimental/aitools/skills/install-specific/output.txt

@@ -0,0 +1,50 @@
+Env.DATABRICKS_SKILLS_BASE_URL = "$DATABRICKS_HOST"
+Env.DATABRICKS_SKILLS_REF = "test-ref"
+
+Ignore = [
+    ".databricks/aitools/skills/.state.json",
+]
+
+[EnvMatrix]
+DATABRICKS_BUNDLE_ENGINE = []
+
+[[Server]]
+Pattern = "GET /test-ref/manifest.json"
+Response.Body = '''
+{
+  "version": "2",
+  "updated_at": "2026-01-01T00:00:00Z",
+  "skills": {
+    "test-stable-a": {"version": "1.0.0", "files": ["SKILL.md"], "repo_dir": "skills"},
+    "test-stable-b": {"version": "1.0.0", "files": ["SKILL.md"], "repo_dir": "skills"},
+    "test-exp":      {"version": "0.0.1", "files": ["SKILL.md"], "repo_dir": "experimental"}
+  }
+}
+'''
+
+[[Server]]
+Pattern = "GET /test-ref/skills/test-stable-a/SKILL.md"
+Response.Body = '''---
+name: test-stable-a
+---
+
+# A
+'''
+
+[[Server]]
+Pattern = "GET /test-ref/skills/test-stable-b/SKILL.md"
+Response.Body = '''---
+name: test-stable-b
+---
+
+# B
+'''
+
+[[Server]]
+Pattern = "GET /test-ref/experimental/test-exp/SKILL.md"
+Response.Body = '''---
+name: test-exp
+---
+
+# Exp
+'''