databricks
diff --git a/‎.github/CODEOWNERS‎
Lines changed: 3 additions & 0 deletions b/‎.github/CODEOWNERS‎
Lines changed: 3 additions & 0 deletions
diff --git a/‎.github/workflows/validate-manifest.yml‎
Lines changed: 1 addition & 0 deletions b/‎.github/workflows/validate-manifest.yml‎
Lines changed: 1 addition & 0 deletions
diff --git a/‎README.md‎
Lines changed: 47 additions & 14 deletions b/‎README.md‎
Lines changed: 47 additions & 14 deletions
diff --git a/‎experimental/README.md‎
Lines changed: 83 additions & 0 deletions b/‎experimental/README.md‎
Lines changed: 83 additions & 0 deletions
diff --git a/‎experimental/databricks-agent-bricks/1-knowledge-assistants.md‎
Lines changed: 74 additions & 0 deletions b/‎experimental/databricks-agent-bricks/1-knowledge-assistants.md‎
Lines changed: 74 additions & 0 deletions
diff --git a/‎experimental/databricks-agent-bricks/2-supervisor-agents.md‎
Lines changed: 107 additions & 0 deletions b/‎experimental/databricks-agent-bricks/2-supervisor-agents.md‎
Lines changed: 107 additions & 0 deletions
@@ -6,4 +6,7 @@
 /skills/databricks-model-serving/ @databricks/eng-apps-devex
 /skills/databricks-jobs/ @lennartkats-db @camielstee-db
 /skills/databricks-pipelines/ @lennartkats-db @camielstee-db
+
+/experimental/ @lennartkats-db @simonfaltum @calreynolds @dustinvannoy-db
+
 /.github/CODEOWNERS @lennartkats-db @simonfaltum @fjakobs
@@ -4,6 +4,7 @@ on:
   pull_request:
     paths:
       - 'skills/**'
+      - 'experimental/**'
       - 'assets/**'
       - 'scripts/skills.py'
       - 'manifest.json'
 
@@ -4,25 +4,55 @@ Skills for AI coding assistants (Claude Code, Cursor, etc.) that provide Databri
 
 ## Installation
 
-**For Claude Code:**
-
 ```bash
 databricks experimental aitools install
 ```
 
-This installs skills to `~/.claude/skills/` for use with Claude Code.
+This auto-detects your coding agent(s) and installs the stable skills to the
+right location:
+
+- **Claude Code** → `~/.claude/skills/`
+- **Cursor**, **Codex CLI**, **OpenCode**, **GitHub Copilot**, **Antigravity**
+  → their respective skill directories
 
-**For Cursor:**
+For finer control, use the `aitools skills install` subcommand directly — it
+accepts a positional skill name and an `--experimental` flag (see the
+[Experimental Skills](#experimental-skills) section).
 
-Run this command in chat:
+**For Cursor (plugin marketplace alternative):**
 
 ```text
 /add-plugin databricks-skills
 ```
 
 ## Available Skills
 
-- **databricks-apps** - Build full-stack TypeScript apps on Databricks using AppKit
+Stable skills shipped from [`skills/`](./skills/):
+
+- **databricks-core** — CLI, authentication, profile selection, data exploration. Parent skill for all product skills.
+- **databricks-apps** — Build full-stack TypeScript apps on Databricks using AppKit.
+- **databricks-dabs** — Declarative Automation Bundles (formerly Asset Bundles) for deploying and managing Databricks resources.
+- **databricks-jobs** — Lakeflow Jobs orchestration: task types, triggers, schedules, notifications.
+- **databricks-lakebase** — Lakebase Postgres: projects, branching, autoscaling, synced tables, Data API.
+- **databricks-model-serving** — Model Serving endpoint management, AI Gateway, traffic config.
+- **databricks-pipelines** — Lakeflow Spark Declarative Pipelines (formerly DLT) for batch and streaming.
+- **databricks-serverless-migration** — Migrate classic-compute workloads to serverless compute.
+
+## Experimental Skills
+
+The [`experimental/`](./experimental/) directory contains additional skills
+imported from [databricks-solutions/ai-dev-kit](https://github.com/databricks-solutions/ai-dev-kit)
+on a **best-effort basis**.
+
+- Experimental skills are **not officially supported** — they may be used, but
+  do not follow the same review / quality bar as the stable skills under
+  [`skills/`](./skills/).
+- They are **not installed by default** by `databricks aitools install`.
+  Pass `--experimental` to install all of them, or install a specific one
+  by name (with the `--experimental` flag — e.g. `databricks aitools install
+  databricks-iceberg --experimental`).
+- See [`experimental/README.md`](./experimental/README.md) for the full list
+  and caveats.
 
 ## Structure
 
@@ -38,21 +68,24 @@ skill-name/
 
 ### Adding New Skills
 
-When experimenting with new skill variations, create a "subskill" that references the main skill and adds specific guidance:
+For a narrower variation of an existing skill, create a subskill that declares
+its parent via frontmatter. This is how the stable skills are organized today
+— each product skill sets `parent: databricks-core`.
 
 ```markdown
 ---
-name: "ai-databricks-apps"
-description: "Databricks apps with AI features"
+name: "databricks-apps-chatbots"
+description: "Databricks apps with chatbot features"
+parent: databricks-apps
 ---
 
-# AI powered Databricks Apps
+# Chatbot Apps
 
-First, load the base databricks-apps skill for foundational guidance.
+**FIRST**: Use the parent `databricks-apps` skill for app development basics.
 
-Then apply these additional patterns:
-- Custom pattern 1
-- Custom pattern 2
+Then apply these patterns:
+- Pattern 1
+- Pattern 2
 ```
 
 This approach:
 
@@ -0,0 +1,83 @@
+> ⚠️ **Experimental: best-effort, not officially supported**
+>
+> The skills in this directory are imported from
+> [databricks-solutions/ai-dev-kit](https://github.com/databricks-solutions/ai-dev-kit)
+> on a best-effort basis. They may be useful, but they are **not officially
+> supported** as part of `databricks-agent-skills`:
+>
+> - They do not follow the same review / quality bar as the skills in
+>   [`../skills/`](../skills/).
+> - They may be out of date relative to upstream `ai-dev-kit`.
+> - They are not installed by `databricks aitools install` by default —
+>   you have to opt in (see the root README).
+>
+> File issues against this directory in this repo; do not file issues against
+> `ai-dev-kit` for skills installed via `databricks-agent-skills`.
+
+---
+
+# Databricks Skills for Claude Code
+
+Skills that teach Claude Code how to work effectively with Databricks - providing patterns, best practices, and code examples that work with Databricks MCP tools.
+
+## Installation
+
+These experimental skills are **not** installed by default. To install them via the Databricks CLI:
+
+```bash
+# Install all experimental skills at once
+databricks aitools install --experimental
+
+# Install a single experimental skill by name
+databricks aitools install databricks-iceberg --experimental
+```
+
+See the root [README](../README.md) for details on the stable install path.
+
+## Available Skills
+
+### 🤖 AI & Agents
+- **databricks-ai-functions** - Built-in AI Functions (ai_classify, ai_extract, ai_summarize, ai_query, ai_forecast, ai_parse_document, and more) with SQL and PySpark patterns, function selection guidance, document processing pipelines, and custom RAG (parse → chunk → index → query)
+- **databricks-agent-bricks** - Knowledge Assistants, Genie Spaces, Supervisor Agents
+- **databricks-mlflow-evaluation** - End-to-end agent evaluation workflow
+- **databricks-unstructured-pdf-generation** - Generate synthetic PDFs for RAG
+- **databricks-vector-search** - Vector similarity search for RAG and semantic search
+
+### 📊 Analytics & Dashboards
+- **databricks-aibi-dashboards** - Databricks AI/BI dashboards (with SQL validation workflow)
+- **databricks-metric-views** - Metric Views for governed metrics
+- **databricks-unity-catalog** - System tables for lineage, audit, billing
+
+### 🔧 Data Engineering
+- **databricks-dbsql** - Databricks SQL warehouse patterns
+- **databricks-iceberg** - Apache Iceberg tables (Managed/Foreign), UniForm, Iceberg REST Catalog, Iceberg Clients Interoperability
+- **databricks-spark-structured-streaming** - Spark Structured Streaming patterns
+- **databricks-synthetic-data-gen** - Realistic test data with Faker
+- **databricks-zerobus-ingest** - Zerobus ingest patterns
+- **spark-python-data-source** - Python data sources for Spark
+
+### 🚀 Development & Deployment
+- **databricks-apps-python** - Databricks apps. Prefers AppKit (TypeScript + React SDK) for new apps; falls back to Python frameworks (Dash, Streamlit, Gradio, Flask, FastAPI, Reflex) when Python is required
+- **databricks-python-sdk** - Python SDK, Connect, CLI, REST API
+- **databricks-execution-compute** - Execute on Databricks compute
+
+> **Use the stable skill instead** for:
+> - **DABs / bundles** — use stable [`databricks-dabs`](../skills/databricks-dabs/)
+> - **Lakebase Postgres** (projects, branching, synced tables, autoscaling) — use stable [`databricks-lakebase`](../skills/databricks-lakebase/)
+> - **CLI auth / profiles / workspace config** — use stable [`databricks-core`](../skills/databricks-core/)
+>
+> Previously experimental copies of these (`databricks-bundles`, `databricks-lakebase-autoscale`, `databricks-config`) were already merged with the stable skills.
+
+### 📚 Reference
+- **databricks-docs** - Documentation index via llms.txt
+
+## Provenance
+
+These skills are imported as a snapshot from
+[`databricks-solutions/ai-dev-kit/databricks-skills/`](https://github.com/databricks-solutions/ai-dev-kit/tree/main/databricks-skills).
+
+**Source SHA**: [`20a92a3`](https://github.com/databricks-solutions/ai-dev-kit/commit/20a92a38144ca5228f1dfe4cc0be46da40ec9177)
+on the `experimental` branch of `databricks-solutions/ai-dev-kit`.
+
+While `ai-dev-kit` is the upstream source, this directory receives periodic
+manual re-syncs.
@@ -0,0 +1,74 @@
+# Knowledge Assistants - Details
+
+For commands, see [SKILL.md](SKILL.md).
+
+## Source Types
+
+Both shapes go inside the `--json` body alongside `display_name` and `description` — see SKILL.md for the full invocation.
+
+### Files (Volume)
+
+```json
+{
+  "display_name": "...",
+  "description": "...",
+  "source_type": "files",
+  "files": {"path": "/Volumes/catalog/schema/volume/folder/"}
+}
+```
+
+Supported formats: PDF, TXT, MD, DOCX.
+
+### Vector Search Index
+
+Use an existing index instead of auto-indexing:
+
+```json
+{
+  "display_name": "...",
+  "description": "...",
+  "source_type": "index",
+  "index": {
+    "index_name": "catalog.schema.my_index",
+    "text_col": "content",
+    "doc_uri_col": "source_url"
+  }
+}
+```
+
+## Updating Content
+
+1. Add/modify/remove files in the Volume
+2. Re-sync: `databricks knowledge-assistants sync-knowledge-sources "knowledge-assistants/{ka_id}"`
+
+## Troubleshooting
+
+**KA stays in CREATING:**
+- Wait up to 10 minutes
+- Check workspace quotas
+- Verify volume path exists
+
+**Documents not indexed:**
+- Check file format (PDF, TXT, MD, DOCX)
+- Verify volume path (trailing slash matters)
+- Check file permissions
+
+**Poor answer quality:**
+- Ensure documents are well-structured
+- Break large documents into smaller files
+- Add clear headings and sections
+
+## Evaluation Questions
+
+When testing a KA, check if the volume or project contains a `pdf_eval_questions.json` file with test questions:
+
+```json
+{
+  "api_errors_guide.pdf": {
+    "question": "What is the solution for error ERR-4521?",
+    "expected_fact": "Call /api/v2/auth/refresh with refresh_token before the 3600s TTL expires"
+  }
+}
+```
+
+Use these questions to validate retrieval accuracy. See [databricks-unstructured-pdf-generation](../databricks-unstructured-pdf-generation/SKILL.md) for generating test PDFs with eval questions.
@@ -0,0 +1,107 @@
+# Supervisor Agents - Details
+
+For commands, see [SKILL.md](SKILL.md). All operations use the native `databricks supervisor-agents` CLI (Beta, requires CLI ≥ 0.299.2).
+
+## Unity Catalog Functions
+
+Call registered UC functions from the Supervisor Agent.
+
+**Prerequisites:**
+- UC Function exists (`CREATE FUNCTION` or Python UDF)
+- Grant execute: `GRANT EXECUTE ON FUNCTION catalog.schema.func TO \`<agent_sp>\`;`
+
+**Attach as a tool:**
+```bash
+databricks supervisor-agents create-tool supervisor-agents/<id> enricher --json '{
+    "tool_type": "uc_function",
+    "description": "Enriches customer records",
+    "uc_function": {"name": "catalog.schema.enrich_data"}
+}'
+```
+
+## External MCP Servers
+
+Connect to external systems (ERP, CRM) via UC HTTP Connection implementing MCP protocol.
+
+**1. Create UC HTTP Connection:**
+```sql
+CREATE CONNECTION my_mcp TYPE HTTP
+OPTIONS (
+  host 'https://my-app.databricksapps.com',
+  port '443',
+  base_path '/api/mcp',
+  client_id '<sp_id>',
+  client_secret '<sp_secret>',
+  oauth_scope 'all-apis',
+  token_endpoint 'https://<workspace>.azuredatabricks.net/oidc/v1/token',
+  is_mcp_connection 'true'
+);
+```
+
+**2. Grant access:**
+```sql
+GRANT USE CONNECTION ON my_mcp TO `<agent_sp>`;
+```
+
+**3. Attach as a tool:**
+```bash
+databricks supervisor-agents create-tool supervisor-agents/<id> operations --json '{
+    "tool_type": "uc_connection",
+    "description": "Execute operations: approve invoices, trigger workflows",
+    "uc_connection": {"name": "my_mcp"}
+}'
+```
+
+**Test connection:**
+```sql
+SELECT http_request(conn => 'my_mcp', method => 'POST', path => '', json => '{"jsonrpc":"2.0","method":"tools/list","id":1}');
+```
+
+## Writing Good Descriptions
+
+The `description` field drives routing. Be specific:
+
+| Good | Bad |
+|------|-----|
+| "Handles billing: invoices, payments, refunds, subscriptions" | "Billing agent" |
+| "Answers API errors, integration issues, product bugs" | "Technical" |
+| "HR policies, PTO, benefits, employee handbook" | "Handles stuff" |
+
+## Adding Examples
+
+Examples help evaluation and routing optimization. **The serving endpoint must be ONLINE.** Right after `create-supervisor-agent` (or a structural `update-supervisor-agent`), the endpoint takes **up to ~10 minutes** to come ONLINE. Examples can be added before that — they're stored on the agent definition — but querying the endpoint to evaluate routing requires readiness.
+
+```bash
+# Add an example (guidelines is a repeated string — must use --json)
+databricks supervisor-agents create-example supervisor-agents/<id> --json '{
+    "question": "I need my invoice for March",
+    "guidelines": ["Route to billing_agent"]
+}'
+
+databricks supervisor-agents create-example supervisor-agents/<id> --json '{
+    "question": "API returns 500 error",
+    "guidelines": ["Route to tech_agent"]
+}'
+
+# List / inspect / remove
+databricks supervisor-agents list-examples supervisor-agents/<id>
+databricks supervisor-agents get-example supervisor-agents/<id>/examples/<ex_id>
+databricks supervisor-agents delete-example supervisor-agents/<id>/examples/<ex_id>
+
+# Check endpoint readiness before querying
+databricks serving-endpoints get <endpoint_name>
+```
+
+## Troubleshooting
+
+**Wrong routing:**
+- Improve agent descriptions (more specific, less overlap)
+- Add examples demonstrating correct routing
+
+**Endpoint not responding:**
+- Verify underlying endpoints are running
+- Check endpoint logs
+
+**Slow responses:**
+- Check underlying endpoint latency
+- Review endpoint scaling settings