Skip to content

Commit 8d1a339

Browse files
suryaiyer95claude
andcommitted
feat: add altimate-dbt CLI package and consolidate dbt skills
- New `@altimateai/dbt-tools` package with `altimate-dbt` CLI binary - Commands: init, doctor, compile, build, run, test, execute, columns, graph, deps - Auto-detects dbt project, Python environment, and adapter type - Friendly error diagnostics with actionable fix suggestions - Consolidate 6 old skills into 5 reference-backed skills: dbt-develop, dbt-test, dbt-docs, dbt-troubleshoot, dbt-analyze - Update builder prompt to prefer `altimate-dbt` CLI over raw `dbt_run` tool - Remove orphaned `dbt-run.ts` tool file and unused `python-bridge` patch - Fix missing `await` on `children`/`parents` graph commands - Fix `--limit 0` to correctly support schema inspection - Wrap `adapter.dispose()` in try-catch to prevent masking command errors - Skip stub config tests pending rewrite Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 3cf8656 commit 8d1a339

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

54 files changed

+3129
-803
lines changed
Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
---
2+
name: dbt-analyze
3+
description: Analyze downstream impact of dbt model changes using column-level lineage and the dependency graph. Use when evaluating the blast radius of a change before shipping. Powered by altimate-dbt.
4+
---
5+
6+
# dbt Impact Analysis
7+
8+
## Requirements
9+
**Agent:** any (read-only analysis)
10+
**Tools used:** bash (runs `altimate-dbt` commands), read, glob, dbt_manifest, lineage_check, sql_analyze
11+
12+
## When to Use This Skill
13+
14+
**Use when the user wants to:**
15+
- Understand what breaks if they change a model
16+
- Evaluate downstream impact before shipping
17+
- Find all consumers of a model or column
18+
- Assess risk of a refactoring
19+
20+
**Do NOT use for:**
21+
- Creating or fixing models → use `dbt-develop` or `dbt-troubleshoot`
22+
- Adding tests → use `dbt-test`
23+
24+
## Workflow
25+
26+
### 1. Identify the Changed Model
27+
28+
Accept from the user, or auto-detect:
29+
```bash
30+
# From git diff
31+
git diff --name-only | grep '\.sql$'
32+
33+
# Or user provides a model name
34+
altimate-dbt compile --model <name> # verify it exists
35+
```
36+
37+
### 2. Map the Dependency Graph
38+
39+
```bash
40+
altimate-dbt children --model <name> # direct downstream
41+
altimate-dbt parents --model <name> # what feeds it
42+
```
43+
44+
For the full downstream tree, recursively call `children` on each downstream model.
45+
46+
### 3. Run Column-Level Lineage
47+
48+
Use the `lineage_check` tool on the changed model's SQL to understand:
49+
- Which source columns flow to which output columns
50+
- Which columns were added, removed, or renamed
51+
52+
### 4. Cross-Reference with Downstream
53+
54+
For each downstream model:
55+
1. Read its SQL
56+
2. Check if it references any changed/removed columns
57+
3. Classify impact:
58+
59+
| Classification | Meaning | Action |
60+
|---------------|---------|--------|
61+
| **BREAKING** | Removed/renamed column used downstream | Must fix before shipping |
62+
| **SAFE** | Added column, no downstream reference | Ship freely |
63+
| **UNKNOWN** | Can't determine (dynamic SQL, macros) | Manual review needed |
64+
65+
### 5. Generate Impact Report
66+
67+
```
68+
Impact Analysis: stg_orders
69+
════════════════════════════
70+
71+
Changed Model: stg_orders (materialized: view)
72+
Columns: 5 → 6 (+1 added)
73+
Removed: total_amount (renamed to order_total)
74+
75+
Downstream Impact (3 models):
76+
77+
Depth 1:
78+
[BREAKING] int_order_metrics
79+
Uses: total_amount → COLUMN RENAMED
80+
Fix: Update column reference to order_total
81+
82+
[SAFE] int_order_summary
83+
No references to changed columns
84+
85+
Depth 2:
86+
[BREAKING] mart_revenue
87+
Uses: total_amount via int_order_metrics → CASCADING
88+
Fix: Verify after fixing int_order_metrics
89+
90+
Tests at Risk: 4
91+
- not_null_stg_orders_order_total
92+
- unique_int_order_metrics_order_id
93+
94+
Summary: 2 BREAKING, 1 SAFE
95+
Recommended: Fix int_order_metrics first, then:
96+
altimate-dbt build --model stg_orders --downstream
97+
```
98+
99+
## Without Manifest (SQL-Only Mode)
100+
101+
If no manifest is available:
102+
1. Run `lineage_check` on the changed SQL
103+
2. Show column-level data flow
104+
3. Note: downstream impact requires a manifest
105+
4. Suggest: `altimate-dbt build-project` to generate one
106+
107+
## Common Mistakes
108+
109+
| Mistake | Fix |
110+
|---------|-----|
111+
| Only checking direct children | Always trace the FULL downstream tree recursively |
112+
| Ignoring test impacts | Check which tests reference changed columns |
113+
| Shipping without building downstream | Always `altimate-dbt build --model <name> --downstream` |
114+
| Not considering renamed columns | A rename is a break + add — downstream still references the old name |
115+
116+
## Reference Guides
117+
118+
| Guide | Use When |
119+
|-------|----------|
120+
| [references/altimate-dbt-commands.md](references/altimate-dbt-commands.md) | Need the full CLI reference |
121+
| [references/lineage-interpretation.md](references/lineage-interpretation.md) | Understanding lineage output |
Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
# altimate-dbt Command Reference
2+
3+
All dbt operations use the `altimate-dbt` CLI. Output is JSON to stdout; logs go to stderr.
4+
5+
```bash
6+
altimate-dbt <command> [args...]
7+
altimate-dbt <command> [args...] --format text # Human-readable output
8+
```
9+
10+
## First-Time Setup
11+
12+
```bash
13+
altimate-dbt init # Auto-detect project root
14+
altimate-dbt init --project-root /path # Explicit root
15+
altimate-dbt init --python-path /path # Override Python
16+
altimate-dbt doctor # Verify setup
17+
altimate-dbt info # Project name, adapter, root
18+
```
19+
20+
## Build & Run
21+
22+
```bash
23+
altimate-dbt build --model <name> [--downstream] # compile + run + test
24+
altimate-dbt run --model <name> [--downstream] # materialize only
25+
altimate-dbt test --model <name> # run tests only
26+
altimate-dbt build-project # full project build
27+
```
28+
29+
## Compile
30+
31+
```bash
32+
altimate-dbt compile --model <name>
33+
altimate-dbt compile-query --query "SELECT * FROM {{ ref('stg_orders') }}" [--model <context>]
34+
```
35+
36+
## Execute SQL
37+
38+
```bash
39+
altimate-dbt execute --query "SELECT count(*) FROM {{ ref('orders') }}" --limit 100
40+
```
41+
42+
## Schema & DAG
43+
44+
```bash
45+
altimate-dbt columns --model <name> # column names and types
46+
altimate-dbt columns-source --source <src> --table <tbl> # source table columns
47+
altimate-dbt column-values --model <name> --column <col> # sample values
48+
altimate-dbt children --model <name> # downstream models
49+
altimate-dbt parents --model <name> # upstream models
50+
```
51+
52+
## Packages
53+
54+
```bash
55+
altimate-dbt deps # install packages.yml
56+
altimate-dbt add-packages --packages dbt-utils,dbt-expectations
57+
```
58+
59+
## Error Handling
60+
61+
All errors return JSON with `error` and `fix` fields:
62+
```json
63+
{ "error": "dbt-core is not installed", "fix": "Install it: python3 -m pip install dbt-core" }
64+
```
65+
66+
Run `altimate-dbt doctor` as the first diagnostic step for any failure.
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Lineage Interpretation Guide
2+
3+
## Understanding Column-Level Lineage
4+
5+
Column-level lineage traces how data flows from source columns through transformations to output columns.
6+
7+
### Direct Lineage
8+
```
9+
source.customers.name → stg_customers.customer_name → dim_customers.full_name
10+
```
11+
Column was renamed at each step. A change to the source column affects all downstream.
12+
13+
### Aggregation Lineage
14+
```
15+
source.orders.amount → (SUM) → fct_daily_revenue.total_revenue
16+
```
17+
Multiple source rows feed into one output value. The column type changes from row-level to aggregate.
18+
19+
### Conditional Lineage
20+
```
21+
source.orders.status → (CASE WHEN) → fct_orders.is_completed
22+
```
23+
The source column feeds a derived boolean. The relationship is logical, not direct.
24+
25+
## Impact Classification
26+
27+
### BREAKING Changes
28+
- **Column removed**: Downstream models referencing it will fail
29+
- **Column renamed**: Same as removed — downstream still uses the old name
30+
- **Type changed**: May cause cast errors or silent data loss downstream
31+
- **Logic changed**: Downstream aggregations/filters may produce wrong results
32+
33+
### SAFE Changes
34+
- **Column added**: No downstream model can reference what didn't exist
35+
- **Description changed**: No runtime impact
36+
- **Test added/modified**: No impact on model data
37+
38+
### REQUIRES REVIEW
39+
- **Filter changed**: May change which rows appear → downstream counts change
40+
- **JOIN type changed**: LEFT→INNER drops rows, INNER→LEFT adds NULLs
41+
- **Materialization changed**: view→table has no logical impact but affects freshness
42+
43+
## Reading the DAG
44+
45+
```bash
46+
altimate-dbt parents --model <name> # what this model depends on
47+
altimate-dbt children --model <name> # what depends on this model
48+
```
49+
50+
A model with many children has high blast radius. A model with many parents has high complexity.
51+
52+
## Depth Matters
53+
54+
- **Depth 1**: Direct consumers — highest risk, most likely to break
55+
- **Depth 2+**: Cascading impact — will break IF depth 1 breaks
56+
- **Depth 3+**: Usually only affected by breaking column removals/renames
57+
58+
Focus investigation on depth 1 first.
Lines changed: 147 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,147 @@
1+
---
2+
name: dbt-develop
3+
description: Create and modify dbt models — staging, intermediate, marts, incremental, medallion architecture. Use when building new SQL models, extending existing ones, scaffolding YAML configs, or reorganizing project structure. Powered by altimate-dbt.
4+
---
5+
6+
# dbt Model Development
7+
8+
## Requirements
9+
**Agent:** builder or migrator (requires file write access)
10+
**Tools used:** bash (runs `altimate-dbt` commands), read, glob, write, edit
11+
12+
## When to Use This Skill
13+
14+
**Use when the user wants to:**
15+
- Create a new dbt model (staging, intermediate, mart, OBT)
16+
- Add or modify SQL logic in an existing model
17+
- Generate sources.yml or schema.yml from warehouse metadata
18+
- Reorganize models into layers (staging/intermediate/mart or bronze/silver/gold)
19+
- Convert a model to incremental materialization
20+
- Scaffold a new dbt project structure
21+
22+
**Do NOT use for:**
23+
- Adding tests to models → use `dbt-test`
24+
- Writing model/column descriptions → use `dbt-docs`
25+
- Debugging build failures → use `dbt-troubleshoot`
26+
- Analyzing change impact → use `dbt-analyze`
27+
28+
## Core Workflow: Plan → Discover → Write → Validate
29+
30+
### 1. Plan — Understand Before Writing
31+
32+
Before writing any SQL:
33+
- Read the task requirements carefully
34+
- Identify which layer this model belongs to (staging, intermediate, mart)
35+
- Check existing models for naming conventions and patterns
36+
- **Check dependencies:** If `packages.yml` exists, check for `dbt_packages/` or `package-lock.yml`. Only run `dbt deps` if packages are declared but not yet installed.
37+
38+
```bash
39+
altimate-dbt info # project name, adapter type
40+
altimate-dbt parents --model <upstream> # understand what feeds this model
41+
altimate-dbt children --model <downstream> # understand what consumes it
42+
```
43+
44+
### 2. Discover — Understand the Data Before Writing
45+
46+
**Never write SQL without deeply understanding your data first.** The #1 cause of wrong results is writing SQL blind — assuming grain, relationships, column names, or values without checking.
47+
48+
**Step 2a: Read all documentation and schema definitions**
49+
- Read `sources.yml`, `schema.yml`, and any YAML files that describe the source/parent models
50+
- These contain column descriptions, data types, tests, and business context
51+
- Pay special attention to: primary keys, unique constraints, relationships between tables, and what each column represents
52+
53+
**Step 2b: Understand the grain of each parent model/source**
54+
- What does one row represent? (one customer? one event? one day per customer?)
55+
- What are the primary/unique keys?
56+
- This is critical for JOINs — joining on the wrong grain causes fan-out (too many rows) or missing rows
57+
58+
```bash
59+
altimate-dbt columns --model <name> # existing model columns
60+
altimate-dbt columns-source --source <src> --table <tbl> # source table columns
61+
altimate-dbt execute --query "SELECT count(*) FROM {{ ref('model') }}" --limit 1
62+
altimate-dbt execute --query "SELECT * FROM {{ ref('model') }}" --limit 5
63+
altimate-dbt column-values --model <name> --column <col> # sample values for key columns
64+
```
65+
66+
**Step 2c: Query the actual data to verify your understanding**
67+
- Check row counts, NULLs, date ranges, cardinality of key columns
68+
- Verify foreign key relationships actually hold (do all IDs in child exist in parent?)
69+
- Check for duplicates in what you think are unique keys
70+
71+
**Step 2d: Read existing models that your new model will reference**
72+
- Read the actual SQL of parent models — understand their logic, filters, and transformations
73+
- Read 2-3 existing models in the same directory to match patterns and conventions
74+
75+
```bash
76+
glob models/**/*.sql # find all model files
77+
read <model_file> # understand existing patterns and logic
78+
```
79+
80+
### 3. Write — Follow Layer Patterns
81+
82+
See [references/layer-patterns.md](references/layer-patterns.md) for staging/intermediate/mart templates.
83+
See [references/medallion-architecture.md](references/medallion-architecture.md) for bronze/silver/gold patterns.
84+
See [references/incremental-strategies.md](references/incremental-strategies.md) for incremental materialization.
85+
See [references/yaml-generation.md](references/yaml-generation.md) for sources.yml and schema.yml.
86+
87+
### 4. Validate — Build, Verify, Check Impact
88+
89+
Never stop at writing the SQL. Always validate:
90+
91+
**Build it:**
92+
```bash
93+
altimate-dbt compile --model <name> # catch Jinja errors
94+
altimate-dbt build --model <name> # materialize + run tests
95+
```
96+
97+
**Verify the output:**
98+
```bash
99+
altimate-dbt columns --model <name> # confirm expected columns exist
100+
altimate-dbt execute --query "SELECT count(*) FROM {{ ref('<name>') }}" --limit 1
101+
altimate-dbt execute --query "SELECT * FROM {{ ref('<name>') }}" --limit 10 # spot-check values
102+
```
103+
- Do the columns match what schema.yml or the task expects?
104+
- Does the row count make sense? (no fan-out from bad joins, no missing rows from wrong filters)
105+
- Are values correct? (spot-check NULLs, aggregations, date ranges)
106+
107+
**Check SQL quality** (on the compiled SQL from `altimate-dbt compile`):
108+
- `sql_analyze` — catches anti-patterns (SELECT *, cartesian products, missing filters)
109+
- `altimate_core_validate` — validates syntax and schema references
110+
- `altimate_core_column_lineage` — traces how source columns flow to output columns. Use this to verify your SELECT is pulling the right columns from the right sources, especially for complex JOINs or multi-CTE models.
111+
112+
**Check downstream impact** (when modifying an existing model):
113+
```bash
114+
altimate-dbt children --model <name> # who depends on this?
115+
altimate-dbt build --model <name> --downstream # rebuild downstream to catch breakage
116+
```
117+
Use `altimate-dbt children` and `altimate-dbt parents` to verify the DAG is intact when changes could affect downstream models.
118+
119+
## Iron Rules
120+
121+
1. **Never write SQL without reading the source columns first.** Use `altimate-dbt columns` or `altimate-dbt columns-source`.
122+
2. **Never stop at compile.** Always `altimate-dbt build` to catch runtime errors.
123+
3. **Match existing patterns.** Read 2-3 existing models in the same directory before writing.
124+
4. **One model, one purpose.** A staging model should not contain business logic. An intermediate model should not be materialized as a table unless it has consumers.
125+
5. **Fix ALL errors, not just yours.** After creating/modifying models, run a full `dbt build`. If ANY model fails — even pre-existing ones you didn't touch — fix them. Your job is to leave the project in a fully working state.
126+
127+
## Common Mistakes
128+
129+
| Mistake | Fix |
130+
|---------|-----|
131+
| Writing SQL without checking column names | Run `altimate-dbt columns` or `altimate-dbt columns-source` first |
132+
| Stopping at `compile` — "it compiled, ship it" | Always `altimate-dbt build` to materialize and run tests |
133+
| Hardcoding table references instead of `{{ ref() }}` | Always use `{{ ref('model') }}` or `{{ source('src', 'table') }}` |
134+
| Creating a staging model with JOINs | Staging = 1:1 with source. JOINs belong in intermediate or mart |
135+
| Not checking existing naming conventions | Read existing models in the same directory first |
136+
| Using `SELECT *` in final models | Explicitly list columns for clarity and contract stability |
137+
138+
## Reference Guides
139+
140+
| Guide | Use When |
141+
|-------|----------|
142+
| [references/altimate-dbt-commands.md](references/altimate-dbt-commands.md) | Need the full CLI reference |
143+
| [references/layer-patterns.md](references/layer-patterns.md) | Creating staging, intermediate, or mart models |
144+
| [references/medallion-architecture.md](references/medallion-architecture.md) | Organizing into bronze/silver/gold layers |
145+
| [references/incremental-strategies.md](references/incremental-strategies.md) | Converting to incremental materialization |
146+
| [references/yaml-generation.md](references/yaml-generation.md) | Generating sources.yml or schema.yml |
147+
| [references/common-mistakes.md](references/common-mistakes.md) | Extended anti-patterns catalog |

0 commit comments

Comments
 (0)