Merge branch 'main' into fix/hardcoded-snowflake-dialect

Author: VJ-yadav

.claude/commands/add-database-driver.md

@@ -0,0 +1,181 @@
+---
+description: Add a new database driver to Altimate Code. Scaffolds the driver, registers it across all 23 integration points, writes E2E tests, and updates docs. Usage - /add-database-driver <database-name>
+---
+
+# Add Database Driver
+
+Scaffold and fully integrate a new database/warehouse driver into Altimate Code. This command handles all 23 integration points — driver code, registry, discovery, finops, tests, and documentation.
+
+## Input
+
+`$ARGUMENTS` = the database name (e.g., `cockroachdb`, `timescaledb`, `cassandra`, `neo4j`).
+
+If empty, ask: "Which database should I add support for?"
+
+## Step 0: Research
+
+Before writing any code, research the database:
+
+1. **Find the official Node.js/TypeScript client package** on npm. Search for `@{database}/client`, `{database}-js`, or similar.
+2. **Check supported server versions** — which versions are not EOL?
+3. **Identify auth methods** — password, token, TLS/certificate, connection string, cloud-specific?
+4. **Check SQL dialect** — standard SQL? Custom syntax? LIMIT vs TOP vs FETCH FIRST? System tables for schemas/tables/columns?
+5. **Find Docker image** — official image on Docker Hub for E2E testing?
+6. **Check if dbt adapter exists** — search for `dbt-{database}` on PyPI.
+
+Present findings to the user before proceeding:
+```
+## Research: {Database}
+
+- **npm package**: `{package}` (v{version})
+- **Server versions**: {non-EOL versions}
+- **Auth methods**: {list}
+- **SQL dialect**: {notes on LIMIT, system tables, parameterized queries}
+- **Docker image**: `{image}:{tag}`
+- **dbt adapter**: {exists/not found}
+
+Proceed with implementation?
+```
+
+## Step 1: Read Reference Document
+
+Read the comprehensive checklist:
+```bash
+cat packages/drivers/ADDING_A_DRIVER.md
+```
+
+This document has all 23 integration points with exact file paths and code patterns.
+
+## Step 2: Read Existing Driver for Pattern
+
+Read a similar existing driver as a template. Choose based on database type:
+
+- **SQL database with password auth** → read `packages/drivers/src/mysql.ts`
+- **Cloud warehouse with token auth** → read `packages/drivers/src/databricks.ts`
+- **Database with connection string support** → read `packages/drivers/src/postgres.ts`
+- **HTTP-based client** → read `packages/drivers/src/clickhouse.ts`
+- **Document database (non-SQL)** → read `packages/drivers/src/mongodb.ts`
+
+Also read:
+- `packages/drivers/src/normalize.ts` — for alias pattern
+- `packages/opencode/src/altimate/native/connections/registry.ts` — for registration pattern
+- `packages/opencode/test/altimate/drivers-docker-e2e.test.ts` — for E2E test pattern
+
+## Step 3: Implement (23 integration points)
+
+Work through all 9 phases from the checklist. Use parallel edits where possible.
+
+### Phase 1: Core Driver (4 files)
+
+1. **Create `packages/drivers/src/{database}.ts`**
+   - Follow the Connector interface: `connect()`, `execute()`, `listSchemas()`, `listTables()`, `describeTable()`, `close()`
+   - Lazy-import the npm package
+   - Use parameterized queries for schema introspection
+   - Handle LIMIT injection with DML guard: `!hasDML` check before appending LIMIT
+   - Handle TLS detection from connection strings
+
+2. **Add export to `packages/drivers/src/index.ts`**
+
+3. **Add optionalDependency to `packages/drivers/package.json`**
+
+4. **Add aliases to `packages/drivers/src/normalize.ts`**
+
+### Phase 2: Registry (4 files in registry.ts)
+
+5. Add to `DRIVER_MAP`
+6. Add to import switch statement
+7. Add to `PASSWORD_DRIVERS` (if applicable)
+8. Remove from `KNOWN_UNSUPPORTED` (if listed)
+
+### Phase 3: Discovery (4 files)
+
+9. Docker discovery — `docker-discovery.ts` (IMAGE_MAP, ENV_MAP, DEFAULT_PORTS, DEFAULT_USERS)
+10. Env var detection — `project-scan.ts` (detectEnvVars warehouses array)
+11. dbt adapter — `dbt-profiles.ts` (ADAPTER_TYPE_MAP)
+12. dbt lineage — `dbt/lineage.ts` (detectDialect dialectMap)
+
+### Phase 4: FinOps (1 file)
+
+13. Query history — `finops/query-history.ts` (SQL template + handler if database has system query log)
+
+### Phase 5: Build (1 file)
+
+14. Peer deps — `script/publish.ts` (driverPeerDependencies)
+
+### Phase 6: Tool Descriptions (1 file)
+
+15. warehouse_add — `tools/warehouse-add.ts` (config description + error message)
+
+### Phase 7: Tests (2 new files + 1 edit)
+
+16. E2E tests — `test/altimate/drivers-{database}-e2e.test.ts`
+17. Normalization tests — add to `test/altimate/driver-normalize.test.ts`
+18. Verify existing tests pass
+
+### Phase 8: Documentation (5 files)
+
+19. `docs/docs/configure/warehouses.md` — config section + update count
+20. `docs/docs/drivers.md` — support matrix + installation + auth + update count
+21. `docs/docs/data-engineering/tools/warehouse-tools.md` — env vars + Docker
+22. `README.md` — warehouse list
+23. `docs/docs/getting-started/index.md` — homepage list
+
+### Phase 9: Optional
+
+- Guide page at `docs/docs/data-engineering/guides/{database}.md`
+- Update `mkdocs.yml` nav and `guides/index.md`
+- Check fingerprint regex in `fingerprint/index.ts`
+
+## Step 4: Run Quality Gates
+
+```bash
+# Tests (from packages/opencode/)
+cd packages/opencode && bun test test/altimate/driver-normalize.test.ts test/altimate/connections.test.ts test/altimate/drivers-{database}-e2e.test.ts
+
+# Typecheck (from repo root)
+cd "$(git rev-parse --show-toplevel)" && bun turbo typecheck
+
+# Marker check (from repo root)
+bun run script/upstream/analyze.ts --markers --base main --strict
+```
+
+All three must pass before proceeding.
+
+## Step 5: Run Code Review
+
+Run `/consensus:code-review` to get the implementation reviewed by multiple models before committing.
+
+## Step 6: Summary
+
+Present final summary:
+```
+## {Database} Driver Added
+
+### Files Created
+- packages/drivers/src/{database}.ts
+- packages/opencode/test/altimate/drivers-{database}-e2e.test.ts
+- docs/docs/data-engineering/guides/{database}.md (if created)
+
+### Files Modified
+- {list all modified files}
+
+### Test Results
+- {N} normalization tests pass
+- {N} connection tests pass
+- Typecheck: pass
+- Marker check: pass
+
+### E2E Test Coverage
+- {list of test suites and server versions}
+
+Ready to commit.
+```
+
+## Rules
+
+1. **Read before writing.** Always read existing drivers and the reference doc before creating new code.
+2. **Don't skip integration points.** All 23 points exist for a reason — missing one causes inconsistencies users will hit.
+3. **Use parameterized queries** for `listTables` and `describeTable` — never interpolate user input into SQL.
+4. **Test multiple server versions** — at minimum: latest stable + oldest non-EOL LTS.
+5. **Run all quality gates** before presenting the summary.
+6. **Don't modify finops tools** (credit-analyzer, warehouse-advisor, unused-resources) unless the database has equivalent cost/credit APIs.

.github/meta/commit.txt

@@ -1,6 +1 @@
-fix: remove flaky `setTimeout` in todo bus event test
-
-`Bus.publish` is synchronous — the event is delivered immediately,
-no 50ms delay needed. Removes resource contention risk in parallel CI.
-
-Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
+feat: add ClickHouse warehouse driver

.github/workflows/ci.yml

@@ -46,6 +46,7 @@ jobs:
               - 'packages/opencode/test/altimate/drivers-e2e.test.ts'
               - 'packages/opencode/test/altimate/drivers-docker-e2e.test.ts'
               - 'packages/opencode/test/altimate/drivers-mongodb-e2e.test.ts'
+              - 'packages/opencode/test/altimate/drivers-clickhouse-e2e.test.ts'
               - 'packages/opencode/test/altimate/connections.test.ts'
             dbt-tools:
               - 'packages/dbt-tools/**'
@@ -84,13 +85,45 @@ jobs:
         run: bun install
 
       - name: Run tests
-        run: bun test --timeout 30000
         working-directory: packages/opencode
         # Cloud E2E tests (Snowflake, BigQuery, Databricks) auto-skip when
         # ALTIMATE_CODE_CONN_* env vars are not set. Docker E2E tests auto-skip
         # when Docker is not available. No exclusion needed — skipIf handles it.
         # --timeout 30000: matches package.json "test" script; prevents 5s default
         # from cutting off tests that run bun install or bootstrap git instances.
+        #
+        # Bun 1.3.x has a known segfault during process cleanup after all tests
+        # pass (exit code 143/SIGTERM or 134/SIGABRT). We capture test output and
+        # check for real failures vs Bun crashes to avoid false CI failures.
+        shell: bash
+        run: |
+          # Redirect bun output to file, then cat it for CI visibility.
+          # This avoids tee/pipe issues where SIGTERM kills tee before flush.
+          bun test --timeout 30000 > /tmp/test-output.txt 2>&1 || true
+          cat /tmp/test-output.txt
+
+          # Extract pass/fail counts from Bun test summary (e.g., " 5362 pass")
+          PASS_COUNT=$(awk '/^ *[0-9]+ pass$/{print $1}' /tmp/test-output.txt || true)
+          FAIL_COUNT=$(awk '/^ *[0-9]+ fail$/{print $1}' /tmp/test-output.txt || true)
+
+          echo ""
+          echo "--- Test Summary ---"
+          echo "pass=${PASS_COUNT:-none} fail=${FAIL_COUNT:-none}"
+
+          # Real test failures — always fail CI
+          if [ -n "$FAIL_COUNT" ] && [ "$FAIL_COUNT" != "0" ]; then
+            echo "::error::$FAIL_COUNT test(s) failed"
+            exit 1
+          fi
+
+          # Tests passed (we have a pass count and zero/no failures)
+          if [ -n "$PASS_COUNT" ] && [ "$PASS_COUNT" -gt 0 ] 2>/dev/null; then
+            exit 0
+          fi
+
+          # No test summary at all — Bun crashed before running tests
+          echo "::error::No test results found in output — Bun may have crashed before running tests"
+          exit 1
 
   # ---------------------------------------------------------------------------
   # Driver E2E tests — only when driver code changes.
@@ -166,6 +199,19 @@ jobs:
           --health-timeout 5s
           --health-retries 10
 
+      clickhouse:
+        image: clickhouse/clickhouse-server:latest
+        env:
+          CLICKHOUSE_DB: testdb
+          CLICKHOUSE_DEFAULT_ACCESS_MANAGEMENT: 1
+        ports:
+          - 18123:8123
+        options: >-
+          --health-cmd "wget --no-verbose --tries=1 --spider http://localhost:8123/ping || exit 1"
+          --health-interval 5s
+          --health-timeout 5s
+          --health-retries 15
+
     steps:
       - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
 
@@ -213,6 +259,13 @@ jobs:
           TEST_MONGODB_HOST: 127.0.0.1
           TEST_MONGODB_PORT: "27017"
 
+      - name: Run ClickHouse driver E2E
+        run: bun test test/altimate/drivers-clickhouse-e2e.test.ts
+        working-directory: packages/opencode
+        env:
+          TEST_CLICKHOUSE_HOST: 127.0.0.1
+          TEST_CLICKHOUSE_PORT: "18123"
+
       # Cloud tests NOT included — they require real credentials
       # Run locally with:
       #   ALTIMATE_CODE_CONN_SNOWFLAKE_TEST='...' bun test test/altimate/drivers-snowflake-e2e.test.ts
@@ -297,6 +350,55 @@ jobs:
           DBT_E2E_VERSIONS: "1.8,1.10,1.11"
           DBT_RESOLVE_SCENARIOS: "venv,uv,system"
 
+  # ---------------------------------------------------------------------------
+  # Verdaccio sanity suite — tests the real `npm install -g` flow.
+  # Only on push to main (too slow for PRs, needs Docker Compose).
+  # Catches publish-pipeline bugs: missing files, broken symlinks, wrong bin
+  # field, dependency resolution failures, postinstall script issues.
+  # ---------------------------------------------------------------------------
+  sanity-verdaccio:
+    name: Sanity (Verdaccio)
+    needs: changes
+    if: github.event_name == 'push'
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
+
+      - uses: oven-sh/setup-bun@ecf28ddc73e819eb6fa29df6b34ef8921c743461 # v2
+        with:
+          bun-version: "1.3.10"
+
+      - name: Cache Bun dependencies
+        uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
+        with:
+          path: ~/.bun/install/cache
+          key: bun-${{ runner.os }}-${{ hashFiles('bun.lock') }}
+          restore-keys: |
+            bun-${{ runner.os }}-
+
+      - name: Install dependencies
+        run: bun install
+
+      - name: Build CLI binary
+        # target-index=1 = linux-x64 (see release.yml matrix)
+        run: bun run packages/opencode/script/build.ts --target-index=1
+        env:
+          OPENCODE_VERSION: 0.0.0-sanity-${{ github.sha }}
+          OPENCODE_RELEASE: "1"
+          MODELS_DEV_API_JSON: test/tool/fixtures/models-api.json
+
+      - name: Build dbt-tools
+        run: bun run build
+        working-directory: packages/dbt-tools
+
+      - name: Run Verdaccio sanity suite
+        run: |
+          docker compose -f test/sanity/docker-compose.verdaccio.yml up \
+            --build --abort-on-container-exit --exit-code-from sanity
+        env:
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+
   marker-guard:
     name: Marker Guard
     runs-on: ubuntu-latest

.github/workflows/release.yml

@@ -125,9 +125,54 @@ jobs:
           path: packages/opencode/dist/
           compression-level: 1
 
+  # ---------------------------------------------------------------------------
+  # Verdaccio sanity suite — tests the real `npm install -g` flow BEFORE
+  # publishing to npm. Catches broken symlinks, missing files, postinstall
+  # failures, and dependency resolution issues that smoke tests miss.
+  # ---------------------------------------------------------------------------
+  sanity-verdaccio:
+    name: Sanity (Verdaccio)
+    needs: build
+    runs-on: ubuntu-latest
+    timeout-minutes: 30
+    steps:
+      - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4
+
+      - uses: oven-sh/setup-bun@ecf28ddc73e819eb6fa29df6b34ef8921c743461 # v2
+        with:
+          bun-version: "1.3.10"
+
+      - name: Cache Bun dependencies
+        uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
+        with:
+          path: ~/.bun/install/cache
+          key: bun-${{ runner.os }}-${{ hashFiles('bun.lock') }}
+          restore-keys: |
+            bun-${{ runner.os }}-
+
+      - name: Install dependencies
+        run: bun install
+
+      - name: Download linux-x64 build artifact
+        uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
+        with:
+          name: dist-linux-x64
+          path: packages/opencode/dist/
+
+      - name: Build dbt-tools
+        run: bun run build
+        working-directory: packages/dbt-tools
+
+      - name: Run Verdaccio sanity suite
+        run: |
+          docker compose -f test/sanity/docker-compose.verdaccio.yml up \
+            --build --abort-on-container-exit --exit-code-from sanity
+        env:
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+
   publish-npm:
     name: Publish to npm
-    needs: build
+    needs: [build, sanity-verdaccio]
     runs-on: ubuntu-latest
     timeout-minutes: 60
     permissions:

.opencode/skills/dbt-analyze/SKILL.md

@@ -111,7 +111,7 @@ If no manifest is available:
 1. Run `lineage_check` on the changed SQL
 2. Show column-level data flow
 3. Note: downstream impact requires a manifest
-4. Suggest: `altimate-dbt build-project` to generate one
+4. Suggest: `altimate-dbt build` to generate one
 
 ## Common Mistakes