feat: support custom dbt profiles.yml path via DBT_PROFILES_DIR and project-local discovery (#605)

anandgupta42 · claude · web-flow · commit 43e5f9a4faa9 · 2026-03-31T23:01:30.000-07:00
Closes #604 - Add `resolveProfilesPath()` with dbt-standard priority: explicit path > `DBT_PROFILES_DIR` env var > project-local `profiles.yml` > `<home>/.dbt/profiles.yml` - Warn when `DBT_PROFILES_DIR` is set but `profiles.yml` not found (graceful fallthrough) - Add `projectDir` param to `DbtProfilesParams` and expose in tool schema - Pass detected dbt project path from `project-scan` to `dbt.profiles` - Use `path.join(homedir(), ".dbt", "profiles.yml")` for OS-independent paths - Add 4 unit tests + 35 E2E tests covering priority chain, edge cases, adapter mapping - Update 6 doc files with platform-neutral path notation Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
diff --git a/README.md b/README.md
@@ -47,7 +47,7 @@ export OPENAI_API_KEY=your_key      # OpenAI
 altimate /discover
 ```
 
-`/discover` auto-detects dbt projects, warehouse connections (from `~/.dbt/profiles.yml`, Docker, environment variables), and installed tools (dbt, sqlfluff, airflow, dagster, and more). Skip this and start building — you can always run it later.
+`/discover` auto-detects dbt projects, warehouse connections (from `profiles.yml` — checks `DBT_PROFILES_DIR`, project directory, then `<home>/.dbt/`; plus Docker and environment variables), and installed tools (dbt, sqlfluff, airflow, dagster, and more). Skip this and start building — you can always run it later.
 
 > **Headless / scripted usage:** `altimate --yolo` auto-approves all permission prompts. Not recommended with live warehouse connections.
 
diff --git a/docs/docs/configure/warehouses.md b/docs/docs/configure/warehouses.md
@@ -467,10 +467,33 @@ The `/discover` command can automatically detect warehouse connections from:
 
 | Source | Detection |
 |--------|-----------|
-| dbt profiles | Parses `~/.dbt/profiles.yml` |
+| dbt profiles | Searches for `profiles.yml` (see resolution order below) |
 | Docker containers | Finds running PostgreSQL, MySQL, SQL Server, and ClickHouse containers |
 | Environment variables | Scans for `SNOWFLAKE_ACCOUNT`, `PGHOST`, `DATABRICKS_HOST`, etc. |
 
+### dbt profiles.yml resolution order
+
+When discovering dbt profiles, altimate checks the following locations **in priority order** and uses the first one found:
+
+| Priority | Location | Description |
+|----------|----------|-------------|
+| 1 | Explicit path | If you pass a `path` parameter to the `dbt_profiles` tool |
+| 2 | `DBT_PROFILES_DIR` env var | Standard dbt environment variable — set it to the directory containing your `profiles.yml` |
+| 3 | Project-local `profiles.yml` | A `profiles.yml` in your dbt project root (next to `dbt_project.yml`) |
+| 4 | `<home>/.dbt/profiles.yml` | The global default location (e.g., `~/.dbt/` on macOS/Linux, `%USERPROFILE%\.dbt\` on Windows) |
+
+This means teams that keep `profiles.yml` in their project repo (a common pattern for CI/CD) will have it detected automatically — no extra configuration needed.
+
+```bash
+# Option 1: Set the environment variable
+export DBT_PROFILES_DIR=/path/to/your/project
+
+# Option 2: Just put profiles.yml next to dbt_project.yml
+# Copy from default location (macOS/Linux)
+cp ~/.dbt/profiles.yml ./profiles.yml
+altimate /discover
+```
+
 See [Warehouse Tools](../data-engineering/tools/warehouse-tools.md) for the full list of environment variable signals.
 
 ## Testing Connections
diff --git a/docs/docs/data-engineering/tools/warehouse-tools.md b/docs/docs/data-engineering/tools/warehouse-tools.md
@@ -53,7 +53,7 @@ env_bigquery  | bigquery | GOOGLE_APPLICATION_CREDENTIALS
 | **Git** | `git` commands (branch, remote) |
 | **dbt project** | Walks up directories for `dbt_project.yml`, reads name/profile |
 | **dbt manifest** | Parses `target/manifest.json` for model/source/test counts |
-| **dbt profiles** | Bridge call to parse `~/.dbt/profiles.yml` |
+| **dbt profiles** | Searches for `profiles.yml`: `DBT_PROFILES_DIR` env var → project root → `<home>/.dbt/profiles.yml` |
 | **Docker DBs** | Bridge call to discover running PostgreSQL/MySQL/MSSQL containers |
 | **Existing connections** | Bridge call to list already-configured warehouses |
 | **Environment variables** | Scans `process.env` for warehouse signals (see table below) |
diff --git a/docs/docs/getting-started/quickstart-new.md b/docs/docs/getting-started/quickstart-new.md
@@ -34,13 +34,13 @@ altimate
 
 ### Option A: Auto-detect from dbt profiles
 
-If you have `~/.dbt/profiles.yml` configured:
+If you have a `profiles.yml` — either in your home directory's `.dbt/` folder, in your project repo, or pointed to by `DBT_PROFILES_DIR`:
 
 ```bash
 /discover
 ```
 
-Altimate reads your dbt profiles and creates warehouse connections automatically. You'll see output like:
+Altimate searches for `profiles.yml` in this order: `DBT_PROFILES_DIR` env var → project root (next to `dbt_project.yml`) → `<home>/.dbt/profiles.yml`. It reads your dbt profiles and creates warehouse connections automatically. You'll see output like:
 
 ```
 Found dbt project: jaffle_shop (dbt-snowflake)
diff --git a/docs/docs/getting-started/quickstart.md b/docs/docs/getting-started/quickstart.md
@@ -189,7 +189,7 @@ You can also set a smaller model for lightweight tasks like summarization:
 altimate /discover
 ```
 
-Auto-detects your dbt projects, warehouse credentials from `~/.dbt/profiles.yml`, running Docker containers, and environment variables (`SNOWFLAKE_ACCOUNT`, `PGHOST`, `DATABASE_URL`, etc.).
+Auto-detects your dbt projects, warehouse credentials from `profiles.yml` (checks `DBT_PROFILES_DIR`, then your project directory, then the default `<home>/.dbt/profiles.yml`), running Docker containers, and environment variables (`SNOWFLAKE_ACCOUNT`, `PGHOST`, `DATABASE_URL`, etc.).
 
 ### Manual configuration
 
diff --git a/docs/docs/reference/troubleshooting.md b/docs/docs/reference/troubleshooting.md
@@ -55,13 +55,13 @@ altimate --print-logs --log-level DEBUG
 
 **Solutions:**
 
-1. **If using dbt:** Run `altimate-dbt init` to set up the dbt integration. The CLI will use your `profiles.yml` automatically, so no separate connection config is needed.
+1. **If using dbt:** Run `/discover` — it automatically finds your `profiles.yml` from `DBT_PROFILES_DIR`, your project directory, or `<home>/.dbt/profiles.yml`. If your `profiles.yml` is in a custom location, set `DBT_PROFILES_DIR` to the directory containing it.
 2. **If not using dbt:** Add a connection via the `warehouse_add` tool, `~/.altimate-code/connections.json`, or `ALTIMATE_CODE_CONN_*` env vars.
 3. Test connectivity: use the `warehouse_test` tool with your connection name.
 4. Check that the warehouse hostname and port are reachable
-3. Verify the role/user has the required permissions
-4. For Snowflake: ensure the warehouse is not suspended
-5. For BigQuery: check that the service account has the required IAM roles
+5. Verify the role/user has the required permissions
+6. For Snowflake: ensure the warehouse is not suspended
+7. For BigQuery: check that the service account has the required IAM roles
 
 ### MCP Server Initialization Failures
 
diff --git a/packages/opencode/src/altimate/native/connections/dbt-profiles.ts b/packages/opencode/src/altimate/native/connections/dbt-profiles.ts
@@ -95,13 +95,41 @@ function mapConfig(dbtType: string, dbtConfig: Record<string, unknown>): Connect
   return config
 }
 
+/**
+ * Resolve the profiles.yml path using dbt's standard priority order:
+ * 1. Explicit path (if provided)
+ * 2. DBT_PROFILES_DIR environment variable
+ * 3. Project-local profiles.yml (in dbt project root)
+ * 4. ~/.dbt/profiles.yml (default)
+ */
+function resolveProfilesPath(explicitPath?: string, projectDir?: string): string {
+  if (explicitPath) return explicitPath
+
+  const envDir = process.env.DBT_PROFILES_DIR
+  if (envDir) {
+    const envPath = path.join(envDir, "profiles.yml")
+    if (fs.existsSync(envPath)) return envPath
+    // Warn when DBT_PROFILES_DIR is set but profiles.yml not found there —
+    // dbt CLI would error here, we fall through for graceful discovery
+    console.warn(`[dbt-profiles] DBT_PROFILES_DIR is set to "${envDir}" but no profiles.yml found there, falling through`)
+  }
+
+  if (projectDir) {
+    const projectPath = path.join(projectDir, "profiles.yml")
+    if (fs.existsSync(projectPath)) return projectPath
+  }
+
+  return path.join(os.homedir(), ".dbt", "profiles.yml")
+}
+
 /**
  * Parse dbt profiles.yml and return discovered connections.
  *
- * @param profilesPath - Path to profiles.yml. Defaults to ~/.dbt/profiles.yml
+ * @param profilesPath - Explicit path to profiles.yml
+ * @param projectDir - dbt project root directory (for project-local profiles.yml)
  */
-export async function parseDbtProfiles(profilesPath?: string): Promise<DbtProfileConnection[]> {
-  const resolvedPath = profilesPath ?? path.join(os.homedir(), ".dbt", "profiles.yml")
+export async function parseDbtProfiles(profilesPath?: string, projectDir?: string): Promise<DbtProfileConnection[]> {
+  const resolvedPath = resolveProfilesPath(profilesPath, projectDir)
 
   if (!fs.existsSync(resolvedPath)) {
     return []
diff --git a/packages/opencode/src/altimate/native/connections/register.ts b/packages/opencode/src/altimate/native/connections/register.ts
@@ -409,7 +409,7 @@ register("schema.inspect", async (params: SchemaInspectParams): Promise<SchemaIn
 // --- dbt.profiles ---
 register("dbt.profiles", async (params: DbtProfilesParams): Promise<DbtProfilesResult> => {
   try {
-    const connections = await parseDbtProfiles(params.path)
+    const connections = await parseDbtProfiles(params.path, params.projectDir)
     return {
       success: true,
       connections,
diff --git a/packages/opencode/src/altimate/native/types.ts b/packages/opencode/src/altimate/native/types.ts
@@ -908,6 +908,8 @@ export interface DbtLineageResult {
 
 export interface DbtProfilesParams {
   path?: string
+  /** dbt project root directory — used to find project-local profiles.yml */
+  projectDir?: string
 }
 
 export interface DbtProfileConnection {
diff --git a/packages/opencode/src/altimate/tools/dbt-profiles.ts b/packages/opencode/src/altimate/tools/dbt-profiles.ts
@@ -1,18 +1,24 @@
 import z from "zod"
+import { homedir } from "os"
+import { join } from "path"
 import { Tool } from "../../tool/tool"
 import { Dispatcher } from "../native"
 import { isSensitiveField } from "../native/connections/credential-store"
 
+const DEFAULT_DBT_PROFILES = join(homedir(), ".dbt", "profiles.yml")
+
 export const DbtProfilesTool = Tool.define("dbt_profiles", {
   description:
-    "Discover dbt profiles from profiles.yml and map them to warehouse connections. Auto-detects Snowflake, BigQuery, Databricks, Postgres, Redshift, MySQL, DuckDB configurations.",
+    `Discover dbt profiles from profiles.yml and map them to warehouse connections. Auto-detects Snowflake, BigQuery, Databricks, Postgres, Redshift, MySQL, DuckDB configurations. Searches: explicit path > DBT_PROFILES_DIR env var > project-local profiles.yml > ${DEFAULT_DBT_PROFILES}.`,
   parameters: z.object({
-    path: z.string().optional().describe("Path to profiles.yml (defaults to ~/.dbt/profiles.yml)"),
+    path: z.string().optional().describe(`Explicit path to profiles.yml. If omitted, checks DBT_PROFILES_DIR, then project directory, then ${DEFAULT_DBT_PROFILES}`),
+    projectDir: z.string().optional().describe("dbt project root directory. Used to find project-local profiles.yml next to dbt_project.yml"),
   }),
   async execute(args, ctx) {
     try {
       const result = await Dispatcher.call("dbt.profiles", {
         path: args.path,
+        projectDir: args.projectDir,
       })
 
       if (!result.success) {
@@ -28,7 +34,7 @@ export const DbtProfilesTool = Tool.define("dbt_profiles", {
         return {
           title: "dbt Profiles: No connections found",
           metadata: { success: true, connection_count: 0 },
-          output: "No dbt profiles found. Ensure ~/.dbt/profiles.yml exists with valid configurations.",
+          output: `No dbt profiles found. Ensure profiles.yml exists in your project directory, DBT_PROFILES_DIR, or ${DEFAULT_DBT_PROFILES}.`,
         }
       }
 
diff --git a/packages/opencode/src/altimate/tools/project-scan.ts b/packages/opencode/src/altimate/tools/project-scan.ts
@@ -472,7 +472,9 @@ export const ProjectScanTool = Tool.define("project_scan", {
       .then((r) => r.warehouses)
       .catch(() => [] as Array<{ name: string; type: string; database?: string }>)
 
-    const dbtProfiles = await Dispatcher.call("dbt.profiles", {})
+    const dbtProfiles = await Dispatcher.call("dbt.profiles", {
+      projectDir: dbtProject.found ? dbtProject.path : undefined,
+    })
       .then((r) => r.connections ?? [])
       .catch(() => [] as Array<{ name: string; type: string; config: Record<string, unknown> }>)
 
diff --git a/packages/opencode/test/altimate/connections.test.ts b/packages/opencode/test/altimate/connections.test.ts
@@ -610,6 +610,151 @@ trino_project:
   })
 })
 
+// ---------------------------------------------------------------------------
+// dbt profiles path resolution (DBT_PROFILES_DIR + project-local)
+// ---------------------------------------------------------------------------
+
+describe("dbt profiles path resolution", () => {
+  const PROFILE_CONTENT = `
+myproject:
+  target: dev
+  outputs:
+    dev:
+      type: postgres
+      host: localhost
+      port: 5432
+      user: test
+      pass: secret
+      dbname: testdb
+      schema: public
+`
+
+  test("finds profiles.yml via DBT_PROFILES_DIR env var", async () => {
+    const fs = await import("fs")
+    const os = await import("os")
+    const path = await import("path")
+
+    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "dbt-envdir-"))
+    fs.writeFileSync(path.join(tmpDir, "profiles.yml"), PROFILE_CONTENT)
+
+    const origEnv = process.env.DBT_PROFILES_DIR
+    process.env.DBT_PROFILES_DIR = tmpDir
+
+    try {
+      const connections = await parseDbtProfiles()
+      expect(connections).toHaveLength(1)
+      expect(connections[0].name).toBe("myproject_dev")
+    } finally {
+      if (origEnv === undefined) delete process.env.DBT_PROFILES_DIR
+      else process.env.DBT_PROFILES_DIR = origEnv
+      fs.rmSync(tmpDir, { recursive: true })
+    }
+  })
+
+  test("finds project-local profiles.yml via projectDir", async () => {
+    const fs = await import("fs")
+    const os = await import("os")
+    const path = await import("path")
+
+    const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "dbt-projdir-"))
+    fs.writeFileSync(path.join(tmpDir, "profiles.yml"), PROFILE_CONTENT)
+
+    try {
+      const connections = await parseDbtProfiles(undefined, tmpDir)
+      expect(connections).toHaveLength(1)
+      expect(connections[0].name).toBe("myproject_dev")
+    } finally {
+      fs.rmSync(tmpDir, { recursive: true })
+    }
+  })
+
+  test("DBT_PROFILES_DIR takes priority over projectDir", async () => {
+    const fs = await import("fs")
+    const os = await import("os")
+    const path = await import("path")
+
+    const envDir = fs.mkdtempSync(path.join(os.tmpdir(), "dbt-env-pri-"))
+    fs.writeFileSync(
+      path.join(envDir, "profiles.yml"),
+      `
+env_profile:
+  outputs:
+    dev:
+      type: postgres
+      host: env-host
+      dbname: envdb
+      schema: public
+`,
+    )
+
+    const projDir = fs.mkdtempSync(path.join(os.tmpdir(), "dbt-proj-pri-"))
+    fs.writeFileSync(
+      path.join(projDir, "profiles.yml"),
+      `
+proj_profile:
+  outputs:
+    dev:
+      type: postgres
+      host: proj-host
+      dbname: projdb
+      schema: public
+`,
+    )
+
+    const origEnv = process.env.DBT_PROFILES_DIR
+    process.env.DBT_PROFILES_DIR = envDir
+
+    try {
+      const connections = await parseDbtProfiles(undefined, projDir)
+      expect(connections).toHaveLength(1)
+      expect(connections[0].name).toBe("env_profile_dev")
+      expect(connections[0].config.host).toBe("env-host")
+    } finally {
+      if (origEnv === undefined) delete process.env.DBT_PROFILES_DIR
+      else process.env.DBT_PROFILES_DIR = origEnv
+      fs.rmSync(envDir, { recursive: true })
+      fs.rmSync(projDir, { recursive: true })
+    }
+  })
+
+  test("explicit path takes priority over DBT_PROFILES_DIR", async () => {
+    const fs = await import("fs")
+    const os = await import("os")
+    const path = await import("path")
+
+    const explicitDir = fs.mkdtempSync(path.join(os.tmpdir(), "dbt-explicit-"))
+    fs.writeFileSync(
+      path.join(explicitDir, "profiles.yml"),
+      `
+explicit_profile:
+  outputs:
+    dev:
+      type: postgres
+      host: explicit-host
+      dbname: explicitdb
+      schema: public
+`,
+    )
+
+    const envDir = fs.mkdtempSync(path.join(os.tmpdir(), "dbt-env-nouse-"))
+    fs.writeFileSync(path.join(envDir, "profiles.yml"), PROFILE_CONTENT)
+
+    const origEnv = process.env.DBT_PROFILES_DIR
+    process.env.DBT_PROFILES_DIR = envDir
+
+    try {
+      const connections = await parseDbtProfiles(path.join(explicitDir, "profiles.yml"), undefined)
+      expect(connections).toHaveLength(1)
+      expect(connections[0].name).toBe("explicit_profile_dev")
+    } finally {
+      if (origEnv === undefined) delete process.env.DBT_PROFILES_DIR
+      else process.env.DBT_PROFILES_DIR = origEnv
+      fs.rmSync(explicitDir, { recursive: true })
+      fs.rmSync(envDir, { recursive: true })
+    }
+  })
+})
+
 // ---------------------------------------------------------------------------
 // dbtConnectionsToConfigs
 // ---------------------------------------------------------------------------
diff --git a/packages/opencode/test/altimate/dbt-profiles-resolution-e2e.test.ts b/packages/opencode/test/altimate/dbt-profiles-resolution-e2e.test.ts

Original file line number	Diff line number	Diff line change
`@@ -908,6 +908,8 @@ export interface DbtLineageResult {`
`908`	`908`
`909`	`909`	`export interface DbtProfilesParams {`
`910`	`910`	`path?: string`
	`911`	`+ /** dbt project root directory — used to find project-local profiles.yml */`
	`912`	`+ projectDir?: string`
`911`	`913`	`}`
`912`	`914`
`913`	`915`	`export interface DbtProfileConnection {`