docs: update setup docs for MalloyConfig discovery and overlay-based config (#304)

Reflects the new config loading modes (--config, --project-dir, default),
includeDefaultConnections, config file discovery with local overrides,
DuckDB working directory resolution via overlays, and updated VS Code
extension connection sidebar behavior.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Author: mtoy-googly-moogly

src/documentation/setup/cli.malloynb

@@ -26,7 +26,15 @@ malloy-cli --help
 
 ## Configure Connections
 
-The CLI stores its own configuration separately from VS Code in `~/.config/malloy/malloy-config.json`. See **[Configuration](config.malloynb)** for the full config file format, connection type properties, environment variables, and default connections.
+The CLI looks for configuration in one of three ways, depending on how you invoke it:
+
+| Flag | Behavior |
+|------|----------|
+| _(neither)_ | Uses the global config at `~/.config/malloy/malloy-config.json` |
+| `--config <path>` | Uses that specific config file (or directory containing `malloy-config.json`). No walk-up discovery. |
+| `--project-dir <dir>` | Walks up from your current directory to `<dir>` looking for `malloy-config.json`, the same way the VS Code extension discovers config. |
+
+See **[Configuration](config.malloynb)** for the full config file format, connection type properties, environment variables, and default connections.
 
 **Migrating from an older version:** If you have an existing `~/.config/malloy/config.json`, the CLI will automatically migrate it to the new `malloy-config.json` format on first run.
 
@@ -83,9 +91,18 @@ malloy-cli connections create postgres pg host=localhost port=5432 databaseName=
 
 If a connection name in your model matches a registered database type (DuckDB, BigQuery, Postgres, Snowflake, Trino, or Presto), the CLI creates one with default settings automatically. For example, `duckdb.table('data.parquet')` works out of the box. Some database types rely on environment variables for their defaults (e.g. `SNOWFLAKE_ACCOUNT`, `TRINO_SERVER`) — see the [Configuration](config.malloynb) reference for the full list of defaults per connection type.
 
+**Note:** When using `--project-dir`, default connections are _not_ automatically included. Only connections explicitly listed in the config file are available — if your model references a connection that isn't configured, it will fail. This matches the VS Code extension behavior where the project config is authoritative.
+
 ### DuckDB Working Directory
 
-When a DuckDB connection does not have `workingDirectory` set in the config, the CLI automatically resolves relative table paths (like `duckdb.table('data.csv')`) relative to the `.malloy` or `.malloysql` file being run. If you set `workingDirectory` explicitly, that value is used instead.
+When a DuckDB connection does not have `workingDirectory` set in the config, relative table paths (like `duckdb.table('data.csv')`) are resolved based on the config mode:
+
+| Mode | Relative paths resolve from |
+|------|-----------------------------|
+| `--project-dir` | The project root directory |
+| `--config` or default | The current working directory |
+
+If you set `workingDirectory` explicitly in the config, that value is always used instead.
 
 ---
 
@@ -142,10 +159,14 @@ See **[Persistence](../experiments/persistence.malloynb)** for full details on s
 ### Global Options
 
 ```bash
-# Use a project-level config instead of the global default
+# Use a specific config file or directory
 malloy-cli --config . run model.malloy
 malloy-cli --config /path/to/malloy-config.json build
 
+# Use project-dir for VS Code-compatible config discovery
+malloy-cli --project-dir . run model.malloy
+malloy-cli --project-dir /path/to/project build
+
 # Get help
 malloy-cli --help
 malloy-cli run --help

src/documentation/setup/config.malloynb

@@ -26,10 +26,34 @@ The file contains a `connections` object where each key is a connection name and
 
 The `is` field identifies the connection type. All other fields are type-specific parameters documented below.
 
-### Where the config file lives
+### Default connections
 
-- **VS Code Extension** — place `malloy-config.json` in your workspace root. The extension detects it automatically and picks up changes on save. In multi-root workspaces, each root can have its own file. You can also set a global fallback directory via the `malloy.globalConfigDirectory` setting (e.g. `~/.config/malloy`) — this is used when no workspace config file exists.
-- **Malloy CLI** — `~/.config/malloy/malloy-config.json`. See the [CLI setup](cli.malloynb) for details.
+Setting `includeDefaultConnections` to `true` automatically creates a connection for every registered database type that you haven't already listed. Each gets the type's name as its connection name and picks up any registered property defaults:
+
+```json
+{
+  "includeDefaultConnections": true,
+  "connections": {
+    "warehouse": {
+      "is": "duckdb",
+      "databasePath": "./data.db"
+    }
+  }
+}
+```
+
+In this example, you get your explicit `warehouse` connection plus auto-generated connections for `bigquery`, `postgres`, `snowflake`, etc. — each with default settings. A generated connection is skipped if its type or name collides with one you already listed.
+
+### Config file discovery
+
+Both VS Code and the CLI discover config files automatically. Starting from the Malloy file you're working with, Malloy walks up through parent directories until it reaches the project root, looking for `malloy-config.json` at each level. The first one found is used.
+
+You can also place a `malloy-config-local.json` alongside `malloy-config.json` for developer-specific overrides (credentials, local paths) that shouldn't be checked into git. When both files exist at the same directory level, they are merged: `connections` entries are deep-merged by name (local wins on conflicts), while all other sections (`manifestPath`, `includeDefaultConnections`, etc.) are replaced wholesale by the local file's value.
+
+Add `malloy-config-local.json` to your `.gitignore`:
+```
+malloy-config-local.json
+```
 
 ---
 
@@ -40,7 +64,7 @@ The `is` field identifies the connection type. All other fields are type-specifi
 | Parameter | Type | Description |
 |---|---|---|
 | `databasePath` | file | Path to .db file (default: `:memory:`) |
-| `workingDirectory` | string | Working directory for relative paths |
+| `workingDirectory` | string | Working directory for relative paths. Default: the project root (the directory discovery walked up to) |
 | `motherDuckToken` | secret | MotherDuck auth token. Default: `{env: "MOTHERDUCK_TOKEN"}` |
 | `additionalExtensions` | string | Comma-separated DuckDB extensions to load (e.g. `"spatial,fts"`). Built-in: json, httpfs, icu |
 | `readOnly` | boolean | Open database read-only |
@@ -110,6 +134,7 @@ Authentication: provide either `token` or the `oauthClientId` + `oauthClientSecr
 | `privateKeyPath` | file | Path to private key (.pem/.key) |
 | `privateKeyPass` | password | Private key passphrase |
 | `timeoutMs` | number | Query timeout in ms |
+| `schemaSampleTimeoutMs` | number | Timeout for the query that samples variant columns to detect their schema (default 15000) |
 | `setupSQL` | text | Connection setup SQL ([see below](#setup-sql)) |
 
 Snowflake also supports TOML configuration at `~/.snowflake/connections.toml`. See [Snowflake connection configuration](https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-connect#connecting-using-the-connections-toml-file) for details.

src/documentation/setup/extension.malloynb

@@ -45,27 +45,37 @@ The Malloy Extension also works in VS Code Web environments like github.dev. Pre
 
 ## Configuring Connections
 
-There are two ways to configure database connections:
+There are three sources of connection configuration, resolved in priority order:
 
-1. **`malloy-config.json`** (recommended) — a project-level config file checked into source control
-2. **VS Code Settings** — user-level configuration via the Connections sidebar
+1. **Project config** (`malloy-config.json`) — a config file discovered by walking up from the active file to the workspace root
+2. **Global config** — a `malloy-config.json` in the directory specified by the `malloy.globalConfigDirectory` setting
+3. **Settings + Defaults** — user-level connections from VS Code settings, plus built-in defaults for each supported database type
 
-When both exist, config file connections take priority. Settings connections are used as a fallback for any connection names not defined in the config file. For example, if `malloy-config.json` defines a `duckdb` connection but not `bigquery`, the config file's `duckdb` is used while `bigquery` falls through to VS Code settings.
+The first source that exists wins — **sources do not merge across levels**. If a project `malloy-config.json` exists, it is fully authoritative: settings connections and defaults are not used at all. To include defaults alongside your config file connections, add `"includeDefaultConnections": true` to your config file.
+
+Within a project config, you can also create a `malloy-config-local.json` in the same directory. Local config connections are deep-merged into the project config by connection name, allowing you to keep credentials or machine-specific overrides out of source control.
 
 ### Connections Sidebar
 
-The **Malloy Connections** sidebar shows the effective connections for the file you are currently editing, grouped by source:
+The **Malloy Connections** sidebar shows the effective connections for the file you are currently editing. It works with both `.malloy` files and `.malloynb` notebook files.
+
+When a config file applies (project or global), the sidebar shows a single **Config** group with the config file path in the header. Click the file icon on the group header to open the JSON file for editing. Config connections open in a read-only view.
+
+When no config file applies, the sidebar shows:
 
-- **Config** — Connections from the `malloy-config.json` that applies to the active file. These are read-only in the editor; click the file icon on the group header to open the JSON file for editing.
-- **Settings** — User-defined connections from VS Code settings. Editable through the connection editor. If a settings connection has the same name as a config connection, it appears as "(shadowed)" since the config version takes priority.
+- **Settings** — User-defined connections from VS Code settings. Editable through the connection editor.
 - **Defaults** — One entry per supported database type, using built-in defaults. Click to create a new settings connection from the default.
 
-The sidebar updates automatically as you switch between files, so it always reflects the connections that would be used if you ran a query from the current file. Click any connection to open the connection editor, or use the **+** button to create a new connection.
+The sidebar updates automatically as you switch between files, so it always reflects the connections that would be used if you ran a query from the current file.
 
 ### Project Configuration: `malloy-config.json`
 
 Place a `malloy-config.json` file in your project directory. The extension finds the config that applies to the current file by walking up from the file's directory toward the workspace root. It uses the first `malloy-config.json` it encounters. This means you can place a config file at the workspace root for project-wide connections, or in a subdirectory to override connections for files in that subtree. In multi-root workspaces, each workspace root has its own independent config search.
 
+**Local overrides:** Place a `malloy-config-local.json` alongside a project `malloy-config.json` to override specific connections without modifying the shared config. Connections in the local file are merged by name into the project config. Add `malloy-config-local.json` to `.gitignore` to keep credentials out of source control. Local configs are only merged for discovered project configs, not global configs.
+
+The extension watches for changes to `malloy-config.json` and `malloy-config-local.json` — edits take effect on the next query without reloading VS Code.
+
 The extension also reads [persistence manifests](../experiments/persistence.malloynb) from the config's `manifestPath` directory. When the builder (`malloy-cli build`) writes a manifest, VS Code picks it up on the next compile — no restart needed.
 
 **Using with `malloy-cli build`:** Both tools need to share the same config file so they read and write the same manifest. There are two ways to set this up:
@@ -100,8 +110,7 @@ Config file connections open in a read-only view. You can still test them, but e
 
 | Setting | Description | Default |
 |---------|-------------|---------|
-| `malloy.globalConfigDirectory` | Directory containing a global `malloy-config.json`, used as a fallback when no workspace config file exists (e.g. `~/.config/malloy`) | `""` |
-| `malloy.projectConnectionsOnly` | Only use workspace config file connections — hides global config, settings, and defaults | `false` |
+| `malloy.globalConfigDirectory` | Directory containing a global `malloy-config.json`, used when no project config file is found (e.g. `~/.config/malloy`) | `""` |
 
 ---
 
@@ -112,14 +121,15 @@ Config file connections open in a read-only view. You can still test them, but e
 DuckDB runs locally with zero configuration. It reads Parquet, CSV, and JSON files.
 
 ```malloy
-// Local files (relative to .malloy file)
+// Local files (relative to workspace root)
 source: flights is duckdb.table('data/flights.parquet')
-source: users is duckdb.table('../users.csv')
 
 // URLs
 source: remote_data is duckdb.table('https://example.com/data.parquet')
 ```
 
+Relative file paths in DuckDB are resolved from the **workspace root**, not from the directory of the `.malloy` file. This means `duckdb.table('data/flights.parquet')` reads `<workspace>/data/flights.parquet` regardless of where the Malloy file is located in the project.
+
 ### MotherDuck
 
 Set your token as an environment variable before launching VS Code: