Skip to content

Commit 99c7898

Browse files
Extend connection docs for secrets (#281)
1 parent 1294dbb commit 99c7898

3 files changed

Lines changed: 56 additions & 95 deletions

File tree

src/documentation/setup/cli.malloynb

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,10 @@ malloy-cli connections create snowflake sf account=myorg warehouse=compute_wh
7979
malloy-cli connections create postgres pg host=localhost port=5432 databaseName=mydb
8080
```
8181

82+
### Default Connections
83+
84+
Two connections are created automatically if you don't already have a connection that overrides them — `bigquery` and `duckdb`. If your Malloy files reference these connection names, they work without explicit setup. DuckDB uses a built-in in-memory instance, and BigQuery attempts to connect using any existing gcloud authentication on your computer.
85+
8286
### DuckDB Working Directory
8387

8488
When a DuckDB connection does not have `workingDirectory` set in the config, the CLI automatically resolves relative table paths (like `duckdb.table('data.csv')`) relative to the `.malloy` or `.malloysql` file being run. If you set `workingDirectory` explicitly, that value is used instead.

src/documentation/setup/config.malloynb

Lines changed: 51 additions & 94 deletions
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,7 @@ The `is` field identifies the connection type. All other fields are type-specifi
4141
|---|---|---|
4242
| `databasePath` | file | Path to .db file (default: `:memory:`) |
4343
| `workingDirectory` | string | Working directory for relative paths |
44-
| `motherDuckToken` | password | MotherDuck auth token |
44+
| `motherDuckToken` | secret | MotherDuck auth token. Default: `{env: "MOTHERDUCK_TOKEN"}` |
4545
| `additionalExtensions` | string | Comma-separated DuckDB extensions to load (e.g. `"spatial,fts"`). Built-in: json, httpfs, icu |
4646
| `readOnly` | boolean | Open database read-only |
4747
| `setupSQL` | text | Connection setup SQL ([see below](#setup-sql)) |
@@ -74,39 +74,53 @@ The `is` field identifies the connection type. All other fields are type-specifi
7474

7575
| Parameter | Type | Description |
7676
|---|---|---|
77-
| `host` | string | Server host |
78-
| `port` | number | Server port (default: 3306) |
79-
| `database` | string | Database name |
80-
| `user` | string | Username |
81-
| `password` | password | Password |
77+
| `host` | string | Server host. Default: `{env: "MYSQL_HOST"}` |
78+
| `port` | number | Server port. Default: `{env: "MYSQL_PORT"}` or 3306 |
79+
| `database` | string | Database name. Default: `{env: "MYSQL_DATABASE"}` |
80+
| `user` | string | Username. Default: `{env: "MYSQL_USER"}` |
81+
| `password` | password | Password. Default: `{env: "MYSQL_PASSWORD"}` |
8282
| `setupSQL` | text | Connection setup SQL ([see below](#setup-sql)) |
8383

8484
### `snowflake` — Snowflake
8585

8686
| Parameter | Type | Description |
8787
|---|---|---|
88-
| `account` | string | Snowflake account identifier (required) |
89-
| `username` | string | Username |
90-
| `password` | password | Password |
88+
| `account` | string | Snowflake account identifier (required). Default: `{env: "SNOWFLAKE_ACCOUNT"}` |
89+
| `username` | string | Username. Default: `{env: "SNOWFLAKE_USER"}` |
90+
| `password` | password | Password. Default: `{env: "SNOWFLAKE_PASSWORD"}` |
9191
| `role` | string | Role |
92-
| `warehouse` | string | Warehouse |
93-
| `database` | string | Database |
94-
| `schema` | string | Schema |
92+
| `warehouse` | string | Warehouse. Default: `{env: "SNOWFLAKE_WAREHOUSE"}` |
93+
| `database` | string | Database. Default: `{env: "SNOWFLAKE_DATABASE"}` |
94+
| `schema` | string | Schema. Default: `{env: "SNOWFLAKE_SCHEMA"}` |
9595
| `privateKeyPath` | file | Path to private key (.pem/.key) |
9696
| `privateKeyPass` | password | Private key passphrase |
9797
| `timeoutMs` | number | Query timeout in ms |
9898
| `setupSQL` | text | Connection setup SQL ([see below](#setup-sql)) |
9999

100-
### `trino` / `presto` — Trino or Presto
100+
Snowflake also supports TOML configuration at `~/.snowflake/connections.toml`. See [Snowflake connection configuration](https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-connect#connecting-using-the-connections-toml-file) for details.
101+
102+
### `trino` — Trino
101103

102104
| Parameter | Type | Description |
103105
|---|---|---|
104-
| `server` | string | Server hostname |
106+
| `server` | string | Server hostname. Default: `{env: "TRINO_SERVER"}` |
105107
| `port` | number | Server port |
106-
| `catalog` | string | Catalog name |
107-
| `schema` | string | Schema name |
108-
| `user` | string | Username |
109-
| `password` | password | Password |
108+
| `catalog` | string | Catalog name. Default: `{env: "TRINO_CATALOG"}` |
109+
| `schema` | string | Schema name. Default: `{env: "TRINO_SCHEMA"}` |
110+
| `user` | string | Username. Default: `{env: "TRINO_USER"}` |
111+
| `password` | password | Password. Default: `{env: "TRINO_PASSWORD"}` |
112+
| `setupSQL` | text | Connection setup SQL ([see below](#setup-sql)) |
113+
114+
### `presto` — Presto
115+
116+
| Parameter | Type | Description |
117+
|---|---|---|
118+
| `server` | string | Server hostname. Default: `{env: "PRESTO_HOST"}` |
119+
| `port` | number | Server port (default: 8080). Default: `{env: "PRESTO_PORT"}` |
120+
| `catalog` | string | Catalog name. Default: `{env: "PRESTO_CATALOG"}` |
121+
| `schema` | string | Schema name. Default: `{env: "PRESTO_SCHEMA"}` |
122+
| `user` | string | Username. Default: `{env: "PRESTO_USER"}` |
123+
| `password` | password | Password. Default: `{env: "PRESTO_PASSWORD"}` |
110124
| `setupSQL` | text | Connection setup SQL ([see below](#setup-sql)) |
111125

112126
---
@@ -131,85 +145,28 @@ SET search_path TO analytics; CREATE TEMP TABLE foo AS SELECT 1;
131145

132146
---
133147

134-
## Environment Variables
135-
136-
Some databases support configuration via environment variables. When environment variables are set, a connection is created automatically using those values. Values set in `malloy-config.json` override environment variables.
137-
138-
### MotherDuck
139-
140-
MotherDuck is configured through a token. In MotherDuck, click **Settings** then copy your token.
141-
142-
```bash
143-
export MOTHERDUCK_TOKEN=your_token_here
144-
```
145-
146-
The default connection name for MotherDuck is `md`.
147-
148-
**Example usage:**
149-
```malloy
150-
source: hacker_news is md.table('sample_data.hn.hacker_news')
151-
```
152-
153-
### MySQL
154-
155-
MySQL connections can be configured entirely through environment variables.
156-
157-
```bash
158-
export MYSQL_USER=readonly
159-
export MYSQL_HOST=db.example.com
160-
export MYSQL_PORT=3306
161-
export MYSQL_PASSWORD=your_password
162-
export MYSQL_DATABASE=analytics
163-
```
164-
165-
The default connection name is `mysql`. `MYSQL_USER` is required; other variables are optional but typically needed.
166-
167-
### Snowflake
168-
169-
```bash
170-
export SNOWFLAKE_ACCOUNT=myorg-myaccount # Required
171-
export SNOWFLAKE_USER=analyst
172-
export SNOWFLAKE_PASSWORD=your_password
173-
export SNOWFLAKE_WAREHOUSE=compute_wh
174-
export SNOWFLAKE_DATABASE=analytics
175-
export SNOWFLAKE_SCHEMA=public
176-
```
177-
178-
Snowflake also supports TOML configuration at `~/.snowflake/connections.toml`. See [Snowflake connection configuration](https://docs.snowflake.com/en/developer-guide/python-connector/python-connector-connect#connecting-using-the-connections-toml-file) for details.
179-
180-
### Trino
181-
182-
```bash
183-
export TRINO_SERVER=https://trino.example.com # Required
184-
export TRINO_USER=analyst # Required
185-
export TRINO_PASSWORD=your_password
186-
export TRINO_CATALOG=hive
187-
export TRINO_SCHEMA=default
188-
```
189-
190-
The default connection name is `trino`.
148+
## Sensitive Values
191149

192-
### Presto
150+
Fields with type `password` or `secret` contain sensitive values like credentials and API tokens. Instead of putting these directly in your config file, you can use an environment variable reference:
193151

194-
```bash
195-
export PRESTO_HOST=presto.example.com # Required
196-
export PRESTO_PORT=8080 # Defaults to 8080
197-
export PRESTO_USER=analyst
198-
export PRESTO_PASSWORD=your_password
199-
export PRESTO_CATALOG=hive
200-
export PRESTO_SCHEMA=default
152+
```json
153+
{
154+
"connections": {
155+
"my_motherduck": {
156+
"is": "duckdb",
157+
"databasePath": "md:my_database",
158+
"motherDuckToken": {"env": "MOTHERDUCK_TOKEN"}
159+
},
160+
"my_postgres": {
161+
"is": "postgres",
162+
"host": "db.example.com",
163+
"password": {"env": "PG_PASSWORD"}
164+
}
165+
}
166+
}
201167
```
202168

203-
The default connection name is `presto`.
204-
205-
### Databases Without Environment Variable Support
206-
207-
- **BigQuery**: Uses `gcloud auth login --update-adc` (OAuth) or a service account key file configured in `malloy-config.json`
208-
- **PostgreSQL**: Uses connection parameters in `malloy-config.json` (credentials stored in system keychain for VS Code)
209-
- **DuckDB**: Uses file paths directly, no authentication needed
210-
211-
---
169+
The `{"env": "VAR_NAME"}` syntax looks up the value from the named environment variable at connection time. If the variable is not set, the field is omitted and the connection proceeds without it.
212170

213-
## Default Connections
171+
You can also provide a plain string value directly — this is useful for testing but not recommended for shared or committed config files.
214172

215-
Two connections are created automatically if you don't already have a connection that overrides them — `bigquery` and `duckdb`. If your Malloy files reference these connection names, they work without explicit setup. DuckDB uses a built-in in-memory instance, and BigQuery attempts to connect using any existing gcloud authentication on your computer.

src/documentation/setup/extension.malloynb

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,7 @@ Add a PostgreSQL connection via **Malloy: Edit Connections**. Enter host, port,
139139

140140
### MySQL
141141

142-
MySQL connections require environment variables — see [Configuration](config.malloynb#mysql). The extension will detect the connection automatically when the variables are set.
142+
MySQL connections can be configured in `malloy-config.json` or via environment variables — see [Configuration](config.malloynb).
143143

144144
### Trino and Presto
145145

0 commit comments

Comments
 (0)