Skip to content

feat: structured database connection config (#39) #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mikkeldamsgaard merged 2 commits into from
Mar 14, 2026
Merged

feat: structured database connection config (#39) #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e4862c2
feat: support structured database connection config as alternative to…
mikkeldamsgaard Mar 14, 2026
0edbdcd
fix: reject unsupported options in MySQL structured config
mikkeldamsgaard Mar 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- `urlencode` template filter for percent-encoding strings in URLs. Useful for embedding passwords or other values containing URL-reserved characters (`@`, `%`, `:`, `/`, etc.) in connection strings.
- `dprint` formatter for Markdown, JSON, TOML, YAML, and Dockerfile with CI check (`dprint/check@v2.2`) and definition-of-done gate.
- Structured database connection config as an alternative to URL (`host`, `port`, `user`, `password`, `name`, `options` fields). Passwords with URL-reserved characters (`@`, `%`, `:`, etc.) work without encoding. Connections are built using driver-native APIs (PostgreSQL key-value DSN, MySQL `OptsBuilder`), bypassing URL parsing entirely. The `url`/`url_env` fields remain supported for backward compatibility. See [#39](https://github.com/KitStream/initium/issues/39).

### Changed

Expand Down
106 changes: 80 additions & 26 deletions docs/seeding.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,54 @@ initium seed --spec /seeds/seed.yaml --json

The seed spec file defines the complete seeding plan. Both YAML and JSON formats are supported (file extension determines parser). The spec file is a MiniJinja template rendered with environment variables before parsing.

### Database connection

Two connection styles are supported. Choose **one** — they cannot be combined.

**URL-based** (existing behavior):

```yaml
database:
driver: postgres
url: "postgres://user:pass@host:5432/dbname"
# or: url_env: DATABASE_URL
```

**Structured fields** (no URL encoding needed):

```yaml
database:
driver: postgres
host: pg.example.com
port: 5432 # Optional. Default: 5432 (postgres), 3306 (mysql)
user: netbird
password: "{{ env.DB_PASSWORD }}" # Special chars just work — no URL encoding
name: mydb
options: # Optional. Driver-specific connection parameters
sslmode: disable
```

Structured config builds the connection using driver-native APIs, bypassing URL parsing entirely. Passwords with `@`, `%`, `:`, or other URL-reserved characters work without encoding.

> **Note:** Structured config is not supported for SQLite — use `url` instead.

### Full schema

```yaml
database:
driver: postgres # Required. One of: postgres, mysql, sqlite
# --- URL-based connection (pick one style) ---
url: "postgres://..." # Direct database URL
url_env: DATABASE_URL # Or: name of env var containing the URL
# --- Structured connection (alternative to url/url_env) ---
host: pg.example.com # Database host
port: 5432 # Optional. Default per driver
user: myuser # Database user
password: "secret" # Database password (special chars OK)
name: mydb # Database name
options: # Optional. Driver-specific parameters
sslmode: disable
# --- Common ---
tracking_table: initium_seed # Default: "initium_seed"

phases:
Expand Down Expand Up @@ -68,30 +111,36 @@ phases:

### Field reference

| Field | Type | Required | Description |
| ----------------------------------------------- | -------- | -------- | ---------------------------------------------------------------- |
| `database.driver` | string | Yes | Database driver: `postgres`, `mysql`, or `sqlite` |
| `database.url` | string | No | Direct database connection URL |
| `database.url_env` | string | No | Environment variable containing the database URL |
| `database.tracking_table` | string | No | Name of the seed tracking table (default: `initium_seed`) |
| `phases[].name` | string | Yes | Unique phase name |
| `phases[].order` | integer | No | Execution order (lower first, default: 0) |
| `phases[].database` | string | No | Target database name (for create/switch) |
| `phases[].schema` | string | No | Target schema name (for create/switch) |
| `phases[].create_if_missing` | boolean | No | Create the database/schema if it does not exist (default: false) |
| `phases[].timeout` | string | No | Default wait timeout (e.g. `30s`, `1m`, `1m30s`; default: `30s`) |
| `phases[].wait_for[].type` | string | Yes | Object type: `table`, `view`, `schema`, or `database` |
| `phases[].wait_for[].name` | string | Yes | Object name to wait for |
| `phases[].wait_for[].timeout` | string | No | Per-object timeout override (e.g. `60s`, `2m`, `1m30s`) |
| `phases[].seed_sets[].name` | string | Yes | Unique name for the seed set (used in tracking) |
| `phases[].seed_sets[].order` | integer | No | Execution order (lower values first, default: 0) |
| `phases[].seed_sets[].mode` | string | No | Seed mode: `once` (default) or `reconcile` |
| `phases[].seed_sets[].tables[].table` | string | Yes | Target database table name |
| `phases[].seed_sets[].tables[].order` | integer | No | Execution order within the seed set (default: 0) |
| `phases[].seed_sets[].tables[].unique_key` | string[] | No | Columns for duplicate detection |
| `phases[].seed_sets[].tables[].auto_id.column` | string | No | Auto-generated ID column name |
| `phases[].seed_sets[].tables[].auto_id.id_type` | string | No | ID type (default: `integer`) |
| `phases[].seed_sets[].tables[].rows[]._ref` | string | No | Internal reference name for cross-table references |
| Field | Type | Required | Description |
| ----------------------------------------------- | ----------------- | -------- | ---------------------------------------------------------------------- |
| `database.driver` | string | Yes | Database driver: `postgres`, `mysql`, or `sqlite` |
| `database.url` | string | No | Direct database connection URL (cannot combine with structured fields) |
| `database.url_env` | string | No | Environment variable containing the database URL |
| `database.host` | string | No | Database host (structured config; cannot combine with url/url_env) |
| `database.port` | integer | No | Database port (default: 5432 for postgres, 3306 for mysql) |
| `database.user` | string | No | Database user (structured config) |
| `database.password` | string | No | Database password — special characters work without encoding |
| `database.name` | string | No | Database name (structured config) |
| `database.options` | map[string]string | No | Driver-specific connection parameters (e.g. `sslmode: disable`) |
| `database.tracking_table` | string | No | Name of the seed tracking table (default: `initium_seed`) |
| `phases[].name` | string | Yes | Unique phase name |
| `phases[].order` | integer | No | Execution order (lower first, default: 0) |
| `phases[].database` | string | No | Target database name (for create/switch) |
| `phases[].schema` | string | No | Target schema name (for create/switch) |
| `phases[].create_if_missing` | boolean | No | Create the database/schema if it does not exist (default: false) |
| `phases[].timeout` | string | No | Default wait timeout (e.g. `30s`, `1m`, `1m30s`; default: `30s`) |
| `phases[].wait_for[].type` | string | Yes | Object type: `table`, `view`, `schema`, or `database` |
| `phases[].wait_for[].name` | string | Yes | Object name to wait for |
| `phases[].wait_for[].timeout` | string | No | Per-object timeout override (e.g. `60s`, `2m`, `1m30s`) |
| `phases[].seed_sets[].name` | string | Yes | Unique name for the seed set (used in tracking) |
| `phases[].seed_sets[].order` | integer | No | Execution order (lower values first, default: 0) |
| `phases[].seed_sets[].mode` | string | No | Seed mode: `once` (default) or `reconcile` |
| `phases[].seed_sets[].tables[].table` | string | Yes | Target database table name |
| `phases[].seed_sets[].tables[].order` | integer | No | Execution order within the seed set (default: 0) |
| `phases[].seed_sets[].tables[].unique_key` | string[] | No | Columns for duplicate detection |
| `phases[].seed_sets[].tables[].auto_id.column` | string | No | Auto-generated ID column name |
| `phases[].seed_sets[].tables[].auto_id.id_type` | string | No | ID type (default: `integer`) |
| `phases[].seed_sets[].tables[].rows[]._ref` | string | No | Internal reference name for cross-table references |

### Wait-for object support by driver

Expand All @@ -115,14 +164,18 @@ phases:

SQLite does not support separate databases or schemas — each file is a database.

### Database URL resolution
### Database connection resolution

The database URL is resolved in this order:
If structured fields (`host`, `port`, `user`, `password`, `name`) are provided, the connection is built using driver-native APIs — no URL is needed.

Otherwise, the database URL is resolved in this order:

1. `database.url_env` — environment variable name containing the URL
2. `database.url` — direct URL in the spec file
3. `DATABASE_URL` — fallback environment variable

Structured fields and URL-based fields (`url`/`url_env`) are mutually exclusive — specifying both is a validation error.

## Features

### MiniJinja Templating
Expand Down Expand Up @@ -385,6 +438,7 @@ spec:
See the [`examples/seed/`](../examples/seed/) directory:

- [`basic-seed.yaml`](../examples/seed/basic-seed.yaml) — PostgreSQL with departments and employees, cross-table references
- [`structured-config-seed.yaml`](../examples/seed/structured-config-seed.yaml) — PostgreSQL with structured connection config (no URL encoding)
- [`sqlite-seed.yaml`](../examples/seed/sqlite-seed.yaml) — SQLite configuration table seeding
- [`env-credentials-seed.yaml`](../examples/seed/env-credentials-seed.yaml) — MySQL with credentials from Kubernetes secrets
- [`phased-seed.yaml`](../examples/seed/phased-seed.yaml) — Multi-phase PostgreSQL seeding with wait-for, create-if-missing, and MiniJinja templating
31 changes: 31 additions & 0 deletions examples/seed/structured-config-seed.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Structured database connection config — no URL encoding needed.
#
# Passwords with special characters (@, %, :, etc.) just work.
# Initium builds the connection using driver-native APIs.
#
# Usage:
# export DB_PASSWORD='p@ss:word/with%special&chars'
# initium seed --spec examples/seed/structured-config-seed.yaml

database:
driver: postgres
host: pg.example.com
port: 5432
user: app_user
password: "{{ env.DB_PASSWORD }}"
name: myapp
options:
sslmode: disable

phases:
- name: setup
seed_sets:
- name: config
tables:
- table: config
unique_key: [key]
rows:
- key: app_name
value: MyApp
- key: version
value: "1.0.0"
Loading
Loading