Skip to content

Commit 8e999b8

Browse files
feat: add reconcile mode for declarative database seeding
Add reconcile mode (`mode: reconcile`) for seed sets, making seeding declarative: the rendered spec becomes the source of truth, and initium reconciles the database to match it whenever the spec changes. New rows are inserted, changed rows are updated, and removed rows are deleted. Key changes: - Per-row tracking table (`initium_seed_rows`) for change detection - Content hash on seed tracking table for fast skip optimization - `--reconcile-all` CLI flag to override all seed sets to reconcile mode - `--dry-run` CLI flag to preview changes without modifying the database - Schema validation: reconcile requires unique_key on every table, validates rows contain all key columns, rejects reserved keys - Runtime guard: --reconcile-all rejects tables missing unique_key - Hash-skip disabled for seed sets with @ref: expressions to prevent stale foreign keys when upstream auto-generated IDs shift - Dry-run treats @ref: as literals to avoid resolution failures - MySQL row tracking uses SHA-256 generated column for PK (no collisions) - CI summary job (`ci`) for branch ruleset status check - Backward compatible: existing seed sets default to mode: once Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent c9ba912 commit 8e999b8

9 files changed

Lines changed: 2496 additions & 10 deletions

File tree

.github/workflows/ci.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,3 +45,13 @@ jobs:
4545
- run: helm lint charts/initium
4646
- run: helm template test-release charts/initium --set sampleDeployment.enabled=true --set 'initContainers[0].name=wait' --set 'initContainers[0].command[0]=wait-for' --set 'initContainers[0].args[0]=--target' --set 'initContainers[0].args[1]=tcp://localhost:5432'
4747
- run: helm unittest charts/initium
48+
ci:
49+
if: always()
50+
needs: [lint, test, build, helm-lint]
51+
runs-on: ubuntu-latest
52+
steps:
53+
- run: |
54+
if [[ "${{ contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') }}" == "true" ]]; then
55+
echo "One or more jobs failed or were cancelled"
56+
exit 1
57+
fi

CHANGELOG.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Added
11+
- Reconcile mode for seed sets (`mode: reconcile`): declarative seeding where the spec is the source of truth. Changed rows are updated, new rows are inserted, and removed rows are deleted automatically.
12+
- `--reconcile-all` CLI flag to override all seed sets to reconcile mode for a single run.
13+
- `--dry-run` CLI flag to preview what changes reconciliation would make without modifying the database.
14+
- Per-row tracking table (`initium_seed_rows`) for change detection and orphan deletion in reconcile mode.
15+
- Content hash (`content_hash` column) on the seed tracking table for fast "anything changed?" checks before row-by-row comparison.
16+
- Automatic migration of existing tracking tables: the `content_hash` column is added transparently on first run. Existing seed sets remain in `once` mode with no behavior change.
17+
18+
### Changed
19+
- Reconcile hash-skip now only applies to seed sets without `@ref:` expressions. Seed sets containing `@ref:` references always run row-level reconciliation to prevent stale foreign keys when upstream auto-generated IDs shift.
20+
- Hash computation sorts tables by `(order, table_name)` instead of just `order` for deterministic hashing when multiple tables share the same order value.
21+
- Dry-run mode treats `@ref:` expressions as literals to avoid failures when references haven't been populated yet (e.g., auto_id + refs within the same seed set).
22+
23+
### Fixed
24+
- `--reconcile-all` now rejects seed sets where any table is missing `unique_key`, preventing reconciliation from generating identical row keys and updating/deleting wrong rows.
25+
- Reconcile mode validation now rejects empty/whitespace-only `unique_key` entries and reserved column names like `_ref`.
26+
- Reconcile mode validation now checks that every row contains all `unique_key` columns, preventing incomplete row keys during reconciliation.
27+
- MySQL row tracking table now uses SHA-256 generated column (`row_key_hash`) for the primary key instead of `row_key(255)` prefix, preventing key collisions for JSON keys exceeding 255 bytes.
28+
1029
## [1.1.0] - 2026-02-26
1130

1231
### Added

docs/seeding.md

Lines changed: 58 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ phases:
5050
seed_sets: # Optional. Seed sets to apply in this phase.
5151
- name: initial_data
5252
order: 1 # Optional. Controls execution order across seed sets.
53+
mode: once # Optional. "once" (default) or "reconcile".
5354
tables:
5455
- table: config
5556
order: 1 # Optional. Controls execution order within a seed set.
@@ -82,6 +83,7 @@ phases:
8283
| `phases[].wait_for[].timeout` | string | No | Per-object timeout override (e.g. `60s`, `2m`, `1m30s`) |
8384
| `phases[].seed_sets[].name` | string | Yes | Unique name for the seed set (used in tracking) |
8485
| `phases[].seed_sets[].order` | integer | No | Execution order (lower values first, default: 0) |
86+
| `phases[].seed_sets[].mode` | string | No | Seed mode: `once` (default) or `reconcile` |
8587
| `phases[].seed_sets[].tables[].table` | string | Yes | Target database table name |
8688
| `phases[].seed_sets[].tables[].order` | integer | No | Execution order within the seed set (default: 0) |
8789
| `phases[].seed_sets[].tables[].unique_key` | string[] | No | Columns for duplicate detection |
@@ -213,6 +215,55 @@ rows:
213215
password_hash: "{{ env.ADMIN_PASSWORD_HASH }}"
214216
```
215217

218+
### Reconcile Mode
219+
220+
By default, seed sets are applied once and never modified (`mode: once`). Reconcile mode makes seeding declarative: the rendered spec becomes the source of truth, and initium reconciles the database to match it whenever the rendered spec changes.
221+
222+
If the rendered spec has not changed since the last run (content hash match), initium treats the seed set as already reconciled and skips it. Out-of-band database changes are not corrected until a spec change triggers reconciliation again.
223+
224+
Enable reconcile mode per seed set:
225+
226+
```yaml
227+
seed_sets:
228+
- name: departments
229+
mode: reconcile # "once" (default) or "reconcile"
230+
tables:
231+
- table: departments
232+
unique_key: [name] # Required for reconcile mode
233+
rows:
234+
- name: Engineering
235+
- name: Sales
236+
```
237+
238+
Or override all seed sets for a single run:
239+
240+
```bash
241+
initium seed --spec /seeds/seed.yaml --reconcile-all
242+
```
243+
244+
**How it works:**
245+
246+
1. On each run, initium computes a content hash of the rendered seed set (after template/env expansion).
247+
2. If the hash matches the stored hash, the seed set is skipped (no-op).
248+
3. If the hash differs, initium reconciles row by row:
249+
- **New rows** (in spec but not in DB) are inserted.
250+
- **Changed rows** (different values for same unique key) are updated.
251+
- **Removed rows** (in DB but not in spec) are deleted.
252+
253+
**Requirements:**
254+
- Every table in a reconciled seed set must have a `unique_key`. Without it, there is no way to identify which rows correspond to which spec entries.
255+
- Environment variable changes trigger reconciliation (resolved values are compared, not raw templates).
256+
257+
**Row tracking:** Initium creates a companion table (`{tracking_table}_rows`, e.g., `initium_seed_rows`) that stores the resolved values of each seeded row. This enables change detection and orphan deletion.
258+
259+
**Dry-run mode:** Preview what reconciliation would do without modifying the database:
260+
261+
```bash
262+
initium seed --spec /seeds/seed.yaml --dry-run
263+
```
264+
265+
This logs insert/update/delete counts per table without executing any changes.
266+
216267
### Reset Mode
217268

218269
Use `--reset` to delete all data from seeded tables and remove tracking entries before re-applying. Tables are deleted in reverse order to respect foreign key constraints:
@@ -276,11 +327,13 @@ spec:
276327

277328
## CLI Reference
278329

279-
| Flag | Default | Description |
280-
| --------- | ---------- | --------------------------------------- |
281-
| `--spec` | (required) | Path to seed spec file (YAML or JSON) |
282-
| `--reset` | `false` | Delete existing data and re-apply seeds |
283-
| `--json` | `false` | Enable JSON log output |
330+
| Flag | Default | Description |
331+
| ------------------ | ---------- | --------------------------------------------------------- |
332+
| `--spec` | (required) | Path to seed spec file (YAML or JSON) |
333+
| `--reset` | `false` | Delete existing data and re-apply seeds |
334+
| `--dry-run` | `false` | Preview changes without modifying the database |
335+
| `--reconcile-all` | `false` | Override all seed sets to reconcile mode for this run |
336+
| `--json` | `false` | Enable JSON log output |
284337

285338
## Failure Modes
286339

src/main.rs

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,18 @@ enum Commands {
145145
help = "Reset mode: delete existing data before re-seeding"
146146
)]
147147
reset: bool,
148+
#[arg(
149+
long,
150+
env = "INITIUM_DRY_RUN",
151+
help = "Dry-run: show what would change without modifying the database"
152+
)]
153+
dry_run: bool,
154+
#[arg(
155+
long,
156+
env = "INITIUM_RECONCILE_ALL",
157+
help = "Override all seed sets to reconcile mode for this run"
158+
)]
159+
reconcile_all: bool,
148160
},
149161

150162
/// Render templates into config files
@@ -313,7 +325,12 @@ fn main() {
313325
lock_file,
314326
args,
315327
} => cmd::migrate::run(&log, &args, &workdir, &lock_file),
316-
Commands::Seed { spec, reset } => seed::run(&log, &spec, reset),
328+
Commands::Seed {
329+
spec,
330+
reset,
331+
dry_run,
332+
reconcile_all,
333+
} => seed::run(&log, &spec, reset, dry_run, reconcile_all),
317334
Commands::Render {
318335
template,
319336
output,

0 commit comments

Comments
 (0)