Skip to content

Commit ed6ff81

Browse files
feat: extend seed with phases, object waiting, schema creation, and MiniJinja templating (#15) (#16)
* feat: extend seed with phases, object waiting, schema creation, and MiniJinja templating. Add seed schema version 2 with phase-based execution supporting ordered phases with create/wait/seed lifecycle, MiniJinja template rendering, embedded database/schema creation via create_if_missing, wait-for logic polling for tables/views/schemas/databases with timeouts, cross-phase @ref: references, and driver-aware error reporting. New Database trait methods: create_database, create_schema, object_exists, driver_name for SQLite, PostgreSQL, MySQL. 32 new tests, all 98 pass. Closes #15
1 parent ac6e592 commit ed6ff81

File tree

14 files changed

+1691
-531
lines changed

14 files changed

+1691
-531
lines changed

CHANGELOG.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### Removed
11+
- Seed schema version 1 (flat `seed_sets` without phases): all seed specs now use phase-based structure
12+
- `version` field from seed spec schema: no longer required or accepted
13+
14+
### Added
15+
- Seed schema with phase-based execution: ordered phases with create → wait → seed lifecycle
16+
- MiniJinja template rendering for seed spec files: dynamic values, conditionals, loops via `{{ env.VAR }}`
17+
- Embedded database/schema creation via `create_if_missing` in seed phases (PostgreSQL, MySQL)
18+
- Embedded wait-for logic: poll for tables, views, schemas, or databases with configurable per-phase and per-object timeouts
19+
- Database trait methods: `create_database`, `create_schema`, `object_exists`, `driver_name` for SQLite, PostgreSQL, MySQL
20+
- Cross-phase `@ref:` references: references defined in earlier phases resolve in later phases
21+
- Documentation: schema reference, driver support tables, MiniJinja templating guide, failure modes in `docs/seeding.md`
22+
- Example: `examples/seed/phased-seed.yaml` — multi-phase PostgreSQL seeding with wait-for, create-if-missing, and MiniJinja
23+
1024
### Changed
1125
- Seed executor tests now verify data actually arrived in the database after execution, using file-based SQLite and post-execution queries to assert row counts, column values, cross-table references, env substitution, ordering, and edge cases
1226

1327
### Fixed
28+
- Aligned all markdown table columns across documentation files (`FAQ.md`, `README.md`, `docs/security.md`, `docs/seeding.md`, `docs/usage.md`)
1429
- Fixed clippy `collapsible_if` lint in seed executor's unique key check
1530
- Removed dead code: unused `src/cmd/seed.rs` module (replaced by `src/seed/`)
1631
- Suppressed unused field warning on `AutoIdConfig.id_type` (reserved for future use)

FAQ.md

Lines changed: 13 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -147,14 +147,14 @@ This is useful when you're shipping logs to a centralized system like Loki, Data
147147

148148
All retry parameters are flags on the `wait-for` subcommand:
149149

150-
| Flag | Default | What it does |
151-
|------|---------|-------------|
152-
| `--max-attempts` | `60` | Total number of attempts before giving up |
153-
| `--initial-delay` | `1s` | Delay after the first failure |
154-
| `--max-delay` | `30s` | Upper bound on delay between retries |
155-
| `--backoff-factor` | `2.0` | Multiplier applied to the delay after each attempt |
156-
| `--jitter` | `0.1` | Random fraction (0.0–1.0) added to each delay to prevent thundering herd |
157-
| `--timeout` | `5m` | Hard deadline across all targets |
150+
| Flag | Default | What it does |
151+
| ------------------ | --------- | ------------------------------------------------------------------------ |
152+
| `--max-attempts` | `60` | Total number of attempts before giving up |
153+
| `--initial-delay` | `1s` | Delay after the first failure |
154+
| `--max-delay` | `30s` | Upper bound on delay between retries |
155+
| `--backoff-factor` | `2.0` | Multiplier applied to the delay after each attempt |
156+
| `--jitter` | `0.1` | Random fraction (0.0–1.0) added to each delay to prevent thundering herd |
157+
| `--timeout` | `5m` | Hard deadline across all targets |
158158

159159
Example — fast retries with low jitter:
160160

@@ -449,12 +449,12 @@ kubectl logs <pod-name> -c <initcontainer-name>
449449

450450
Common causes:
451451

452-
| Symptom | Likely cause | Fix |
453-
|---------|-------------|-----|
452+
| Symptom | Likely cause | Fix |
453+
| ----------------------------------------- | -------------------------------------------- | ---------------------------------------------------------------------------- |
454454
| `target not reachable` after all attempts | Target service isn't running or DNS is wrong | Check the service/endpoint exists and is in the same namespace (or use FQDN) |
455-
| `unsupported target scheme` | Missing `tcp://` or `http://` prefix | Add the scheme: `tcp://postgres:5432` not just `postgres:5432` |
456-
| `path traversal detected` | Output path tries to escape workdir | Use a relative path for `--output` |
457-
| `context cancelled` | Overall `--timeout` was too short | Increase `--timeout` or check why the target takes so long |
455+
| `unsupported target scheme` | Missing `tcp://` or `http://` prefix | Add the scheme: `tcp://postgres:5432` not just `postgres:5432` |
456+
| `path traversal detected` | Output path tries to escape workdir | Use a relative path for `--output` |
457+
| `context cancelled` | Overall `--timeout` was too short | Increase `--timeout` or check why the target takes so long |
458458

459459
### Does Initium support ARM-based nodes (e.g., AWS Graviton)?
460460

README.md

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -38,26 +38,26 @@ kubectl apply -f https://raw.githubusercontent.com/kitstream/initium/main/exampl
3838

3939
## Why Initium?
4040

41-
| | Bash scripts | Initium |
42-
|---|---|---|
43-
| **Retries with backoff** | DIY, error-prone | Built-in, configurable |
44-
| **Structured logging** | `echo` statements | JSON or text with timestamps |
45-
| **Security** | Runs as root, full shell | Non-root, no shell, read-only FS |
46-
| **Secret handling** | Easily leaked in logs | Automatic redaction |
47-
| **Multiple tools** | Install curl, netcat, psql… | Single 2MB image |
48-
| **Reproducibility** | Shell differences across distros | Single Rust binary, `FROM scratch` |
49-
| **Vulnerability surface** | Full OS + shell utils | Zero OS packages |
41+
| | Bash scripts | Initium |
42+
| ------------------------- | -------------------------------- | ---------------------------------- |
43+
| **Retries with backoff** | DIY, error-prone | Built-in, configurable |
44+
| **Structured logging** | `echo` statements | JSON or text with timestamps |
45+
| **Security** | Runs as root, full shell | Non-root, no shell, read-only FS |
46+
| **Secret handling** | Easily leaked in logs | Automatic redaction |
47+
| **Multiple tools** | Install curl, netcat, psql… | Single 2MB image |
48+
| **Reproducibility** | Shell differences across distros | Single Rust binary, `FROM scratch` |
49+
| **Vulnerability surface** | Full OS + shell utils | Zero OS packages |
5050

5151
## Subcommands
5252

53-
| Command | Description | Status |
54-
|---------|-------------|--------|
55-
| `wait-for` | Wait for TCP/HTTP/HTTPS endpoints | ✅ Available |
56-
| `migrate` | Run database migrations | ✅ Available |
57-
| `seed` | Structured database seeding from YAML/JSON | ✅ Available |
58-
| `render` | Render config templates | ✅ Available |
59-
| `fetch` | Fetch secrets/config from HTTP | ✅ Available |
60-
| `exec` | Run commands with structured logging | ✅ Available |
53+
| Command | Description | Status |
54+
| ---------- | -------------------------------------------------------------------- | ----------- |
55+
| `wait-for` | Wait for TCP/HTTP/HTTPS endpoints | ✅ Available |
56+
| `migrate` | Run database migrations | ✅ Available |
57+
| `seed` | Structured database seeding from YAML/JSON with MiniJinja templating | ✅ Available |
58+
| `render` | Render config templates | ✅ Available |
59+
| `fetch` | Fetch secrets/config from HTTP | ✅ Available |
60+
| `exec` | Run commands with structured logging | ✅ Available |
6161

6262
### wait-for
6363

docs/security.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -15,16 +15,16 @@ Initium runs as an initContainer in Kubernetes pods. Its threat model considers:
1515

1616
### Attack Vectors Addressed
1717

18-
| Vector | Mitigation |
19-
|--------|-----------|
20-
| **Path traversal** | All file writes constrained to `--workdir`; absolute paths rejected; `..` sequences resolved and validated |
21-
| **Secret leakage via logs** | Automatic redaction of keys matching `token`, `password`, `secret`, `auth`, `api_key`, `authorization` |
22-
| **Privilege escalation** | Container runs as UID 65534 (nobody); `allowPrivilegeEscalation: false`; all capabilities dropped |
23-
| **Filesystem tampering** | `readOnlyRootFilesystem: true`; writes only to mounted emptyDir volumes |
24-
| **Unintended network access** | All target URLs must be explicitly provided via flags; no default outbound connections |
25-
| **TLS downgrade** | TLS verification enabled by default; `--insecure-tls` requires explicit opt-in |
26-
| **Shell injection** | Commands executed via `execve` (no shell); `--` separator for command arguments |
27-
| **Supply chain** | Minimal `scratch` base image; SBOM and provenance attestation in CI; pinned dependencies |
18+
| Vector | Mitigation |
19+
| ----------------------------- | ---------------------------------------------------------------------------------------------------------- |
20+
| **Path traversal** | All file writes constrained to `--workdir`; absolute paths rejected; `..` sequences resolved and validated |
21+
| **Secret leakage via logs** | Automatic redaction of keys matching `token`, `password`, `secret`, `auth`, `api_key`, `authorization` |
22+
| **Privilege escalation** | Container runs as UID 65534 (nobody); `allowPrivilegeEscalation: false`; all capabilities dropped |
23+
| **Filesystem tampering** | `readOnlyRootFilesystem: true`; writes only to mounted emptyDir volumes |
24+
| **Unintended network access** | All target URLs must be explicitly provided via flags; no default outbound connections |
25+
| **TLS downgrade** | TLS verification enabled by default; `--insecure-tls` requires explicit opt-in |
26+
| **Shell injection** | Commands executed via `execve` (no shell); `--` separator for command arguments |
27+
| **Supply chain** | Minimal `scratch` base image; SBOM and provenance attestation in CI; pinned dependencies |
2828

2929
## Safe Defaults
3030

0 commit comments

Comments
 (0)