Skip to content

Commit f782b40

Browse files
josecelanojosecelano
committed
docs(issues): add finalized PostgreSQL epic and child specs
1 parent c536c53 commit f782b40

7 files changed

Lines changed: 1056 additions & 0 deletions

‎docs/issues/469-01-config-model-and-schema-postgresql-driver.md‎

Lines changed: 177 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,177 @@
1+
# Vertical Slice A: BYO PostgreSQL (External DB) End-To-End
2+
3+
**Issue**: TBD (`469-01` planned child subissue)
4+
**Parent Epic**: #469 - Add PostgreSQL Support To Tracker Deployer
5+
**Related**:
6+
7+
- `schemas/environment-config.json`
8+
- `src/application/command_handlers/create/config/tracker/tracker_core_section.rs`
9+
- `src/domain/tracker/config/core/database/mod.rs`
10+
11+
## Overview
12+
13+
Deliver the minimum independently deployable PostgreSQL feature: users can configure the deployer to use an existing external PostgreSQL server (Bring Your Own DB), render valid artifacts, and run tracker without managed local PostgreSQL service support.
14+
15+
This slice intentionally excludes local PostgreSQL container provisioning and backup support. Those are delivered in later slices.
16+
17+
## Deployable Outcome
18+
19+
After this slice, a user can provide:
20+
21+
- `driver = postgresql`
22+
- `host`, `port`, `database_name`, `username`, `password`
23+
24+
and successfully run `create -> provision -> configure -> release -> run` using an external PostgreSQL endpoint.
25+
26+
## Goals
27+
28+
- [ ] JSON schema validates PostgreSQL database configuration.
29+
- [ ] DTO conversion supports PostgreSQL.
30+
- [ ] Domain database enum includes PostgreSQL variant and constants.
31+
- [ ] Rendered `.env` and `tracker.toml` support PostgreSQL DSN override path.
32+
- [ ] Validation errors are actionable and consistent with existing style.
33+
34+
## Inside-Out Execution Order
35+
36+
Implement this slice from runtime internals toward user-facing contract:
37+
38+
1. Templating/runtime internals: add PostgreSQL DSN and driver mapping in rendering contexts and manually validate rendered artifacts with a controlled input.
39+
2. Automation and workflow: ensure release flow does not accidentally require managed DB service for external PostgreSQL mode.
40+
3. Command wiring: complete DTO/domain conversion so runtime path is exercised through command handlers.
41+
4. Presentation and errors: ensure user-visible output and validation errors are actionable.
42+
5. Schema hardening: finalize JSON schema branch after runtime behavior is proven.
43+
6. Slice gate: run tests plus one end-to-end external PostgreSQL scenario before closing.
44+
45+
## Architecture Requirements
46+
47+
**DDD Layer**: Domain + Application
48+
**Module Path**: `src/domain/tracker/config/core/database/`, `src/application/command_handlers/create/config/tracker/`
49+
**Pattern**: Value objects + DTO-to-domain conversion
50+
51+
### Module Structure Requirements
52+
53+
- [ ] Keep database-driver-specific validation in domain config modules.
54+
- [ ] Keep schema/DTO constraints aligned with domain invariants.
55+
- [ ] Do not leak infrastructure concerns into domain validation.
56+
57+
### Architectural Constraints
58+
59+
- [ ] No breaking changes to serialized format for existing environments.
60+
- [ ] Existing persisted environment files remain loadable.
61+
- [ ] Error text stays user-oriented and action-guiding.
62+
63+
## Code-Level Implementation Details
64+
65+
### 0. Existing Code Paths To Extend
66+
67+
- Schema: `schemas/environment-config.json`
68+
- DTO enum + conversion: `src/application/command_handlers/create/config/tracker/tracker_core_section.rs`
69+
- Domain DB enum: `src/domain/tracker/config/core/database/mod.rs`
70+
- Domain DB types: `src/domain/tracker/config/core/database/mysql.rs` and `sqlite.rs` (reference patterns)
71+
- Tracker context driver mapping: `src/infrastructure/templating/tracker/template/wrapper/tracker_config/context.rs`
72+
- Env DSN generation: `src/infrastructure/templating/docker_compose/template/wrappers/env/context.rs`
73+
- Compose rendering service branch: `src/application/services/rendering/docker_compose.rs`
74+
75+
### 1. Schema Changes
76+
77+
Extend `DatabaseSection` oneOf in `schemas/environment-config.json` with a PostgreSQL branch:
78+
79+
- `driver`: `postgresql`
80+
- Required fields:
81+
- `host`
82+
- `port`
83+
- `database_name`
84+
- `username`
85+
- `password`
86+
- No `ssl_mode` field in this slice.
87+
88+
Rationale: keep DTO/domain/rendering aligned with currently supported tracker override shape and avoid introducing config keys not consumed end-to-end.
89+
90+
### 2. DTO Changes (`tracker_core_section.rs`)
91+
92+
Update `DatabaseSection` enum in `tracker_core_section.rs` to add `Postgresql { ... }`, mirroring MySQL layout with PostgreSQL naming.
93+
94+
Implement `TryFrom<DatabaseSection> for DatabaseConfig` branch to produce domain `DatabaseConfig::Postgresql(...)`.
95+
96+
### 3. Domain Changes (`core/database`)
97+
98+
In `src/domain/tracker/config/core/database/`:
99+
100+
- Add `postgresql.rs` with `PostgresqlConfig` and `PostgresqlConfigError`.
101+
- Add `DRIVER_POSTGRESQL` constant.
102+
- Add `DatabaseConfig::Postgresql(PostgresqlConfig)` variant.
103+
- Update helper methods:
104+
- `driver_name()`
105+
- `database_name()`
106+
- `docker_image()` behavior returns `None` in this slice for PostgreSQL, because this slice is external DB only.
107+
108+
Validation should mirror MySQL quality:
109+
110+
- non-empty host
111+
- port != 0
112+
- non-empty database_name
113+
- non-empty username
114+
- non-empty password
115+
- no reserved username policy in this slice
116+
117+
### 4. Rendering Changes (External DB Mode)
118+
119+
- Extend `DatabaseDriver` enum in tracker context (`tracker_config/context.rs`) with `Postgresql` and map `DatabaseConfig::Postgresql(..)`.
120+
- Extend env context (`env/context.rs`) with PostgreSQL DSN constructor (parallel to `new_with_mysql`) but without PostgreSQL service block.
121+
- Update `DockerComposeTemplateRenderingService` (`rendering/docker_compose.rs`) to route PostgreSQL DB to env/tracker context creation and to avoid enabling DB service dependency.
122+
- Keep `templates/tracker/tracker.toml.tera` as SQL-driver DSN override model (`path` only for sqlite3).
123+
124+
### 5. Explicit Non-Goals In This Slice
125+
126+
- No `postgresql` service section in `docker-compose.yml.tera`.
127+
- No release step creating PostgreSQL storage directories.
128+
- No backup support for PostgreSQL.
129+
130+
### 6. Backward Compatibility
131+
132+
Confirm existing environment files deserialize unchanged for:
133+
134+
- `driver=sqlite3`
135+
- `driver=mysql`
136+
137+
## Implementation Plan
138+
139+
### Phase 1: Runtime Rendering Internals (0.5 day)
140+
141+
- [ ] Task 1.1: Extend tracker context driver enum/mapping.
142+
- [ ] Task 1.2: Add PostgreSQL DSN constructor in env context.
143+
- [ ] Task 1.3: Extend docker compose rendering service branch logic for external mode.
144+
145+
### Phase 2: Domain + Command Wiring (1 day)
146+
147+
- [ ] Task 2.1: Add `postgresql.rs` domain config with invariants.
148+
- [ ] Task 2.2: Add `DatabaseSection::Postgresql` in DTO enum.
149+
- [ ] Task 2.3: Extend DTO-to-domain conversion.
150+
- [ ] Task 2.4: Confirm workflow path treats this slice as external DB mode.
151+
152+
### Phase 3: Presentation + Schema Finalization (0.5 day)
153+
154+
- [ ] Task 3.1: Ensure actionable validation/output text for PostgreSQL config errors.
155+
- [ ] Task 3.2: Add PostgreSQL branch to `schemas/environment-config.json` as final contract exposure.
156+
157+
### Phase 4: Tests + Regression Safety (1 day)
158+
159+
- [ ] Task 4.1: Add DTO serde/try_from tests in `tracker_core_section.rs` test module.
160+
- [ ] Task 4.2: Add domain config tests in `core/database/mod.rs` + `postgresql.rs`.
161+
- [ ] Task 4.3: Add env context DSN encoding tests in `env/context.rs`.
162+
- [ ] Task 4.4: Add tracker context mapping tests in `tracker_config/context.rs`.
163+
- [ ] Task 4.5: Re-run existing sqlite/mysql tests touched by enum branching.
164+
- [ ] Task 4.6: Run one end-to-end external PostgreSQL scenario.
165+
166+
## Acceptance Criteria
167+
168+
- [ ] `EnvironmentCreationConfig` accepts valid PostgreSQL external-db config.
169+
- [ ] Rendering produces PostgreSQL DSN override for tracker.
170+
- [ ] Deployment works when PostgreSQL is externally available.
171+
- [ ] Invalid PostgreSQL config yields clear validation errors.
172+
- [ ] Existing SQLite/MySQL config parsing and rendering remain unchanged.
173+
- [ ] Unit/integration tests for added branches are present and passing.
174+
175+
## Notes
176+
177+
Keep naming canonical as `postgresql` (not `postgres`) unless tracker integration proves otherwise.

‎docs/issues/469-02-compose-and-template-rendering-postgresql.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
# Vertical Slice B: Managed PostgreSQL Service (Compose + Release)
2+
3+
**Issue**: TBD (`469-02` planned child subissue)
4+
**Parent Epic**: #469 - Add PostgreSQL Support To Tracker Deployer
5+
**Related**:
6+
7+
- `templates/docker-compose/.env.tera`
8+
- `templates/docker-compose/docker-compose.yml.tera`
9+
- `templates/ansible/variables.yml.tera`
10+
- `templates/ansible/create-mysql-storage.yml`
11+
- `src/application/services/rendering/docker_compose.rs`
12+
- `src/application/command_handlers/release/workflow.rs`
13+
- `src/application/command_handlers/release/steps/mysql.rs`
14+
- `src/domain/topology/service.rs`
15+
- `src/domain/environment/state/release_failed.rs`
16+
- `src/infrastructure/templating/docker_compose/template/wrappers/env/context.rs`
17+
18+
## Overview
19+
20+
Extend the external PostgreSQL support from Slice A to a fully managed local PostgreSQL deployment mode, including compose service definition, topology wiring, and release-time remote storage preparation.
21+
22+
Generated artifacts and workflow should be sufficient to deploy PostgreSQL alongside tracker on the target host.
23+
24+
## Deployable Outcome
25+
26+
After this slice, users can run the full workflow using PostgreSQL with local service provisioning:
27+
28+
- compose file includes `postgresql` service
29+
- tracker depends on healthy `postgresql`
30+
- release prepares PostgreSQL storage directory on remote host
31+
- `show`/status-level service metadata includes PostgreSQL image when applicable
32+
33+
## Goals
34+
35+
- [ ] Render `.env` with PostgreSQL DSN overrides.
36+
- [ ] Render `docker-compose.yml` with PostgreSQL service and healthcheck.
37+
- [ ] Add release workflow steps to prepare PostgreSQL storage on remote host.
38+
- [ ] Add topology/service model support for PostgreSQL.
39+
- [ ] Preserve current SQLite/MySQL behavior.
40+
41+
## Inside-Out Execution Order
42+
43+
Implement this slice from runtime internals toward user-facing contract:
44+
45+
1. Templating/runtime internals: implement managed PostgreSQL compose/env rendering and validate by running generated artifacts manually.
46+
2. Automation and workflow: add ansible storage prep and release step wiring for managed PostgreSQL.
47+
3. Command wiring: ensure command handlers route managed PostgreSQL mode through new workflow branches.
48+
4. Presentation and errors: expose service/image/status and actionable failure messages.
49+
5. Schema/contract refinement: confirm user-facing config contract remains aligned with implemented managed mode.
50+
6. Slice gate: tests plus one managed PostgreSQL end-to-end run before closing.
51+
52+
## Code-Level Implementation Details
53+
54+
### 0. Existing Code Paths To Extend
55+
56+
- Compose env context: `src/infrastructure/templating/docker_compose/template/wrappers/env/context.rs`
57+
- Compose builder/database context: `src/infrastructure/templating/docker_compose/template/wrappers/docker_compose/context/database.rs`, `.../builder.rs`, `.../mod.rs`
58+
- Compose rendering service: `src/application/services/rendering/docker_compose.rs`
59+
- Topology service enum: `src/domain/topology/service.rs`
60+
- Release workflow + step modules: `src/application/command_handlers/release/workflow.rs`, `src/application/command_handlers/release/steps/mysql.rs`
61+
- Release state steps: `src/domain/environment/state/release_failed.rs`
62+
- Ansible templates + static copy list: `templates/ansible/variables.yml.tera`, `templates/ansible/create-mysql-storage.yml`, `src/infrastructure/templating/ansible/template/renderer/project_generator.rs`
63+
- Show command docker image logic: `src/application/command_handlers/show/handler.rs`
64+
65+
- Correct tracker database driver override
66+
- Correct PostgreSQL DSN override
67+
- Optional local PostgreSQL service definition
68+
- Correct service dependency and healthcheck behavior
69+
70+
## Architecture Requirements
71+
72+
**DDD Layer**: Application + Infrastructure
73+
**Module Path**: `src/application/services/rendering/`, `src/infrastructure/templating/docker_compose/`, `templates/`
74+
**Pattern**: Context builder + Tera templating
75+
76+
### Architectural Constraints
77+
78+
- [ ] No secrets hardcoded into template files.
79+
- [ ] DSN generation escapes reserved URL characters in user/password.
80+
- [ ] Generated output remains deterministic in field ordering and optional blocks.
81+
82+
## Specifications
83+
84+
### 1. EnvContext + DSN
85+
86+
Extend `EnvContext` with PostgreSQL mode:
87+
88+
- Add optional `postgresql` service config block analogous to `mysql`.
89+
- Add constructor similar to `new_with_mysql`, e.g. `new_with_postgresql(...)`.
90+
- Ensure DSN format uses `postgresql://user:pass@host:port/database`.
91+
- Keep percent-encoding behavior for user/password.
92+
93+
### 2. Docker Compose Context + Templates
94+
95+
Extend builder/context to support PostgreSQL service setup:
96+
97+
- Add PostgreSQL setup config type.
98+
- Add `with_postgresql(...)` on builder.
99+
- Add optional `postgresql` service context in final compose context.
100+
101+
### 3. Template Updates
102+
103+
#### `.env.tera`
104+
105+
- Render `TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER='postgresql'` when PostgreSQL selected.
106+
- Render `TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__PATH` with PostgreSQL DSN.
107+
- Add PostgreSQL service env vars (container-specific) when local PostgreSQL service is enabled.
108+
109+
#### `docker-compose.yml.tera`
110+
111+
- Add `postgresql` service block with:
112+
- image
113+
- environment
114+
- volume mount path
115+
- healthcheck
116+
- database network
117+
- Make tracker `depends_on` target `postgresql` in PostgreSQL mode.
118+
- Ensure MySQL and PostgreSQL blocks are mutually exclusive per selected driver.
119+
120+
### 3. Topology + Show Metadata
121+
122+
- Add `Service::PostgreSQL` in `src/domain/topology/service.rs` and include serialization/name/all() updates.
123+
- Update topology derivations that currently check `Service::MySQL` when behavior should apply to any managed SQL service.
124+
- Update show command (`show/handler.rs`) to include PostgreSQL image when tracker uses managed PostgreSQL.
125+
126+
### 4. Release + Ansible Storage Preparation
127+
128+
- Add `postgresql_enabled` in `templates/ansible/variables.yml.tera`.
129+
- Add `templates/ansible/create-postgresql-storage.yml` (parallel to mysql playbook).
130+
- Register new playbook in static template copy list (`project_generator.rs`).
131+
- Add release step module for PostgreSQL storage prep (parallel to `steps/mysql.rs`).
132+
- Add state step and error mapping for PostgreSQL storage failures.
133+
134+
### 5. Rendering Service Wiring
135+
136+
Update `DockerComposeTemplateRenderingService` decision tree to handle 3-way DB selection:
137+
138+
- sqlite -> existing path
139+
- mysql -> existing path
140+
- postgresql -> new path (managed mode)
141+
142+
## Implementation Plan
143+
144+
### Phase 1: Compose Context + Templates (1 day)
145+
146+
- [ ] Task 1.1: Extend env context and compose context builders for managed PostgreSQL.
147+
- [ ] Task 1.2: Update `.env.tera` and `docker-compose.yml.tera` blocks.
148+
- [ ] Task 1.3: Add rendering tests for managed PostgreSQL snapshots.
149+
150+
### Phase 2: Release + Ansible (1 day)
151+
152+
- [ ] Task 2.1: Add `create-postgresql-storage.yml` + ansible variable wiring.
153+
- [ ] Task 2.2: Add PostgreSQL release step + workflow integration + error mapping.
154+
- [ ] Task 2.3: Add release step unit tests (skip/execute/failure mapping).
155+
156+
### Phase 3: Command + Topology + Show (0.5 day)
157+
158+
- [ ] Task 3.1: Add `Service::PostgreSQL` and update topology-related tests.
159+
- [ ] Task 3.2: Update show command docker image reporting.
160+
- [ ] Task 3.3: Verify command-handler routing for managed PostgreSQL path.
161+
162+
### Phase 4: Presentation + Contract Check + E2E Gate (0.5 day)
163+
164+
- [ ] Task 4.1: Validate PostgreSQL-specific release failure messages are actionable.
165+
- [ ] Task 4.2: Confirm schema/docs contract is still aligned with managed mode behavior.
166+
- [ ] Task 4.3: Run one managed PostgreSQL end-to-end scenario.
167+
168+
## Acceptance Criteria
169+
170+
- [ ] Managed PostgreSQL deployments render valid `.env` and compose files with healthy dependency checks.
171+
- [ ] Release workflow creates required PostgreSQL storage directories on remote host.
172+
- [ ] Topology and show output include PostgreSQL where appropriate.
173+
- [ ] SQLite/MySQL snapshots and release behavior remain unchanged unless intentionally updated.
174+
- [ ] New/updated tests for templates, topology, and release step are passing.

0 commit comments

Comments
 (0)