|
| 1 | += Architecture Decision Record: 0011-sidecar-storage-is-relational |
| 2 | +<!-- SPDX-License-Identifier: PMPL-1.0-or-later --> |
| 3 | +<!-- Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk> --> |
| 4 | + |
| 5 | +# 11. The sidecar octad store is relational; drop the JSON backend |
| 6 | + |
| 7 | +Date: 2026-05-30 |
| 8 | + |
| 9 | +## Status |
| 10 | + |
| 11 | +Accepted (2026-05-30) — implemented in #144 (json dropped from the storage |
| 12 | +enum, docs, template, and validate/doctor messaging) with test coverage in |
| 13 | +#147. Closes #112 (V-L2-F2). Builds on #45 (V-L2-F1, the per-dialect DDL |
| 14 | +split) and ADR-0004 (octad ontology). |
| 15 | + |
| 16 | +## Context |
| 17 | + |
| 18 | +The manifest's `[sidecar].storage` historically advertised `sqlite | json`, |
| 19 | +but only the SQLite path — and later a PostgreSQL DDL dialect — had code. |
| 20 | + |
| 21 | +* **#45 (V-L2-F1)** split the sidecar DDL by dialect and stopped silently |
| 22 | + emitting SQLite DDL for a `json` store, rejecting `storage = "json"` at |
| 23 | + `generate` time with a pointer to #112. |
| 24 | +* **#112 (V-L2-F2)** posed the decision squarely: *implement* a JSON |
| 25 | + document store mirroring the `verisimdb_*` tables (with the same |
| 26 | + octad-dimension coverage as the SQLite path, honoured by |
| 27 | + `generate`/`drift`/`gc`), or *drop* `json` from the schema, docs, and |
| 28 | + validation. |
| 29 | +
|
| 30 | +## Decision |
| 31 | + |
| 32 | +Drop `json`. The supported `[sidecar].storage` set is **closed** to: |
| 33 | + |
| 34 | +* `sqlite` — the reference runtime store, used by `generate`, `drift`, and |
| 35 | + `gc`; and |
| 36 | +* `postgres` / `postgresql` — a `generate`-time DDL dialect. |
| 37 | +
|
| 38 | +`codegen::overlay::SqlDialect::from_storage` is the single source of truth |
| 39 | +for accepted values. `validate`/`doctor` (the `sidecar-storage-supported` |
| 40 | +check) and `generate` all defer to it, so an unsupported store is rejected |
| 41 | +uniformly and up front rather than only at code-generation time, and with no |
| 42 | +"coming soon" pointer. |
| 43 | + |
| 44 | +### Why relational, not document |
| 45 | + |
| 46 | +The octad data layer's correctness rests on invariants that SQL enforces |
| 47 | +*structurally*, and that a JSON document store would have to re-implement in |
| 48 | +application code with weaker guarantees: |
| 49 | + |
| 50 | +* **Provenance** is an append-only hash chain advanced under a |
| 51 | + `BEGIN IMMEDIATE` write transaction; the multi-head tip set and the |
| 52 | + duplicate guard are the `hash` PRIMARY KEY plus a non-unique predecessor |
| 53 | + index (ADR-0010). |
| 54 | +* **Temporal** versioning enforces "exactly one current version per |
| 55 | + `(entity, table)`" with a partial `UNIQUE` index, and `valid_to >= |
| 56 | + valid_from` with a `CHECK`. |
| 57 | +* **Lineage** acyclicity begins with a self-edge `CHECK` at the storage |
| 58 | + layer (ADR-0005). |
| 59 | +* **Closed enums** (`operation`, `derivation_type`, `access_level`, |
| 60 | + `status`) are `CHECK` constraints; simulation branch parentage is a |
| 61 | + self-referencing foreign key. |
| 62 | +
|
| 63 | +There is also no storage-abstraction layer to host a second *runtime* |
| 64 | +backend — provenance, temporal, drift, and gc are all SQLite-specific — and |
| 65 | +`postgres` itself is already `generate`-only. A half-built JSON store would |
| 66 | +re-introduce exactly the advertised-but-unimplemented surface #45 set out to |
| 67 | +remove. |
| 68 | + |
| 69 | +## Consequences |
| 70 | + |
| 71 | +### Positive |
| 72 | + |
| 73 | +- The advertised configuration surface is honest: every accepted |
| 74 | + `[sidecar].storage` value has code behind it. |
| 75 | +- One validation path — `validate`, `doctor`, and `generate` cannot |
| 76 | + disagree on the accepted set. |
| 77 | +- No application-layer re-encoding of invariants the relational engine |
| 78 | + enforces for free. |
| 79 | +
|
| 80 | +### Negative |
| 81 | + |
| 82 | +- No document-store option. Should one be wanted later (e.g. an embedded, |
| 83 | + dependency-free flat-file deployment), it is a deliberate and larger piece |
| 84 | + of work: introduce a real storage trait, re-encode every storage-layer |
| 85 | + invariant listed above against it, and re-add the value to |
| 86 | + `from_storage`. Until then, `json` is simply another unsupported value. |
| 87 | +
|
| 88 | +## References |
| 89 | + |
| 90 | +- Issue #112 (V-L2-F2); issue context from #45 (V-L2-F1). |
| 91 | +- PRs #144 (drop) and #147 (test coverage). |
| 92 | +- ADR-0004 (octad ontology), ADR-0005 (lineage acyclicity), |
| 93 | + ADR-0010 (provenance forks are first-class). |
0 commit comments