|
| 1 | +# ADR-0057: System data has a lifecycle — declarative retention, rotation, and reclamation for platform-generated objects |
| 2 | + |
| 3 | +**Status**: Proposed (2026-06-20) |
| 4 | +**Deciders**: ObjectStack Protocol Architects |
| 5 | +**Builds on**: [ADR-0052](./0052-audit-is-not-the-activity-feed.md) (decomposed audit / activity / collaboration into bounded contexts — this ADR adds the *orthogonal* axis those contexts never specified: **how long each one lives and how its space is reclaimed**), [ADR-0049](./0049-no-unenforced-security-properties.md) (enforce-or-remove — a declared `retention` that drives no sweeper is dead surface; this ADR wires it to a runtime consumer), [ADR-0030](./0030-notification-platform-convergence.md) (notification objects are messaging-owned with their own lifecycle), [ADR-0021](./0021-analytics-dataset-semantic-layer.md) (precedent for moving event-shaped data off the primary OLTP store) |
| 6 | +**Consumers**: `@objectstack/spec` (`lifecycle` object property + `LifecycleClass`), `@objectstack/objectql` (LifecycleService: Reaper / Rotator / Archiver), `@objectstack/plugin-audit` (audit/activity exclusion of telemetry objects — shipped P0), `@objectstack/driver-sql` (`auto_vacuum=INCREMENTAL` default + incremental-vacuum hook — shipped P0), `@objectstack/dogfood` (storage-growth regression gate), platform spec-liveness gate |
| 7 | +**Pilot**: `app-showcase` (its 20s `showcase_scheduled_digest` flow grew `dev.db` to 260 MB+ over a multi-day `pnpm dev` — the symptom this ADR generalizes from) |
| 8 | + |
| 9 | +--- |
| 10 | + |
| 11 | +## TL;DR |
| 12 | + |
| 13 | +The platform can declare an object's structure, validations, permissions, and |
| 14 | +relationships — but **not how long its data should live**. Every |
| 15 | +platform-generated object therefore defaults to *immortal*. Any high-frequency |
| 16 | +write source then grows the database without bound, and because the SQLite |
| 17 | +driver ships `auto_vacuum=NONE`, the file never shrinks even after rows are |
| 18 | +deleted. |
| 19 | + |
| 20 | +A 20-second scheduled digest flow in `app-showcase` demonstrated this: each tick |
| 21 | +fanned out to a job-run row, a notification (+ delivery + receipt), and an inbox |
| 22 | +message — **and every one of those writes was mirrored into both `sys_audit_log` |
| 23 | +and `sys_activity`** (~21 append-only rows/tick, of which audit+activity were |
| 24 | +~76%). Over a multi-day dev session this reached **260 MB of pure |
| 25 | +platform-generated telemetry — zero business data**. |
| 26 | + |
| 27 | +This ADR makes **data lifecycle a first-class, declarable, runtime-enforced |
| 28 | +metadata concern**, the same posture the platform already takes for validation |
| 29 | +and security: |
| 30 | + |
| 31 | +1. **Classify** every object by `lifecycleClass`: `record` | `audit` | |
| 32 | + `telemetry` | `transient` | `event`. |
| 33 | +2. **Declare** retention / rotation / archival on the object metadata. |
| 34 | +3. **Enforce** it with a single platform-owned **LifecycleService** |
| 35 | + (Reaper + Rotator + Archiver) — not N per-plugin implementations. |
| 36 | +4. **Reclaim** space (SQLite `auto_vacuum=INCREMENTAL` + incremental vacuum). |
| 37 | +5. **Stop the amplifier**: operational telemetry is excluded from the |
| 38 | + audit+activity writer at the seam (ADR-0052's event-spine finishes this). |
| 39 | +6. **Gate it**: spec-liveness requires a `lifecycle` declaration on every |
| 40 | + non-`record` system object; a dogfood test asserts bounded growth so this |
| 41 | + class of regression turns CI red instead of filling a disk. |
| 42 | + |
| 43 | +--- |
| 44 | + |
| 45 | +## 1. Context — how a few seed records became 260 MB |
| 46 | + |
| 47 | +Forensics on the showcase `dev.db`: |
| 48 | + |
| 49 | +- The file was **261 MB**; a freshly seeded one is **2.3 MB**. ~110×. |
| 50 | +- **No business table grew.** `showcase_*`, `crm_*`, `pm_*` are the same seed |
| 51 | + rows every boot (upsert overwrites). The growth was entirely in append-only |
| 52 | + platform tables. |
| 53 | +- **Driver pragmas:** `auto_vacuum=NONE`, `journal_mode=delete`, |
| 54 | + `freelist_count=0` → freed pages are never returned to the OS; the file is |
| 55 | + pinned at its high-water mark. |
| 56 | +- **Two always-on scheduled jobs** drove the writes: |
| 57 | + `showcase_scheduled_digest` (`interval`, **20 000 ms**) and |
| 58 | + `approvals-sla-escalation` (`interval`, 300 000 ms). |
| 59 | +- **Measured fan-out** (controlled ~120 s run): `sys_audit_log` +120, |
| 60 | + `sys_activity` +120, and `sys_job_run` / `sys_notification` / |
| 61 | + `sys_notification_delivery` / `sys_notification_receipt` / `sys_inbox_message` |
| 62 | + +15 each. Audit+activity were **~76 %** of all new rows — the dual-write |
| 63 | + amplifier described in ADR-0052 §1. |
| 64 | +- **Rate:** ~+400 KB / 120 s ≈ **~290 MB/day** of continuous running. |
| 65 | + |
| 66 | +Root causes, in order of leverage: |
| 67 | + |
| 68 | +| # | Cause | Effect | |
| 69 | +|---|-------|--------| |
| 70 | +| 1 | No retention contract on any platform-generated object | append-only tables grow forever | |
| 71 | +| 2 | Audit+activity dual-write on *every* mutation incl. internal plumbing | 4–8× row amplification | |
| 72 | +| 3 | `auto_vacuum=NONE` | deleting rows would not shrink the file anyway | |
| 73 | +| 4 | Demo-tuned 20 s interval | turns a slow leak into a fast one | |
| 74 | + |
| 75 | +ADR-0052 already named #2 as a design flaw and set the long-term fix (an event |
| 76 | +spine; audit becomes a pure sink). This ADR addresses #1, #3 — the lifecycle |
| 77 | +axis — and ships the immediate mitigations for #2 and #4. |
| 78 | + |
| 79 | +## 2. The principle — telemetry is not a system of record |
| 80 | + |
| 81 | +The platform conflates two data kinds with opposite contracts into one OLTP |
| 82 | +store with one persistence policy: |
| 83 | + |
| 84 | +| Kind | Examples | True contract | |
| 85 | +|------|----------|---------------| |
| 86 | +| **System of record** | account, project, invoice, `sys_audit_log` | durable, retained, often immutable | |
| 87 | +| **Operational telemetry** | activity, job_run, notification_*, inbox, http_delivery, ai_traces | value decays with time; **must be bounded** | |
| 88 | + |
| 89 | +Every mature low-code platform separates these and bounds the second: |
| 90 | + |
| 91 | +| Platform | Mechanism | |
| 92 | +|---|---| |
| 93 | +| **Salesforce** | **Big Objects** for high-volume/historical; **Field Audit Trail** `HistoryRetentionPolicy` (archive ≤10y); **Platform Events** retained ≤72h | |
| 94 | +| **ServiceNow** | **Table Rotation** (time-sharded, DROP oldest shard = O(1) reclaim) + **Table Cleaner** (`sys_auto_flush`, delete by table/field/age) | |
| 95 | +| **Dataverse / Power Platform** | **Elastic tables** with native **TTL** column; **Bulk Deletion** jobs; configurable **audit retention**; long-term retention to a data lake | |
| 96 | +| **OutSystems** | logs in a **separate database** with configurable retention + cleanup | |
| 97 | +| **Mendix** | **non-persistable entities** (memory-only) for transient state | |
| 98 | + |
| 99 | +The consistent answer: **declarative, bounded-by-default lifecycle, enforced by a |
| 100 | +platform-owned sweeper, with space actually reclaimed.** |
| 101 | + |
| 102 | +## 3. Decision |
| 103 | + |
| 104 | +### 3.1 Classify — `lifecycleClass` on every object |
| 105 | + |
| 106 | +| Class | Meaning | Examples | Default contract | |
| 107 | +|---|---|---|---| |
| 108 | +| `record` | business truth | account, project, invoice | permanent, recoverable | |
| 109 | +| `audit` | compliance ledger | `sys_audit_log`, `sys_metadata_audit` | retain → archive → delete | |
| 110 | +| `telemetry` | high-frequency log / run flow | `sys_activity`, `sys_job_run`, `sys_notification_delivery`, `ai_traces`, `sys_http_delivery` | **rotation**, short retention | |
| 111 | +| `transient` | workflow / ephemeral state | `sys_notification_receipt`, read inbox, `sys_device_code` | **TTL** auto-expire | |
| 112 | +| `event` | event-bus messages | scheduled/trigger fan-out | very short TTL (hours) | |
| 113 | + |
| 114 | +`record` is the safe default (back-compat: undeclared objects keep today's |
| 115 | +behavior). The spec-liveness gate (§3.5) requires an explicit `lifecycle` on any |
| 116 | +object declared `audit`/`telemetry`/`transient`/`event`. |
| 117 | + |
| 118 | +### 3.2 Declare — `lifecycle` object metadata |
| 119 | + |
| 120 | +```ts |
| 121 | +defineObject({ |
| 122 | + name: 'sys_activity', |
| 123 | + lifecycle: { |
| 124 | + class: 'telemetry', |
| 125 | + retention: { maxAge: '14d' }, |
| 126 | + storage: { strategy: 'rotation', shards: 14, unit: 'day' }, // DROP oldest shard |
| 127 | + reclaim: true, |
| 128 | + }, |
| 129 | +}) |
| 130 | + |
| 131 | +defineObject({ |
| 132 | + name: 'sys_audit_log', |
| 133 | + lifecycle: { |
| 134 | + class: 'audit', |
| 135 | + retention: { maxAge: '90d' }, // hot window |
| 136 | + archive: { after: '90d', to: 'datalake', keep: '7y' }, |
| 137 | + strategy: 'archive-then-delete', |
| 138 | + }, |
| 139 | +}) |
| 140 | + |
| 141 | +defineObject({ |
| 142 | + name: 'sys_notification_receipt', |
| 143 | + lifecycle: { |
| 144 | + class: 'transient', |
| 145 | + ttl: { field: 'created_at', expireAfter: '7d' }, |
| 146 | + }, |
| 147 | +}) |
| 148 | +``` |
| 149 | + |
| 150 | +Policies are overridable per environment / tenant via `SettingsServicePlugin` |
| 151 | +(regulated tenants set years; dev sets days). |
| 152 | + |
| 153 | +### 3.3 Enforce — one platform-owned LifecycleService |
| 154 | + |
| 155 | +``` |
| 156 | +LifecycleService (scans every object carrying a `lifecycle` declaration) |
| 157 | +├── Reaper — delete by TTL/age in batches (transient + low-freq telemetry); then incremental_vacuum [≈ ServiceNow Table Cleaner] |
| 158 | +├── Rotator — time-shard high-freq telemetry; rotate by DROPping the oldest shard (O(1), real reclaim) [≈ ServiceNow Table Rotation] |
| 159 | +└── Archiver — copy audit cold data to an archive datasource, then delete from the hot store [≈ SF Field Audit Trail / Dataverse data lake] |
| 160 | +``` |
| 161 | + |
| 162 | +Hard rule: **the LifecycleService's own deletes/rotations are not audited or |
| 163 | +activity-logged** (otherwise cleanup re-feeds the tables it is draining — the |
| 164 | +same self-audit trap ADR-0052 already guards). Aggregate one summary row at |
| 165 | +most. |
| 166 | + |
| 167 | +### 3.4 Reclaim — driver space hygiene |
| 168 | + |
| 169 | +SQLite driver defaults to `auto_vacuum=INCREMENTAL` (shipped P0); the Reaper |
| 170 | +issues `PRAGMA incremental_vacuum` after a sweep. `auto_vacuum` only re-lays-out |
| 171 | +a *fresh* DB (or one after a one-time `VACUUM`), so existing files need a single |
| 172 | +`VACUUM` to adopt it — acceptable, since the Reaper / a `db:clean` covers legacy |
| 173 | +files. |
| 174 | + |
| 175 | +### 3.5 Gate — make the regression impossible to reintroduce |
| 176 | + |
| 177 | +- **Spec-liveness**: any object with `class ∈ {audit, telemetry, transient, |
| 178 | + event}` and no `lifecycle.retention`/`ttl`/`rotation` ⇒ CI red. (Mirrors |
| 179 | + ADR-0049 enforce-or-remove: a zero-retention telemetry object is a false |
| 180 | + surface.) |
| 181 | +- **Dogfood storage-growth gate** (`@objectstack/dogfood`): boot an example |
| 182 | + app, let scheduled timers tick N times, assert each telemetry table ≤ its |
| 183 | + bound and DB-file delta ≤ threshold. Reverting any retention policy makes it |
| 184 | + red — the golden test for this class. |
| 185 | + |
| 186 | +### 3.6 Physically separate telemetry (target state) |
| 187 | + |
| 188 | +`telemetry`/`audit`/`event` objects move to a dedicated `datasource` (the |
| 189 | +platform already supports `defaultDatasource`). Even on SQLite that is a |
| 190 | +separate file, so telemetry bloat can never again pollute the business DB, and |
| 191 | +the store can later be swapped for an append-only log / time-series / object |
| 192 | +store. This is the ADR-0021 dataset-migration pattern applied to event-shaped |
| 193 | +system data. |
| 194 | + |
| 195 | +## 4. Rollout |
| 196 | + |
| 197 | +| Phase | Scope | Status | |
| 198 | +|---|---|---| |
| 199 | +| **P0 — stop the bleed** | (a) exclude operational/plumbing objects from the audit+activity writer (`plugin-audit` `SKIP_OBJECTS`); (b) SQLite `auto_vacuum=INCREMENTAL` driver default; (c) showcase digest interval 20s→60s, flagged demo-only | **this ADR ships P0** | |
| 200 | +| **P1 — contract** | `lifecycleClass` + `lifecycle` spec; LifecycleService Reaper; spec-liveness enforcement; dogfood growth gate | proposed | |
| 201 | +| **P2 — rotation/TTL** | Rotator (shard + DROP) for high-freq telemetry; transient TTL expiry | proposed | |
| 202 | +| **P3 — separation** | telemetry/audit on a dedicated datasource; Archiver cold-store | proposed | |
| 203 | +| **P4 — governance** | per-table storage quotas, growth alerts, tenant-level retention overrides | proposed | |
| 204 | + |
| 205 | +P0 alone removes ~76 % of the row growth (audit/activity exclusion) and a |
| 206 | +further 3× (interval), and lets space be reclaimed — without any schema change |
| 207 | +or migration. P1+ makes it bounded by construction. |
| 208 | + |
| 209 | +## 5. Consequences |
| 210 | + |
| 211 | +- **Positive.** Platform-generated data is bounded by default and reclaimable. |
| 212 | + Retention becomes a one-line declaration, enforced uniformly. Audit becomes a |
| 213 | + clean ledger (finishing ADR-0052's direction). The 260 MB regression becomes a |
| 214 | + CI assertion. Compliance retention (years) and dev ergonomics (days) are the |
| 215 | + same knob at different settings. |
| 216 | +- **Cost.** A new spec property + a platform service + a datasource split. |
| 217 | + Sequenced P0→P4 so each step is independently shippable and back-compatible. |
| 218 | +- **Back-compat.** Undeclared objects default to `record` (today's behavior). |
| 219 | + Object names and schemas are unchanged; only lifecycle policy and (P3) owning |
| 220 | + datasource change. The P0 audit exclusion only *stops creating* telemetry |
| 221 | + audit/activity rows — it neither reads nor deletes existing ones. |
| 222 | + |
| 223 | +## 6. Alternatives considered |
| 224 | + |
| 225 | +- **Just delete `dev.db` periodically.** Treats the symptom; production and |
| 226 | + long-lived dev DBs still grow unbounded, and nothing stops the next |
| 227 | + high-frequency writer. |
| 228 | +- **Skip auditing by `context.isSystem`.** Too broad — the seed loader and |
| 229 | + server-side automations write *business* records with `isSystem: true`; |
| 230 | + dropping those from the ledger is a real compliance hole. The precise axis is |
| 231 | + the object's lifecycle class, not who wrote it. |
| 232 | +- **Per-plugin cleanup jobs.** Produces N inconsistent implementations and |
| 233 | + re-introduces the self-audit trap each time. Lifecycle is a platform |
| 234 | + primitive, owned once. |
| 235 | +- **Lower the interval only.** Reduces the rate but leaves growth unbounded over |
| 236 | + time and does not reclaim space. |
0 commit comments