feat(lifecycle): ADR-0057 P1–P4 — lifecycle contract, Reaper/Rotator/Archiver, datasource separation, governance (#2786)#2791
Merged
Conversation
…racked in #2786 The ADR header still said 'Proposed' with no pointer to the shipped P0 implementation or the new P1–P4 tracking issue. Per repo convention (cf. ADR-0030/0055/0069 headers), reflect implementation reality: - Status: Accepted — P0 shipped with this ADR (#2079); P1–P4 proposed, tracked in #2786 - §4 Rollout: link #2079 on the P0 row; add the #2786 tracking pointer Doc-only change; no code or policy content altered. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
github-actions
Bot
added
size/xs
documentation
…eaper, liveness gate, dogfood growth gate (#2786) The platform can declare an object's structure, validations and permissions — but not how long its data lives. P1 makes data lifecycle a first-class, declarable, runtime-enforced metadata concern: spec: - `lifecycle` object property (LifecycleSchema): class (record | audit | telemetry | transient | event) + retention.maxAge / ttl / storage(rotation) / archive / reclaim, with duration literals ('6h', '14d', '7y') - §3.5 enforce-or-remove invariants in a superRefine: a non-record class with no bounding policy is rejected; record-class policies are rejected; the archive window must start where the hot window ends - IDataDriver.reclaimSpace?() contract for post-sweep space reclamation objectql: - LifecycleService (Reaper), registered by ObjectQLPlugin as a platform service (default-on, OS_LIFECYCLE_DISABLED=1 / lifecycle.enabled=false to opt out): hourly sweep deletes rows past retention.maxAge (created_at) and past ttl.field + expireAfter, bounds rotation-declared telemetry by shards×unit until the P2 Rotator lands, and never hot-deletes an archive-declared audit ledger before the Archiver copies it out - Sweeps run bulk multi-deletes under a system context — at most ONE afterDelete hook per object per sweep, and telemetry sys_* objects are in the audit writer's SKIP_OBJECTS, so cleanup never re-feeds the tables it drains (the ADR-0052 self-audit trap) driver-sql: - reclaimSpace(): SQLite `PRAGMA incremental_vacuum` (the P0 ADR text claimed this hook shipped; it had not — this closes it), no-op on Postgres/MySQL platform objects (11 annotated): sys_activity 14d+rotation, sys_audit_log 90d→archive 7y, sys_metadata_audit 365d→archive, sys_job_run/sys_automation_run/sys_http_delivery 30d, notification pipeline (sys_notification / delivery / receipt / inbox) one 90d window, sys_device_code expires_at+1d. gates: - spec-liveness: object/lifecycle classified + bound to a dogfood proof (HIGH_RISK_CLASSES 'data-lifecycle') - dogfood storage-growth gate (@proof: adr0057-lifecycle-bounded-growth): boots showcase, proves contract coverage, reaps a backdated runaway writer, asserts record-class data untouched and archive-declared ledgers retained Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3
…(1) shard DROP (#2786) High-frequency telemetry declared with lifecycle.storage.strategy='rotation' is now physically time-sharded on SQLite (the ServiceNow Table Rotation model, ADR-0057 §3.3): driver-sql: - rotateShards(objectDef, nowMs): idempotent Rotator entry — ensures the current shard (<table>__r<key>: day/week/month UTC keys), adopts a legacy pre-rotation table as its first shard (no data loss), column-syncs every retained shard so the UNION stays uniform, DROPs shards past the shards×unit time window (O(1) reclaim), and rebuilds the read view - The base name becomes a READ-ONLY UNION ALL view (SQLite views reject writes and RETURNING), so every write path redirects shard-wise instead: create/bulkCreate/upsert target the current shard; by-id update/delete probe shards newest-first; updateMany/deleteMany/bulkDelete fan out and sum — reads (find/count/aggregate) are untouched - Per-shard bookkeeping aliases (datetime/date/json/boolean coercion, tenant scope, timestamp stamping) so a shard builder behaves exactly like the view - initObjects routes rotation-declared objects to the Rotator (the plain create/alter DDL path would collide with the view) - supportsRotation gates on the dialect; declared indexes get per-shard names objectql LifecycleService: - Rotator invocation each sweep when the driver supports it; the fallback age-based reap covers every other dialect so the declared bound holds everywhere, only the reclamation mechanics differ - An explicit retention.maxAge still trims inside live shards after rotation (shard drops are unit-granular; this also immediately bounds a legacy table the Rotator just adopted whole) - Dropped shards trigger the datasource incremental-vacuum pass Transient TTL expiry (the other P2 line item) shipped with the P1 Reaper — TTL and age reaping are one mechanism (§3.3). Proof: sql-driver-rotation.test.ts drives the full shard lifecycle (create → rotate → window drop → adoption → cross-shard writes); the dogfood storage-growth gate now shards a probe stream through the real engine→driver path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3
…store (#2786) Separation (§3.6): objects whose lifecycle.class is telemetry / event / audit now route to a dedicated 'telemetry' datasource whenever one is registered — engine getDriver() step 3, between datasourceMapping rules and the package-manifest defaultDatasource. Opt-in purely by the datasource's existence: no 'telemetry' driver ⇒ resolution is byte-for-byte what it was. Even on SQLite that is a separate file, so platform-generated growth can never again pollute the business DB. 'transient' deliberately stays on the primary: those objects are user-session data and some (better-auth's sys_device_code) are accessed outside the engine — splitting their storage would split their brain. Archiver (§3.3): audit-class objects with 'archive' declared now get the full retain → archive → delete flow instead of the P1 retain-only skip: - copies rows past archive.after to the archive datasource in bounded batches (500 × 20/sweep — a backlog drains across sweeps), as per-row idempotent upserts so an interrupted sweep re-converges - hot-deletes exactly the copied ids (bulkDelete), then the reclaim pass vacuums the hot store - archive.keep prunes the cold store itself - the cold schema is mirrored via idempotent syncSchema - no archive datasource registered ⇒ rows retained, object reported 'archive-pending' — a compliance ledger is never dropped unarchived Proof: engine-lifecycle-datasource.test.ts (routing incl. explicit-datasource precedence), Archiver unit tests (copy/delete id-exactness, keep-pruning, missing-datasource retention), and a dogfood case that provisions a real second SQL store mid-run and watches the ledger drain hot → cold. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3
…ant retention overrides (#2786) Adds the operations layer over the P1-P3 enforcement machinery, resolved from a new 'lifecycle' settings namespace (registered by ObjectQLPlugin at kernel:ready when a SettingsService is present; absent settings = declared policies apply unmodified): - enabled — runtime master switch (on top of OS_LIFECYCLE_DISABLED) - retention_overrides — per-object window overrides { object: { maxAge, expireAfter } }. TENANT-scoped: the sweep enumerates organizations, and a tenant whose override is genuinely tenant-scoped (not inherited) gets its own cutoff on its own rows, while the global pass covers everyone else INCLUDING NULL-org rows (a bare $nin would silently skip them). This is ADR-0057 §3.2 verbatim: regulated tenants set years, dev sets days — the same knob at different settings. An unparseable override keeps the declared window (never fails open into unbounded growth). - quotas / quota_defaults — per-object and per-class row ceilings - growth_alert_rows — per-sweep growth spike threshold Quota and growth breaches are OBSERVE-AND-ALERT only (report.alerts + onAlert sink, default logger warning) — governance never deletes beyond the declared policy; an operator decides. Every settings read is best-effort and snapshotted once per sweep; the tenant scan is capped (200) and skipped entirely on single-tenant kernels. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3
… lifecycle release (#2786) - ADR-0057 header + §4 rollout table now record P1–P4 as implemented in #2791; §3.3 gains the archive-safety hard rule and §3.6 the as-implemented 'telemetry' datasource routing note (transient exclusion rationale). - Changeset: minor bumps across spec / objectql / driver-sql / driver-sqlite-wasm / platform-objects / metadata-core / service-messaging / service-automation / plugin-audit, documenting the bounded-by-default behavior change and the override knobs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3
github-actions
Bot
added
protocol:data
tests
tooling
size/xl
and removed
size/xs
labels
Contributor
📓 Docs Drift CheckThis PR changes 8 package(s): 104 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
os-zhuang
changed the title
…le exports The 5 additions (LifecycleSchema / LifecycleClassSchema / Lifecycle / LifecycleClass / LIFECYCLE_DURATION_REGEX) are the intentional P1 spec surface; the api-surface gate flagged them as uncommitted snapshot drift. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3
os-zhuang
marked this pull request as ready for review
July 11, 2026 01:31
os-zhuang
pushed a commit
that referenced
this pull request
Jul 11, 2026
…gistry conflicts Both this branch and the merged ADR-0057 lifecycle work (#2791) appended a HIGH_RISK_CLASSES entry and extended the BOUND_PROOF_PATHS expectation: kept BOTH ('delegation' → position/delegatable, 'data-lifecycle' → object/lifecycle). proof-registry tests and the spec-liveness gate pass on the merged tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3
os-zhuang
added a commit
that referenced
this pull request
Jul 11, 2026
…ow-up) (#2818) The lifecycle contract shipped in #2791 with no hand-written docs or skill coverage — anyone modeling an append-only object through the docs or the objectstack-data skill would still build the unbounded-growth bug the ADR exists to prevent. Four gaps closed: - data-modeling/objects.mdx: "Data Lifecycle (Retention & Rotation)" under Enterprise Features — classes, three valid policy examples (each verified against the real LifecycleSchema parse), enforcement rules, archive safety, telemetry-datasource separation, and the governance knobs. - kernel/services.mdx: `lifecycle` row in the Standard Services table. - data-modeling/drivers.mdx: SQLite space reclamation (auto_vacuum + reclaimSpace) and physical table rotation; PG/MySQL fallback semantics. - deployment/production-readiness.mdx: go-live checklist item — review the default retention windows / tenant overrides / archive datasource before launch. - skills/objectstack-data: new rules/lifecycle.md (decision tree, correct/incorrect examples, ops table) + SKILL.md property-table row and Quick Reference link, explicitly disambiguated from lifecycle *hooks*. Docs/skills only — no runtime change, no changeset (skip-changeset). Claude-Session: https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3 Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Jul 11, 2026
os-zhuang
added a commit
that referenced
this pull request
Jul 11, 2026
…v telemetry datasource + db:clean, Studio lifecycle form (#2835) Implements #2834 ①–③ (④ PG-partition rotation design-sketched on the issue, deferred until CI has a PostgreSQL service). ① JobRunRetention / NotificationRetention and their retentionDays/retentionSweepMs options removed — the LifecycleService is the one sweeper (windows tuned via the lifecycle settings namespace). Includes a fix: the blanket 30d lifecycle retention on sys_automation_run (from #2791) could strand suspended approval runs; removed — bounding stays with the automation store's terminal-only sweep. ② objectstack dev provisions a dedicated telemetry datasource (<primary>.telemetry.db; OS_TELEMETRY_DB to opt out/override) and new `os db clean` runs the one-time VACUUM legacy files need to adopt auto_vacuum=INCREMENTAL. ③ object.form.ts exposes the lifecycle block; metadata-forms i18n bundles regenerated with curated zh-CN. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements ADR-0057 data lifecycle P1–P4 (tracking issue #2786) — platform-generated data becomes bounded by construction. Grew out of the 260 MB
dev.dbincident: every platform-generated object defaulted to immortal, and the file never shrank.P1 — contract (
spec,objectql,driver-sql)lifecycleobject property:class: record | audit | telemetry | transient | event+retention.maxAge/ttl.{field, expireAfter}/storage(rotation)/archive.{after, to, keep}/reclaim, with duration literals ('6h','14d','7y'). Parse-time invariants (ADR §3.5 enforce-or-remove): a non-recordclass with no bounding policy is rejected; policies onrecordare rejected;archive.aftermust equalretention.maxAge.ObjectQLPlugin(platform-owned, default-on;OS_LIFECYCLE_DISABLED=1orlifecycle.enabled=falseto opt out). The Reaper bulk-deletes rows pastretention.maxAge/ttlhourly under a system context — one afterDelete hook max per object per sweep, and telemetrysys_*objects are in the audit writer'sSKIP_OBJECTS, so cleanup never re-feeds the tables it drains (ADR-0052 self-audit trap).SqlDriver.reclaimSpace()— thePRAGMA incremental_vacuumhook the ADR's P0 text described but which had never actually landed.sys_activity14d+rotation,sys_audit_log90d→archive 7y,sys_metadata_audit365d→archive,sys_job_run/sys_automation_run/sys_http_delivery30d, notification pipeline (event/delivery/receipt/inbox) one 90d window,sys_device_codeexpires_at+1d.object/lifecycleand binds it to a dogfood proof (HIGH_RISK_CLASSESdata-lifecycle); new dogfood storage-growth gate (@proof: adr0057-lifecycle-bounded-growth) boots showcase, reaps a backdated runaway writer, and asserts record-class data is untouched.P2 — Rotator (
driver-sql)storage: { strategy: 'rotation', shards, unit }physically time-shards the table on SQLite (ServiceNow Table Rotation model): writes land in the current shard (<table>__r<key>), reads go through a UNION-ALL view under the base name, expiry is an O(1)DROPof shards past theshards × unitwindow. Legacy tables are adopted as the first shard on upgrade (no data loss); by-id and bulk writes fan out shard-wise; other dialects fall back to an equivalent age-based reap so the declared bound holds everywhere. (Transient TTL expiry shipped with the P1 Reaper — TTL and age reaping are one mechanism per §3.3.)P3 — separation + Archiver (
objectql)telemetryroutestelemetry/event/auditobjects to it (enginegetDriverstep 3; opt-in purely by the datasource's existence).transientdeliberately stays on the primary — some of those objects (better-auth'ssys_device_code) are accessed outside the engine.archivedeclared get retain → archive → delete — batched idempotent copies to the cold store, hot-delete of exactly the copied ids,archive.keeppruning of the archive itself. Hard rule: no archive datasource ⇒ rows retained and reportedarchive-pending; a compliance ledger is never dropped unarchived.P4 — governance (
objectql+ settings)New
lifecyclesettings namespace: runtime enable switch,retention_overrides(tenant-scoped — regulated tenants set years, dev sets days, ADR §3.2; per-tenant windows sweep each overriding tenant on its own cutoff while the global pass covers everyone else including NULL-org rows),quotas/quota_defaultsandgrowth_alert_rows— quota/growth breaches are observe-and-alert only (report.alerts+onAlertsink), never deletion beyond declared policy.Verification
Also included: the initial doc-only commit syncing the ADR header (P0 shipped in #2079), and a final commit marking §4 P1–P4 implemented + the changeset (all
minor, per the launch-window convention).Closes #2786.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BNBzMWmSECrbiEDdVzwBt3