feat(security): split managedBy:'system' into engine-owned vs admin-writable, guard engine-owned writes (#3220)

.changeset/adr-0103-managedby-write-policy.md

@@ -0,0 +1,25 @@
+---
+"@objectstack/plugin-security": minor
+"@objectstack/objectql": minor
+"@objectstack/plugin-hono-server": minor
+"@objectstack/spec": patch
+"@objectstack/platform-objects": patch
+"@objectstack/plugin-approvals": patch
+"@objectstack/service-automation": patch
+"@objectstack/service-messaging": patch
+"@objectstack/plugin-sharing": patch
+"@objectstack/rest": patch
+---
+
+**Split the overloaded `managedBy: 'system'` bucket into engine-owned vs. admin-writable, and enforce engine-owned writes (ADR-0103, #3220).** The `system` bucket conflated two incompatible write policies: rows a platform service owns end to end (never user-written), and platform-defined schema whose rows are legitimately admin/user-writable. It carried the same all-false affordance row as `better-auth`/`append-only` but, unlike `better-auth`, had no engine enforcement — a wildcard admin could raw-write these rows through the generic data API (ADR-0049 gap).
+
+Rather than add a new `managedBy` enum value (which would fall through to fully-editable `platform` defaults on already-deployed Console clients), the write policy is now the **resolved affordance** (`resolveCrudAffordances` = bucket default + `userActions`), and *engine-owned* is defined as a `system`/`append-only` object that grants no write:
+
+- **Writable set declares `userActions`** — the RBAC link tables (`sys_user_position`, `sys_user_permission_set`, `sys_position_permission_set`), `sys_user_preference`, `sys_approval_delegation`, and the messaging config grids (`sys_notification_preference` / `…_subscription` / `…_template`) now declare `userActions: { create, edit, delete: true }`. The affordance is a declaration only — the `DelegatedAdminGate` / RLS / permission sets remain the authz.
+- **Engine-owned objects locked to reads** — `apiMethods: ['get','list']` added where absent (jobs, notifications, approval request/approver/token/action, `sys_record_share`, `sys_automation_run`, mail/settings/secret audit, the messaging delivery pipeline). `sys_secret` is explicitly read-locked (an empty `apiMethods` array fails open).
+- **`sys_import_job`** stays engine-owned: the REST import route now writes its job rows `isSystem`-elevated (attribution preserved via the explicit `created_by` stamp) and the object is locked to `['get','list']`.
+- **New engine write guard** (`assertEngineOwnedWriteAllowed`, plugin-security) fail-closed rejects user-context generic writes to engine-owned `system`/`append-only` objects, keyed off the resolved affordance; `isSystem` and context-less engine/service writes bypass by construction. Wired into the security middleware alongside the other data-layer gates.
+- **`reconcileManagedApiMethods`** (objectql registry) now runs for **every** managed bucket, not just `better-auth`: any advertised write verb an object's resolved affordances forbid is stripped at registration with a warning (the drift backstop, ADR-0049).
+- **`/me/permissions` clamp** (plugin-hono-server) now clamps `system`/`append-only` as well as `better-auth`, so the client hint reflects `permission ∩ guard`.
+
+**Potentially breaking:** a downstream/third-party `system` object that advertised generic write verbs relying on today's fail-open behaviour will have those verbs stripped (with a warning) and user-context generic writes to it rejected. Declare `userActions` opening the verbs the object legitimately takes from a user context. `better-auth` keeps plugin-auth's identity write guard unchanged; the row-level `managed_by` provenance vocabulary (ADR-0066) is a different axis and is untouched.

docs/adr/0103-managedby-write-policy-and-engine-write-guard.md

@@ -0,0 +1,146 @@
+# ADR-0103: `managedBy` write policy — split the overloaded `system` bucket, enforce engine-owned writes
+
+- **Status**: Accepted
+- **Date**: 2026-07-19
+- **Issue**: #3220 (root cause surfaced by the #1591 / #3213 better-auth guard work; safe slice shipped in #3222)
+- **Relates to**: ADR-0049 (no unenforced security properties), ADR-0092 (identity
+  write guard mechanism), ADR-0066 (system-row provenance gate)
+
+## Context
+
+`managedBy` is the object-lifecycle bucket UI clients read to render CRUD
+affordances, and the security layer reads to enforce matching write defaults.
+Its `system` value had come to mean two **incompatible** things:
+
+1. **Engine-owned** — runtime rows a platform service owns end to end and no user
+   ever writes directly: `sys_automation_run`, `sys_job` / `sys_job_queue`,
+   `sys_notification`, the approval engine's request/approver/token rows, the
+   sharing engine's `sys_record_share`, `sys_setting` / `sys_secret` /
+   `sys_setting_audit`, metadata history/commit, the messaging delivery pipeline
+   (`sys_notification_delivery`, `…_receipt`, `http_delivery`). Every write goes
+   through `isSystem: true` / a `SYSTEM_CTX` service engine, or a context-less
+   engine call (the messaging service's raw-engine writes, the metadata-protocol
+   repository's transaction context — neither carries a `userId`).
+2. **Platform-schema, admin/user-writable data** — objects whose *schema* is
+   platform-defined but whose *rows* are legitimately written from a user
+   context by design:
+   - the RBAC link tables `sys_user_position`, `sys_user_permission_set`,
+     `sys_position_permission_set` — governed at write time by the
+     `DelegatedAdminGate` (ADR-0090 D12), which depends on the write carrying the
+     real caller (e.g. `suggested-audience-bindings.ts` inserts a direct grant
+     with `context: callerCtx`, deliberately **not** `isSystem`);
+   - `sys_user_preference` — a user authors their own preferences (RLS self-grant);
+   - `sys_approval_delegation` — user-facing "who covers for me" config;
+   - the messaging config grids surfaced in Setup — `sys_notification_preference`
+     (a user mutes their own topics), `sys_notification_subscription`,
+     `sys_notification_template` (admin-authored).
+
+All three restricted buckets (`system`, `append-only`, `better-auth`) collapsed
+to one identical all-false affordance row, but only `better-auth` had real
+enforcement: an engine write guard (ADR-0092) plus registration-time
+`apiMethods` reconciliation. `system` had **neither**. A blast-radius audit
+(#3220) disproved the original "just guard `system` wholesale" proposal — a
+blanket guard would break delegated administration and self-service — and
+narrowed the safe, mechanical slice to two latent holes (`sys_presence`,
+`sys_metadata`), which shipped as #3222. This ADR is the root-cause fix: make the
+write policy **expressible and enforced** without breaking the writable set.
+
+> **Not to be confused with** the row-level `managed_by` *provenance* column
+> (`'platform' | 'package' | 'admin'`) on `sys_sharing_rule` / `sys_position` /
+> `sys_capability`, guarded by `assertSystemRowWriteGate` (ADR-0066). That is a
+> per-row provenance axis; this ADR is about the per-object schema bucket. The
+> shared word is a coincidence.
+
+## Decision
+
+### D1 — The bucket is a default; the *resolved affordance* is the policy
+
+`managedBy` supplies only the **default** CRUD affordance row. The authoritative,
+enforced write policy for an object is `resolveCrudAffordances(schema)` — the
+bucket default with `userActions` overrides applied. We do **not** add a new
+enum value. Instead:
+
+- **Engine-owned** ≔ an object in the `system` or `append-only` bucket whose
+  resolved affordances grant **no** write (`create`/`edit`/`delete` all false) —
+  i.e. the bucket default, un-opened.
+- **Admin/user-writable** ≔ a `system`/`append-only` object that declares
+  `userActions` opening the writes it legitimately takes. `userActions` is an
+  affordance declaration, **not** an authorization: the real authz for these
+  rows remains the `DelegatedAdminGate` / RLS self-grant / permission sets. The
+  declaration exists so the affordance matrix, the `apiMethods` reconciliation,
+  and the engine guard below all agree on what the object permits.
+
+Why not a new enum value: a new `managedBy` string falls through to the
+fully-editable `platform` default on every already-deployed Console client
+(the UI keeps a hand-maintained closed union and an open-ended type), silently
+re-granting New/Edit/Delete on an engine table until each client is updated. The
+`userActions` approach changes no wire vocabulary, and every UI surface already
+honours `userActions`. Splitting the enum later remains possible as a pure
+rename on top of the now-correct affordances.
+
+### D2 — Engine write guard for the engine-owned set
+
+A new `system-write-guard` (plugin-security), modelled on the ADR-0092 identity
+guard, fails-closed on any **user-context** write to a `system`/`append-only`
+object whose resolved affordances do not grant the corresponding verb. Verb →
+affordance: `insert → create`, `update`/`upsert` → `edit`,
+`delete`/`purge`/`restore` → `delete`, `transfer → edit` (aligned with the
+`DelegatedAdminGate` governed-operation set). A write is **user-context** when
+`context.userId` is set and `context.isSystem !== true`; `isSystem` and
+context-less engine/service writes bypass by construction — exactly the calls the
+legitimate engine writers make. It is wired into the security middleware
+alongside the other unconditional data-layer gates (package-managed, system-row,
+audience-anchor, delegated-admin), after the `isSystem` short-circuit. Denials
+raise `PermissionDeniedError` (HTTP 403).
+
+The writable set passes the guard because its `userActions` grant the verb; the
+`DelegatedAdminGate` / RLS then adjudicate the principal, unchanged.
+
+### D3 — Generalize `apiMethods` reconciliation beyond `better-auth`
+
+`reconcileManagedApiMethods` (objectql registry) previously ran only for
+`better-auth`. It now runs for **every** managed bucket, still purely
+affordance-driven: any generic write verb an object advertises in
+`enable.apiMethods` that its resolved affordances do not grant is stripped at
+registration with a warning, so the HTTP exposure gate answers a clean 405
+instead of admitting a write that the engine guard (or permission layer) would
+later 403. This makes the contradiction impossible to ship and is the drift
+backstop for future `system` objects.
+
+### D4 — Generalize the `/me/permissions` clamp
+
+The hono-server `clampManagedObjectWrites` previously clamped only `better-auth`
+objects' advertised affordances. It now clamps `system`/`append-only` too, so the
+permissions payload the Console consumes reflects the true (guarded) write
+policy.
+
+### D5 — `sys_import_job` writes become attributed system writes
+
+The REST import route created/updated `sys_import_job` under the caller's
+context. Those rows are engine-owned (the import worker owns the lifecycle), so
+the route now writes them `isSystem`-elevated while preserving `created_by`
+attribution, and the object is locked to `apiMethods: ['get', 'list']`. This
+keeps the object engine-owned rather than carving a `userActions` exception for
+a row users never hand-edit.
+
+## Alternatives considered
+
+- **New `managedBy` enum value** (e.g. `engine`) — rejected: silent
+  fully-editable fallthrough on deployed clients, plus enum/matrix churn and
+  ~30 re-bucketed declarations across three UI mirrors and ten locale bundles,
+  for a distinction `userActions` already expresses. Revisitable later as a
+  rename.
+- **`writeVia` capability** for third-party-managed objects — deferred (#1878):
+  no such object exists today; speculative until a concrete need arises
+  (explicitly out of scope per #3220).
+
+## Consequences
+
+- `system`/`append-only` objects reject user-context generic writes unless
+  `userActions` opens them — defence-in-depth for non-REST callers, matching the
+  HTTP-layer `apiMethods` lock several already carry.
+- Third-party or downstream `system` objects that advertised write verbs relying
+  on today's fail-open behaviour will have those verbs stripped with a warning —
+  the intended ADR-0049 correction, called out in the release note.
+- The taxonomy is now honest: a reader can tell engine-owned from writable by the
+  resolved affordances, and the guard enforces it.

packages/objectql/src/registry.test.ts

@@ -723,9 +723,10 @@ describe('applySystemFields', () => {
 });
 
 // ==========================================
-// reconcileManagedApiMethods — #1591 / ADR-0092
-// Registration-time consistency: a better-auth-managed object may not
-// advertise generic write verbs it doesn't grant.
+// reconcileManagedApiMethods — #1591 / ADR-0092 / ADR-0103
+// Registration-time consistency: ANY managed object may not advertise generic
+// write verbs its resolved affordances don't grant. Scope generalized from
+// better-auth to every managed bucket in ADR-0103.
 // ==========================================
 describe('reconcileManagedApiMethods', () => {
     const managed = (extra: any = {}): any => ({
@@ -773,7 +774,7 @@ describe('reconcileManagedApiMethods', () => {
         expect(out.enable.apiMethods).toEqual(['get']);
     });
 
-    it('leaves non-better-auth objects untouched (platform bucket keeps full CRUD)', () => {
+    it('leaves platform-bucket objects untouched (grant full CRUD → nothing stripped)', () => {
         const warn = vi.fn();
         const platform: any = {
             name: 'sys_business_unit',
@@ -786,6 +787,58 @@ describe('reconcileManagedApiMethods', () => {
         expect(warn).not.toHaveBeenCalled();
     });
 
+    it('leaves unmanaged objects (no managedBy) untouched', () => {
+        const warn = vi.fn();
+        const plain: any = {
+            name: 'crm_lead',
+            enable: { apiEnabled: true, apiMethods: ['get', 'list', 'create', 'update', 'delete'] },
+        };
+        const out = reconcileManagedApiMethods(plain, { warn });
+        expect(out).toBe(plain);
+        expect(warn).not.toHaveBeenCalled();
+    });
+
+    // ADR-0103 — the scope generalization: engine-owned system/append-only
+    // objects derive to reads, while the writable set keeps its verbs.
+    it('strips write verbs from a system-bucket object with no write affordances (engine-owned)', () => {
+        const warn = vi.fn();
+        const engineOwned: any = {
+            name: 'sys_automation_run',
+            managedBy: 'system',
+            enable: { apiEnabled: true, apiMethods: ['get', 'list', 'create', 'update', 'delete'] },
+        };
+        const out = reconcileManagedApiMethods(engineOwned, { warn });
+        expect(out.enable.apiMethods).toEqual(['get', 'list']);
+        expect(warn).toHaveBeenCalledTimes(1);
+        expect(warn.mock.calls[0][0]).toContain("managedBy:'system'");
+    });
+
+    it('strips write verbs from an append-only object with no write affordances', () => {
+        const warn = vi.fn();
+        const appendOnly: any = {
+            name: 'sys_email',
+            managedBy: 'append-only',
+            enable: { apiEnabled: true, apiMethods: ['get', 'list', 'update'] },
+        };
+        const out = reconcileManagedApiMethods(appendOnly, { warn });
+        expect(out.enable.apiMethods).toEqual(['get', 'list']);
+        expect(warn).toHaveBeenCalledTimes(1);
+    });
+
+    it('keeps CRUD on a system-bucket object whose userActions open the writes (writable set)', () => {
+        const warn = vi.fn();
+        const writable: any = {
+            name: 'sys_user_position',
+            managedBy: 'system',
+            userActions: { create: true, edit: true, delete: true },
+            enable: { apiEnabled: true, apiMethods: ['get', 'list', 'create', 'update', 'delete'] },
+        };
+        const out = reconcileManagedApiMethods(writable, { warn });
+        expect(out).toBe(writable); // nothing stripped
+        expect(out.enable.apiMethods).toEqual(['get', 'list', 'create', 'update', 'delete']);
+        expect(warn).not.toHaveBeenCalled();
+    });
+
     it('applies on registerObject (the contradiction cannot be stored)', () => {
         const reg = new SchemaRegistry({ multiTenant: false });
         reg.registerObject(managed(), 'sys', 'sys', 'own');