feat(spec,objectql,driver-sql)!: ADR-0113 accepted — required is a write contract, storage.notNull is the column constraint (#4031)

The three adjudications on ADR-0113's open questions, and P0.

Q1 storage.notNull spelling: the write contract and the column constraint form a genuine 2x2 (required alone = the criteria_json posture; storage.notNull alone = the engine-populated column) — a single enum cannot express the storage-only cell. Q2 uniform semantics by EXPLICITIZATION: the field-required-notnull-explicit conversion stamps storage.notNull onto every pre-17 required field (writing down what the old text already meant, migration-chain-only — the loader never infers semantics from the physical column; the draft's load-time inference is rejected as self-contradictory). Q3 generalized: the non-regression invariant — a write may not take a record from compliant to violating; a pre-existing violation does not block writes that leave it in place — with required and requiredWhen as corollaries.

Implementation: FieldSchema.storage.{notNull} + parse-seam exclusivity vs requiredWhen (superRefine); record-validator rejects a PATCH nulling a required field (the update path previously skipped the required check entirely — a real hole closed); rule-validator applies the invariant to requiredWhen (the #3929 legacy-row lockout cured, its pinning test deliberately flipped); sql-driver createColumn + schema-drift read storage.notNull (a column stricter than its declaration is needs_confirm, silent when write-gated; dev auto-reconcile no longer strips a stray NOT NULL — supersedes the #2178 self-heal); conversion + chain step replayed by the composability gate; ADR-0113 -> Accepted; v17 notes + upgrade checklist; field ledger; major changesets.

spec 6903 / objectql 1179 / driver-sql 485 / wasm 126 green; all gates green.

Author: os-zhuang

.changeset/adr-0113-required-write-contract.md

@@ -0,0 +1,40 @@
+---
+"@objectstack/spec": major
+"@objectstack/objectql": major
+"@objectstack/driver-sql": major
+---
+
+feat!: ADR-0113 — `required` is a write contract; the column constraint becomes the explicit `storage.notNull`
+
+`field.required` bound three meanings to one knob (write check, `NOT NULL` DDL,
+drift expectation), so tightening any invariant on a deployed object was a
+destructive migration blocked by the very legacy nulls that motivated it — the
+reason `criteria_json`'s mandatory-in-substance contract lived in three
+imperative guards instead of one declaration.
+
+Split, with the **non-regression invariant** as the unifying rule — *a write
+may not take a record from compliant to violating; a pre-existing violation
+does not block writes that leave it in place*:
+
+- `required: true` = the write contract, uniformly on new and deployed objects:
+  insert must provide; **an update PATCHing `null` into a required field is now
+  rejected** (it silently passed before); omitted fields never block, so legacy
+  null rows rest. The column stays nullable.
+- `storage: { notNull: true }` = the explicit physical constraint, owning the
+  DDL (`sql-driver` `createColumn`) and the destructive drift ceremony.
+  Orthogonal to `required` — all four combinations are legitimate, including
+  the engine-populated column (`storage.notNull` without `required`).
+- `requiredWhen` inherits the same invariant: flipping the condition true
+  without providing the field is rejected (the write *creates* the violation);
+  a row violating since before the rule tightened no longer locks out
+  unrelated edits (#3929's objection, cured). `storage.notNull` ×
+  `requiredWhen` rejects at parse (`FieldSchema.superRefine`).
+- **Pre-17 sources keep their exact meaning** via the migration-chain-only
+  `field-required-notnull-explicit` conversion: `os migrate meta` stamps
+  `storage.notNull` onto every previously-required field — writing down what
+  the old text already meant. The loader never infers semantics from the
+  physical column.
+- Drift compares nullability against `storage.notNull`; a column stricter than
+  its declaration is `needs_confirm` (never auto-applied — dev auto-reconcile
+  no longer silently strips a stray `NOT NULL`), and silent when the field is
+  write-gated by `required`.

content/docs/references/data/field.mdx

@@ -76,7 +76,8 @@ const result = Address.parse(data);
 | **type** | `Enum<'text' \| 'textarea' \| 'email' \| 'url' \| 'phone' \| 'password' \| 'secret' \| 'markdown' \| 'html' \| 'richtext' \| 'number' \| 'currency' \| 'percent' \| 'date' \| 'datetime' \| 'time' \| 'boolean' \| 'toggle' \| 'select' \| 'multiselect' \| 'radio' \| 'checkboxes' \| 'lookup' \| 'master_detail' \| 'tree' \| 'user' \| 'image' \| 'file' \| 'avatar' \| 'video' \| 'audio' \| 'formula' \| 'summary' \| 'autonumber' \| 'composite' \| 'repeater' \| 'record' \| 'location' \| 'address' \| 'code' \| 'json' \| 'color' \| 'rating' \| 'slider' \| 'signature' \| 'qrcode' \| 'progress' \| 'tags' \| 'vector'>` | ✅ | Field Data Type |
 | **description** | `string` | optional | Tooltip/Help text |
 | **format** | `string` | optional | Format string (e.g. email, phone) |
-| **required** | `boolean` | optional | Is required |
+| **required** | `boolean` | optional | Write-time contract (ADR-0113): an insert must provide a non-null value, and an update may not null it out. NOT a column constraint — the physical NOT NULL is a separate explicit opt-in (`storage.notNull`), so tightening this on a deployed object is safe: existing null rows stay readable, and editable as long as the write does not touch this field. |
+| **storage** | `{ notNull?: boolean }` | optional | Physical storage constraints (ADR-0113). Owns the DDL the write contract deliberately does not imply. Absent = no storage-level constraint requested. |
 | **searchable** | `boolean` | optional | Is searchable |
 | **multiple** | `boolean` | optional | Allow multiple values (Stores as Array/JSON). Applicable for select, lookup, file, image. |
 | **unique** | `boolean \| 'global'` | optional | Unique constraint. true = unique within the tenant (composite with the tenant column on tenant-scoped objects); 'global' = unique platform-wide across all tenants |

content/docs/releases/v17.mdx

@@ -395,6 +395,39 @@ empty shape must state its predicate, or the object should use `sharingModel`.
   tombstoned with a fix-it prescription; `os migrate meta` deletes the key
   mechanically, and policy outcomes are identical with or without it.
 
+### `required` is a write contract; the column constraint is `storage.notNull` (ADR-0113)
+
+`field.required` used to mean three things through one knob: the write check,
+the physical `NOT NULL`, and the drift expectation. Bound together, tightening
+any invariant on a deployed object was a destructive migration blocked by the
+very legacy nulls that motivated it — so real invariants ended up as imperative
+guards (three of them for `sys_sharing_rule.criteria_json`) or didn't happen.
+Split in v17:
+
+- **`required: true`** is the write-time contract, uniformly: an insert must
+  provide a non-null value, an update may not null it out — a PATCH clearing a
+  required field is now rejected (it silently passed before) — and legacy null
+  rows rest: a write that doesn't touch the field never blocks. The column
+  stays nullable.
+- **`storage: { notNull: true }`** is the explicit physical constraint. It owns
+  the `NOT NULL` DDL and the destructive-migration ceremony (backfill first).
+  Declaring it at field creation is free; declaring it over existing nulls is
+  loud.
+- **`requiredWhen` inherits the same non-regression rule**: a write that flips
+  the condition true without providing the field is rejected (it *creates* the
+  violation), while a row violating since before the rule tightened no longer
+  locks out unrelated edits. `storage.notNull` × `requiredWhen` is rejected at
+  parse — a conditional contract cannot be an unconditional constraint.
+- **Pre-17 sources keep their exact meaning**: `os migrate meta` stamps
+  `storage: { notNull: true }` onto every previously-required field
+  (`field-required-notnull-explicit`) — under the old semantics that column
+  *was* created `NOT NULL`, so the rewrite writes down what the text already
+  meant. Migration-chain-only: the loader never guesses.
+- **Drift**: nullability now compares against `storage.notNull`. A column
+  stricter than its declaration is `needs_confirm` (ratify or relax — dev
+  auto-reconcile no longer silently strips a stray `NOT NULL`), and silent when
+  the field is `required` (the write gate makes the constraint unreachable).
+
 ### Membership grade is not a capability channel (ADR-0108, #3723)
 
 `sys_member.role` answers "what is your standing in this organization", not
@@ -1020,6 +1053,12 @@ objectui commits on top of the pin 16.1.0 shipped.
   metadata (`os migrate meta` does it; none ever had an effect). To gate a tool,
   gate the underlying action (`action.requiredPermissions`); to withdraw one,
   remove it from the skills/agents that reference it.
+- **Required fields:** run `os migrate meta` — it stamps
+  `storage: { notNull: true }` onto every pre-17 `required: true` field so your
+  columns keep their exact constraints. New fields: `required: true` alone
+  write-gates with a nullable column; add `storage.notNull` when you want the
+  DDL. Audit any client that PATCHes `null` into required fields — that write
+  is now rejected.
 - **Membership:** move business capability off `sys_member.role` and onto
   positions (`sys_user_position`).
 - **SDK callers:** remove calls to `client.permissions.*`, `client.realtime.*`,

docs/adr/0113-required-write-contract-vs-column-constraint.md

@@ -1,6 +1,6 @@
 # ADR-0113: `required` is a write-time contract — the column constraint becomes its own, explicitly-authored axis
 
-**Status**: Proposed (2026-07-30) — **awaiting adjudication**. Drafted from the #3896 close-out; no code in this ADR has been implemented.
+**Status**: Accepted (2026-07-30) — Q1/Q2/Q3 adjudicated same day (uniform semantics; `storage.notNull` spelling; the non-regression invariant unifies `required` and `requiredWhen`). **P0 implemented**: spec knob + parse-seam exclusivity, record-validator null-out rejection, rule-validator non-regression, driver DDL + drift rewired, `field-required-notnull-explicit` conversion. P1 (Console required-marker from the write contract) and P2 (criteria_json flip) follow.
 **Deciders**: ObjectStack Protocol Architects
 **Builds on**: [ADR-0049](./0049-no-unenforced-security-properties.md) (enforce-or-remove), [ADR-0078](./0078-no-silently-inert-metadata.md) (no silently inert metadata), the #3896/#3929 sharing-criteria case study, `driver-sql/schema-drift.ts` (the drift classifier whose `destructive` class this ADR exists to route around)
 **Consumers**: `@objectstack/spec` (`FieldSchema`), `@objectstack/objectql` (`validation/record-validator.ts`), `@objectstack/driver-sql` (`sql-driver.ts` column DDL, `schema-drift.ts`), `@objectstack/lint`, objectui form renderers (the required marker)
@@ -78,13 +78,32 @@ polices in the other direction.
 
 ## Decision — proposed rulings
 
-### D1 — `required` means the write contract
-
-`required: true` asserts: **an insert must provide a non-null value, and an
-update may not null it out.** Existing rows are untouched — a legacy null row
-remains readable and editable so long as the write does not touch the required
-field. This is exactly the semantics #3929 hand-built for `criteria_json`,
-promoted from three guards to the platform's one word.
+### D1 — the non-regression invariant (unifies `required` and `requiredWhen`)
+
+> **A write may not take a record from compliant to violating; a pre-existing
+> violation does not block writes that leave it in place.**
+> Operationally: *reject iff the merged post-state violates AND the pre-state
+> complied.* (On insert the pre-state complies vacuously.)
+
+`required` is the degenerate always-true-condition case: an insert must
+provide a non-null value; an update may not null it out; a legacy null row
+remains readable and editable so long as the write does not touch the field.
+This is exactly the semantics #3929 hand-built for `criteria_json`, promoted
+from three guards to one platform word.
+
+`requiredWhen` inherits the same invariant with the condition generalized —
+which also catches the case the old merged-record check missed in the OTHER
+direction and over-blocked in this one: an update that flips the condition
+true without providing the field **creates** the violation and is rejected,
+while a row violating since before the rule tightened no longer locks out
+unrelated edits (the #3929 objection, cured rather than worked around).
+
+One deliberate over-approximation, chosen to keep the hot write path free of
+prior-state reads for the unconditional knob: an explicit `null` written onto
+an *already-null* legacy required field is rejected even though the pre-state
+already violated. That write was a no-op carrying a false claim; rejecting it
+costs nothing and needs no prior fetch. (`requiredWhen` evaluates against the
+prior record it already fetches, so it applies the exact rule.)
 
 ### D2 — the column constraint becomes explicit
 
@@ -118,21 +137,39 @@ contract), not from the column. `criteria_json` gets its asterisk back, and
 client-side validation aligns with what the server will actually reject —
 removing the last reason for the objectui#2962-style mirror hints.
 
-### D5 — back-compat is the deliberate asymmetry
-
-For existing metadata, `required: true` today implies both meanings, so the
-migration must not silently drop the column constraint fields already have:
-
-- Fields whose column is **already `NOT NULL`** (created as required): loader
-  treats them as `required + storage.notNull` — nothing changes, no DDL, no
-  drift.
-- Fields declared required whose column is nullable: today that is
-  un-declarable without destructive drift; under this ADR it becomes the D1
-  posture. This set is currently EMPTY by construction (nobody could declare
-  it), so no deployed tenant changes behaviour.
-- New fields: `required: true` alone creates a **nullable** column with a
-  write-contract gate. Authors who want the constraint say so (Q2 disputes
-  this default).
+### D5 — back-compat by explicitization, never by inference (Q2 resolved: uniform)
+
+`required: true` has exactly ONE meaning at any point in time — the write
+contract — and the transition is carried by rewriting old sources, not by the
+loader guessing:
+
+- **The `field-required-notnull-explicit` conversion** (D2 table, protocol-17
+  step) stamps `storage: { notNull: true }` onto every `required: true` field
+  of a pre-17 source. Under the old semantics that column WAS created
+  `NOT NULL`, so the rewrite writes down what the text already meant — a pure
+  semantic explicitization, lossless by construction. It is
+  **migration-chain-only** (retired from the load path): this is a default
+  flip, not a rename, and a loader that auto-applied it would stamp the
+  constraint back onto 17-authored sources that deliberately omit it.
+- The earlier draft's "loader treats an already-NOT NULL column as
+  `storage.notNull`" is **rejected as self-contradictory** — inferring
+  semantics from the physical column at load is itself the history-dependent
+  behaviour this ADR exists to eliminate.
+- New fields: `required: true` alone creates a **nullable** column with the
+  write-contract gate — uniformly, on new and old objects alike. The
+  history-dependent alternative (new objects still get `NOT NULL`) would make
+  correct authoring depend on a fact absent from the author's context — when
+  the object was first deployed — which an AI author (ADR-0033) cannot know
+  from source. Authors who want the constraint say so; at creation time the
+  ceremony is free (no rows, no backfill).
+- **Drift direction** (refined during implementation): a column *stricter*
+  than its declaration is reported `needs_confirm` (ratify with
+  `storage.notNull` or deliberately relax; never auto-applied) — **except**
+  when the field is `required: true`, where it is *silent*: that is every
+  pre-17 source after a runtime upgrade, the write gate makes the column
+  constraint unreachable, and nagging every legacy required field would bury
+  real drift. A declaration stricter than the column stays the destructive
+  ceremony it always was.
 
 ### D6 — `criteria_json` is the first consumer
 
@@ -145,29 +182,38 @@ pointer as evidence.
 
 ## Rollout (proposed)
 
-- **P0** (`@objectstack/spec` + `objectql`): the storage property + D1
-  validator semantics + D3 drift change. Additive; no tenant behaviour change.
+- **P0 — DONE** (this ADR's landing PR): the storage property + parse-seam
+  exclusivity; record-validator null-out rejection; rule-validator
+  non-regression; driver DDL + drift rewired; the explicitization conversion
+  + chain step.
 - **P1** (objectui): D4 marker + client validation from the write contract.
 - **P2**: D6 criteria_json flip + guard consolidation; sweep for other
   "mandatory in substance, optional in metadata" fields (the empty-state
   registry's `closed` entries are the seed list).
 
-Not v17-blocking: additive surface, no breaking change. Target: early 17.x.
-
-## Open questions for adjudication
-
-- **Q1 — spelling.** `storage: { notNull: true }` (proposed: room for future
-  storage knobs — collation, computed defaults) vs a flat
-  `requiredEnforcement: 'write' | 'column'` (one knob, no nesting, but closes
-  the namespace).
-- **Q2 — the new-field default.** This draft says `required: true` alone
-  creates a nullable column (uniform semantics; the constraint is opt-in).
-  The alternative — new objects still get `NOT NULL`, deployed objects don't —
-  keeps today's clean-database behaviour but makes the same declaration mean
-  different DDL depending on when it was authored, which is the kind of
-  history-dependent semantics ADR-0087 exists to eliminate.
-- **Q3 — `requiredWhen` interplay.** The conditional variant evaluates
-  server-side against the merged record on update (#3929 rejected it for
-  criteria_json because it blocks legacy-row edits). Should `requiredWhen`
-  adopt D1's insert-vs-update asymmetry too? Out of scope here unless the
-  deciders want it folded in.
+Lands inside the unreleased 17.0.0 train (the DDL-emission change and the two
+validator tightenings ride the major).
+
+## Resolved questions (adjudicated 2026-07-30)
+
+- **Q1 — spelling: `storage: { notNull: true }`.** The decisive argument is
+  orthogonality: the write contract and the column constraint form a genuine
+  2×2, and every cell exists in the platform's own system objects — `required`
+  alone is the criteria_json posture, `storage.notNull` alone is the
+  engine-populated column (audit fields, the tenant column). A single
+  `requiredEnforcement` enum cannot express the storage-only cell at all, and
+  hands the author a wrong option on every required field. `notNull` is also
+  vocabulary a model already knows from the SQL corpus — no invented term to
+  hallucinate values for. The namespace earns future entries one *enforced*
+  knob at a time (ADR-0078); nothing is pre-reserved.
+  **Rider:** `storage.notNull` × `requiredWhen` is rejected at the parse seam
+  (`FieldSchema.superRefine`) — when the condition is false the contract
+  permits null but the column would refuse it; two gates that cannot both be
+  honest are a contradiction the author must resolve.
+- **Q2 — uniform semantics** (folded into D5 above). Semantics conditioned on
+  deployment history are unreasonable-about by construction for an author —
+  human or model — who sees only the source.
+- **Q3 — folded in and generalized** (D1 above): rather than `requiredWhen`
+  "adopting" `required`'s asymmetry, both are corollaries of the one
+  non-regression invariant. The knobs stay one family with one temporal
+  semantics; no divergence window ever exists.