fix(spec): ratchet the authorable key surface — the one contract no witness watched (#3855) (#3895)

For a metadata-driven platform the third-party API is WHAT AN AUTHOR MAY WRITE:
the keys inside each schema. Nothing guarded them.

Both existing witnesses watch the TypeScript surface instead. `api-surface.json`
records exported `name (kind)`, and a key inside a schema is not an export.
`api-surface-signatures.json` hashes each `defineX` factory's type — but via
`checker.typeToString()`, which prints a type REFERENCE (`z.input<typeof
ActionSchema>`) and never expands it, so member-level narrowing cannot reach the
hash. `spec-changes.json` inherits the blind spot: its added/removed arrays are a
diff of `api-surface.json`.

Verified concretely: #3883 narrowed `defineAction`'s input by three keys and the
signature snapshot did not move. And #3733 was the ACCIDENTAL version —
`dataQuality` / `cached` outlived their keys, stayed discoverable in the
reference docs, and were silently stripped on parse. ADR-0059 §5 deferred a
deeper gate "until a narrowing actually slips both". It has, twice.

`content/docs/protocol/backward-compatibility.mdx` had meanwhile been promising
third parties the exact rule nothing enforced: "removing or renaming properties
is a MAJOR change".

`authorable-surface.json` ratchets all 8588 authorable keys, derived from the
walk that already emits the JSON Schemas — one level deeper, no new
introspection. Three states, distinguishable for free because a tombstoned key
(`retiredKey()`, #3883) is `z.never()`, which Zod renders as `{ "not": {} }`:
live, [RETIRED], absent.

Three failure modes now fail the build, each verified non-vacuous by simulation
rather than assumed — the whole point being that a gate can look present and be
dead:

  1. A key vanishes with no tombstone. These schemas are not `.strict()`, so Zod
     silently strips an unknown key: clean parse, setting never takes effect. The
     error spells out the retirement protocol.
  2. A key is tombstoned with no registered migration. Audible to whoever hits
     it, but the change documentation is still empty. Requires a D2 conversion /
     D3 chain entry naming the surface — mechanizing what #3883 did by hand with
     nothing to enforce it.
  3. An addition is left uncommitted (`--check`). An unrecorded key is invisible
     to the ratchet forever after, since it can only detect the disappearance of
     something it once saw.

Failures exit before the snapshot write, so a red run never clobbers the
baseline.

Runs in `check:docs` (unconditional, required — the job whose own comment says
the gate cannot go dormant) and as an explicit step in Check Generated Artifacts,
with the paths filter widened in lockstep per that filter's own stated rule.

Value-level narrowing (an enum losing a member) stays ungated per ADR-0059 §5's
evidence gate; noise kills a ratchet faster than a gap does. What is fixed is the
over-promise: `build-api-surface.ts` advertised its signature snapshot as
answering "did the accepted authoring shape narrow?" — it does not, and believing
it did is what let this gap sit.

Author: os-zhuang

.changeset/authorable-surface-ratchet.md

@@ -0,0 +1,58 @@
+---
+"@objectstack/spec": patch
+---
+
+fix(spec): ratchet the authorable key surface — the one contract no witness watched (#3855)
+
+For a metadata-driven platform the third-party API is **what an author may
+write**: the keys inside each schema. Nothing guarded them.
+
+The two existing witnesses look at the TypeScript surface instead:
+
+- `api-surface.json` records exported `name (kind)`. A key inside a schema is
+  not an export, so removing one never moves it.
+- `api-surface-signatures.json` hashes each `defineX` factory's type — but via
+  `checker.typeToString()`, which prints a type *reference*
+  (`z.input<typeof ActionSchema>`) and never expands it structurally. Member-level
+  narrowing cannot reach the hash.
+
+`spec-changes.json` inherits the same blind spot: its `added`/`removed` arrays
+are a diff of `api-surface.json`.
+
+So #3883 removed three authorable keys with every witness green — and #3733 did
+the same **by accident**, when `dataQuality` / `cached` outlived their keys and
+were silently stripped. ADR-0059 §5 deferred a deeper gate "until a narrowing
+actually slips both". It has, twice.
+
+**New: `authorable-surface.json`**, a committed ratchet of all 8588 authorable
+keys, derived from the same walk that already emits the JSON Schemas — one level
+deeper, no new introspection. It distinguishes three states, because a tombstoned
+key (`retiredKey()`) is `z.never()`, which Zod renders as `{ "not": {} }`:
+
+| State | Meaning |
+|---|---|
+| live | a normal property — the author may write it |
+| `[RETIRED]` | present but unwritable, carrying its own upgrade prescription |
+| absent | gone from the contract with nothing left to say |
+
+Three failure modes now fail the build, each verified non-vacuous by simulation:
+
+1. **A key vanishes without a tombstone.** These schemas are not `.strict()`, so
+   Zod silently strips an unknown key: the author gets a clean parse and a
+   setting that never takes effect. The error spells out the retirement protocol.
+2. **A key is tombstoned with no registered migration.** The tombstone is audible
+   to whoever hits it, but the change documentation — `spec-changes.json`, the
+   generated upgrade guide, the `spec_changes` MCP tool, `os migrate meta` — is
+   still empty. Requires a D2 conversion / D3 chain entry naming the surface.
+3. **An addition is left uncommitted** (`--check`). An unrecorded key is
+   invisible to the ratchet forever after, since it can only detect the
+   disappearance of something it once saw.
+
+It runs in `check:docs` (unconditional, required — cannot go dormant) and as an
+explicit `check:authorable-surface` step in `Check Generated Artifacts`, with the
+paths filter widened in lockstep per that filter's own rule.
+
+Also corrects the `build-api-surface.ts` docblock, which advertised the signature
+snapshot as answering "did the accepted authoring shape narrow?" — it does not,
+and believing it did is what let this gap sit. Value-level narrowing (an enum
+losing a member) remains ungated, per ADR-0059 §5's evidence gate.

.github/workflows/ci.yml

@@ -62,6 +62,12 @@ jobs:
               - 'packages/spec/api-surface.json'
               - 'packages/spec/spec-changes.json'
               - 'docs/protocol-upgrade-guide.md'
+              # check:authorable-surface — the ratchet's input is every Zod schema
+              # in the spec, so the filter has to be that broad or the gate goes
+              # dormant on exactly the PRs that remove an authorable key (#3855).
+              - 'packages/spec/src/**'
+              - 'packages/spec/scripts/build-schemas.ts'
+              - 'packages/spec/authorable-surface.json'
               - '.github/workflows/ci.yml'
             core:
               - 'packages/**'
@@ -558,3 +564,13 @@ jobs:
 
       - name: Check the protocol upgrade guide is regenerated with the ADR-0087 registries
         run: pnpm --filter @objectstack/spec check:upgrade-guide
+
+      # The authorable KEY surface — what a metadata author may write, which for
+      # this platform is the third-party API. `api-surface.json` records exported
+      # names and `api-surface-signatures.json` hashes factory types as TypeScript
+      # PRINTS them (a reference, never structurally expanded), so neither sees a
+      # key added to or removed from a schema. #3883 removed three authorable keys
+      # with every witness green; #3733 did it by accident. ADR-0059 §5 deferred
+      # this gate until a narrowing actually slipped both — it has.
+      - name: Check the authorable key surface is recorded and nothing vanished
+        run: pnpm --filter @objectstack/spec check:authorable-surface