Skip to content

Commit 19365b7

Browse files
authored
feat(spec): hold the #4001 strictness ledger to the code it describes (#4232)
The ledger is the campaign's map — which z.object sites are authorable (the .strict() ratchet target), which are wire, which are deliberately open. Every step reads it to pick the next move, and nothing kept it honest. It went stale twice in one week: a classification verification disproved (hook.zod.ts carried a blanket `authorable (p)` covering a runtime shape, #4207), and a "next step" that had shipped months earlier (the unknown-key warning layer, #4218), which sent the next reader off to rebuild something already in production. A map that drifts is worse than no map, because it gets followed. CHECKS. Site counts (the ledger states its own method — "z.object( occurrences per file" — so every number is verifiable); coverage (every *.zod.ts with sites in a triaged directory needs a row); section totals; and that a row claiming "strict as of" names a file that really contains .strict(). The Class column is deliberately NOT checked. Authorable vs wire vs open is a judgement about who writes the input, and this campaign's rule is verify-before-tightening — encoding it would automate exactly the call that must stay human. The gate protects the arithmetic and coverage so that judgement is always made against current code. Zero-site files are skipped: five of the six the first run flagged hold no z.object at all (enum/token modules). Not a hole — the day one grows its first z.object( it becomes undeclared and the gate says so. FIRST RUN: 11 DRIFTS, in a file being actively edited by the campaign that owns it. Six counts had moved (ui/app.zod.ts 11 → 18, ui/touch.zod.ts 4 → 7, ui/action.zod.ts 8 → 9, ui/view.zod.ts 51 → 50, ui/responsive.zod.ts 6 → 4, automation/flow.zod.ts 12 → 11), two section totals no longer summed, and automation/io-node-config.zod.ts had no row. The app.zod.ts gap was self-inflicted: the app step (#4165) added seven schemas and updated the row's prose without touching its count. Then it worked for real before merging: main landed automation/builtin-node-config.zod.ts (#4045/#4228, eight sites), and merging main turned the gate red on a branch whose own diff touches no schema. Now classified. Every pre-existing count survived that merge unchanged, so the failure was exactly as narrow as it should have been. Verified the gate bites rather than just passes: a moved count fails (plus its section total), a deleted row resurfaces as an undeclared file, and stripping .strict() from approval.zod.ts fails its "strict as of" claim. Refs #4001
1 parent 55bbefc commit 19365b7

5 files changed

Lines changed: 311 additions & 13 deletions

File tree

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
"@objectstack/spec": patch
3+
---
4+
5+
Add a gate that holds the #4001 strictness ledger to the code it describes.
6+
7+
`docs/audits/2026-07-unknown-key-strictness-ledger.md` is the campaign's map:
8+
which `z.object` sites are authorable (the `.strict()` ratchet target), which are
9+
wire, which are deliberately open. Every step reads it to pick the next move, and
10+
nothing kept it honest. It went stale twice in one week — a classification that
11+
verification disproved (`hook.zod.ts`, #4207) and a "next step" that had shipped
12+
months earlier (the warning layer, #4218). A map that drifts is worse than no
13+
map, because it gets followed.
14+
15+
`check:strictness-ledger` enforces the claims in it that are mechanically
16+
checkable:
17+
18+
- **Site counts** — the ledger states its own method (`z.object(` occurrences per
19+
file), so every number is verifiable. A stale count means schemas were added or
20+
removed under a `Class` verdict nobody re-examined.
21+
- **Coverage** — every `*.zod.ts` with sites in a triaged directory must have a
22+
row. Zero-site files (pure enum/token modules) are skipped, and become
23+
reportable the day they grow their first `z.object(`.
24+
- **Section totals**, and that a row claiming "strict as of" names a file that
25+
really contains `.strict()`.
26+
27+
The `Class` column is deliberately not checked — authorable vs wire vs open is a
28+
human judgement, and the campaign's rule is verify-before-tightening. The gate
29+
protects the arithmetic and coverage so that judgement is made against current
30+
code.
31+
32+
First run found 11 drifts, including six moved counts (`ui/app.zod.ts` had gone
33+
11 → 18), two unbalanced section totals, and one genuinely unclassified file
34+
(`automation/io-node-config.zod.ts`). All corrected here.

.github/workflows/spec-liveness-check.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,9 @@ on:
2121
# The variant/doc gate reads hand-written pages, so editing one can break it
2222
# without touching packages/spec at all.
2323
- 'content/docs/**'
24+
# Same for the strictness ledger — it is a doc, and editing it can break
25+
# the gate that now holds it to the code.
26+
- 'docs/audits/**'
2427

2528
permissions:
2629
contents: read
@@ -61,3 +64,12 @@ jobs:
6164
# apps.mdx said the nav tree had "eight item types" while the schema had nine.
6265
- name: Check discriminated-union variants are documented
6366
run: pnpm --filter @objectstack/spec check:variant-docs
67+
68+
# #4001 follow-up. The strictness ledger is the campaign's map — which
69+
# z.object sites are authorable, wire or open. Every step reads it to pick
70+
# the next move, and nothing kept it honest: it went stale twice in one
71+
# week (a classification verification disproved, and a "next step" that had
72+
# shipped months earlier). A map that drifts is worse than no map, because
73+
# it gets followed. This holds its arithmetic and its coverage to the code.
74+
- name: Check the strictness ledger matches the code
75+
run: pnpm --filter @objectstack/spec check:strictness-ledger

docs/audits/2026-07-unknown-key-strictness-ledger.md

Lines changed: 56 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -129,24 +129,24 @@ Classification is per the rule above; **(p)** marks a provisional call made
129129
from the file's exports/JSDoc rather than a full read — verify before
130130
tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
131131

132-
### `ui/`192 sites
132+
### `ui/`197 sites
133133

134134
| File | Sites | Class | Note / next action |
135135
|---|---|---|---|
136-
| `action.zod.ts` | 8 | authorable | param schema strict (#3746); remaining blocks ride later steps |
137-
| `view.zod.ts` | 51 | authorable | partially strict (ADR-0089); long tail of sub-blocks |
136+
| `action.zod.ts` | 9 | authorable | param schema strict (#3746); remaining blocks ride later steps |
137+
| `view.zod.ts` | 50 | authorable | partially strict (ADR-0089); long tail of sub-blocks |
138138
| `component.zod.ts` | 29 | authorable | **next candidate** — SDUI component defs; check React-prop open slots first (p) |
139139
| `theme.zod.ts` | 14 | authorable (p) | authored themes |
140-
| `app.zod.ts` | 11 | authorable | **strict as of #4001 PR B**`AppSchema` + branding / area / context-selector / contribution, and the nav-item union converted to `z.discriminatedUnion('type', …)` (the union-error question, settled empirically: matched-branch-only errors, exact recursive paths, `toJSONSchema` clean). Per-target `params` stay open. PR A (#4142) tombstoned the seven audit-dead keys first |
140+
| `app.zod.ts` | 18 | authorable | **strict as of #4001 PR B**`AppSchema` + branding / area / context-selector / contribution, and the nav-item union converted to `z.discriminatedUnion('type', …)` (the union-error question, settled empirically: matched-branch-only errors, exact recursive paths, `toJSONSchema` clean). Per-target `params` stay open. PR A (#4142) tombstoned the seven audit-dead keys first |
141141
| `dashboard.zod.ts` | 11 | authorable | partially strict |
142142
| `widget.zod.ts` | 9 | authorable (p) | |
143143
| `page.zod.ts` | 7 | authorable | partially strict (ADR-0089) |
144-
| `chart.zod.ts` / `i18n.zod.ts` / `responsive.zod.ts` | 6+6+6 | authorable (p) | i18n label shapes are wide-open records by design — verify |
145-
| `dataset.zod.ts` / `animation.zod.ts` / `dnd.zod.ts` / `keyboard.zod.ts` / `touch.zod.ts` | 4 ea | authorable (p) | interaction configs |
144+
| `chart.zod.ts` / `i18n.zod.ts` / `responsive.zod.ts` | 6+6+4 | authorable (p) | i18n label shapes are wide-open records by design — verify |
145+
| `dataset.zod.ts` / `animation.zod.ts` / `dnd.zod.ts` / `keyboard.zod.ts` / `touch.zod.ts` | 4+4+4+4+7 | authorable (p) | interaction configs |
146146
| `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | |
147147
| `sharing.zod.ts` | 2 | authorable (p) | public-sharing config |
148148

149-
### `data/`163 sites
149+
### `data/`154 sites
150150

151151
| File | Sites | Class | Note |
152152
|---|---|---|---|
@@ -165,11 +165,11 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
165165
| `external-catalog.zod.ts` | 4 | wire (p) | |
166166
| `field-value.zod.ts` / `seed.zod.ts` / `validation.zod.ts` | 1 ea | mixed (p) | |
167167

168-
### `automation/`80 sites
168+
### `automation/`89 sites
169169

170170
| File | Sites | Class | Note |
171171
|---|---|---|---|
172-
| `flow.zod.ts` | 12 | authorable | **strict as of #4001** (4 schemas; `FlowVersionHistorySchema` is runtime — stays tolerant) |
172+
| `flow.zod.ts` | 11 | authorable | **strict as of #4001** (4 schemas; `FlowVersionHistorySchema` is runtime — stays tolerant) |
173173
| `sync.zod.ts` / `etl.zod.ts` | 12+10 | authorable (p) | authored pipelines — **candidates** |
174174
| `trigger-registry.zod.ts` | 11 | mixed | descriptors are code-registered (wire-ish); bindings authored |
175175
| `execution.zod.ts` | 8 | wire | run-state envelopes — never strict |
@@ -178,6 +178,8 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
178178
| `bpmn-interop.zod.ts` | 5 | wire (p) | interop import shapes |
179179
| `approval.zod.ts` | 4 | authorable | **strict as of #4001 step 3** — all four authoring schemas (node config / approver / escalation / decision-output). The published JSON schema carries `additionalProperties: false` into the Studio form AND `registerFlow()` config validation (#4027/#4040), so an unknown key in an approval node's `config` is rejected at registration too — verified: `z.toJSONSchema` on the strict lazySchema does not throw (#3746 hazard checked) |
180180
| `node-executor.zod.ts` | 4 | wire | executor contract |
181+
| `io-node-config.zod.ts` | 2 | authorable | `NotifyConfigSchema` / `HttpConfigSchema` (#4045) — the sibling contracts that validate the **open** `config` slot on flow `notify` / `http` nodes. Authored per-node, so the open-slot exemption above does not extend to them; candidate once the executors' own drift is verified |
182+
| `builtin-node-config.zod.ts` | 8 | authorable | Same family (#4045): the CRUD quartet, `screen`, `map`. Written from what the executors read rather than from the descriptors' `configSchema` literals, and reconciled bidirectionally by `builtin-node-form-zod-ledger.test.ts` — so unlike most rows here, this one already has a drift check of its own. Same candidacy note as `io-node-config` |
181183
| `webhook.zod.ts` | 1 | authorable (p) | spec-only (#3461) |
182184

183185
### `security/` — 20 sites
@@ -213,10 +215,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
213215

214216
## Next steps (verify-then-enforce, one shape at a time)
215217

216-
1. Promote this ledger to a machine-checked gate (pattern of
217-
`packages/spec/liveness/` + `check:liveness`) once enough of the surface is
218-
classified that the table above is enforceable rather than descriptive.
219-
2. Let the warning layer run in the wild for a release, then schedule the v18
218+
1. Let the warning layer run in the wild for a release, then schedule the v18
220219
strict close-out on what it actually reports — which is the whole point of
221220
having built it. Nothing more to do here until there is field data.
222221

@@ -292,5 +291,49 @@ apps (23 + 6 objects, 20 + 1 pages, 4 + 1 datasets, 3 + 1 dashboards) report
292291
**zero** unknown keys. No finding this time — worth recording precisely because
293292
the app step's `ACCOUNT_APP.defaultOpen` came from exactly this class of check.
294293

294+
## This file is now machine-checked
295+
296+
`pnpm --filter @objectstack/spec check:strictness-ledger` (wired into the Spec
297+
Liveness Check workflow) holds the two claims here that are mechanically
298+
checkable, so this map cannot go stale in silence again:
299+
300+
- **Site counts.** The method is stated above — `z.object(` occurrences per file
301+
— so every number in the triage tables is verifiable. A count that no longer
302+
matches means schemas were added or removed under a `Class` verdict nobody
303+
re-examined. Touching a file forces you back through this ledger.
304+
- **Coverage.** Every `*.zod.ts` in a triaged directory that HAS sites must have
305+
a row. A new one is undeclared surface. Zero-site files (pure enum/token
306+
modules like `data/date-macros.zod.ts`) are skipped — there is nothing to
307+
classify — and become reportable the day they grow their first `z.object(`.
308+
- **Section totals**, and that any row claiming "strict as of" names a file that
309+
really contains `.strict()`.
310+
311+
Deliberately NOT checked: the `Class` column. Authorable vs wire vs open is a
312+
judgement about who writes the input, and this campaign's rule is
313+
verify-before-tightening. The gate protects the arithmetic and the coverage so
314+
that judgement is always made against current code.
315+
316+
**What it found on its first run — 11 drifts, in a file being actively edited
317+
by the campaign that owns it.** Six counts had moved (`ui/app.zod.ts` 11 → 18,
318+
`ui/touch.zod.ts` 4 → 7, `ui/action.zod.ts` 8 → 9, `ui/view.zod.ts` 51 → 50,
319+
`ui/responsive.zod.ts` 6 → 4, `automation/flow.zod.ts` 12 → 11), two section
320+
totals no longer summed, and six files had no row at all. The `app.zod.ts` gap
321+
was **self-inflicted**: the app step (#4165) added seven schemas to that file
322+
and updated the row's prose without touching its count. Five of the six
323+
undeclared files turned out to have zero sites — which is what motivated the
324+
skip rule above — leaving `automation/io-node-config.zod.ts` as the one genuine
325+
omission, now classified.
326+
327+
That is the argument for the gate in one paragraph: the people most familiar
328+
with this ledger, editing it in the same week, still left eleven drifts in it.
329+
330+
**And then it worked for real, before it had even merged.** While the gate sat
331+
in review, `main` landed `automation/builtin-node-config.zod.ts` (#4045/#4228) —
332+
eight new sites, sibling to `io-node-config.zod.ts`. Merging `main` turned the
333+
gate red on a branch whose own diff had not touched a single schema, which is
334+
precisely the intended behaviour: the file arrived, so someone had to classify
335+
it. It is now a row. Every existing count survived that merge unchanged, so the
336+
failure was exactly as narrow as it should have been.
337+
295338
Long tail stays gated on a verification pass per shape — never a one-shot
296339
"make all ~453 sites strict" (ADR-0054 ratchet; #4001's own recommendation).

packages/spec/package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -210,6 +210,7 @@
210210
"check:liveness": "tsx scripts/liveness/check-liveness.mts",
211211
"check:empty-state": "tsx scripts/liveness/check-empty-state.mts",
212212
"check:variant-docs": "tsx scripts/check-variant-docs.mts",
213+
"check:strictness-ledger": "tsx scripts/check-strictness-ledger.mts",
213214
"gen:react-blocks": "tsx scripts/build-react-blocks-contract.ts",
214215
"check:react-blocks": "tsx scripts/build-react-blocks-contract.ts --check",
215216
"check:react-conformance": "tsx scripts/check-react-blocks-conformance.ts",

0 commit comments

Comments
 (0)