ci(spec): run the check:generated ledger reconciliation on every PR (#4255) (#4257)

The aggregate reconciles its GATED/NO_GENERATOR ledger against package.json
on every run, in both directions — but only where the aggregate runs, which
was locally. CI runs the gates as individual steps, so a PR adding a
check:/gen: script without classifying it kept every CI gate green while the
wrapper AGENTS.md prescribes exited red on main at the reconcile stage,
running zero gates.

Twice in three days. #4177 added check:variant-docs and main stayed red for
local wrappers until #4194 collided with the same wall (#4203). Then #4232
added check:strictness-ledger — found unclassified while wiring this fix, so
the wrapper was red on main again at the moment of writing. Classified here
as NO_GENERATOR: it audits the hand-written strictness ledger (docs/audits/)
against the code it describes; there is no generator to name.

The gate is a --reconcile-only mode (reconcile, report, exit — no gates;
reads package.json only, no build, sub-second) wired into lint.yml's
"TypeScript Type Check" job. Deliberately not ci.yml's check-generated job:
its `generated` paths filter does not watch packages/spec/package.json — the
one file every offending PR must touch — and both offenders' changed files
missed every entry in that filter, so both skipped the job entirely. The
typecheck job is unfiltered and required, so the meta-gate cannot go
dormant — the reasoning that already placed check:docs, check:skill-refs and
check:react-blocks there.

Verified: --reconcile-only exits 0 on this tree (16 check: + 10 gen:, all
classified) after catching #4232's script pre-classification; an injected
unclassified script and a deleted ledger-named script both fail with the
existing diagnostics; the default mode still reconciles and then runs the
gates. AGENTS.md's wrapper section now states the CI backstop.

Closes #4255
Refs #4203, #4232

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QCrKazC5o5a77yXd7ykaZg

Author: os-zhuang

.changeset/reconcile-ledger-in-ci.md

@@ -0,0 +1,27 @@
+---
+"@objectstack/spec": patch
+---
+
+`check:generated --reconcile-only` runs the package.json ↔ ledger reconciliation and
+nothing else, and lint.yml's unfiltered required job now runs it on every PR.
+
+The aggregate already reconciled its GATED/NO_GENERATOR ledger against `package.json`
+on every run, in both directions, so an unclassified `check:`/`gen:` script fails the
+run instead of quietly dropping out of coverage. But the reconciliation only executed
+where the aggregate executed — locally. CI runs the gates as separate steps, so a PR
+adding a script without classifying it kept every CI gate green while
+`pnpm --filter @objectstack/spec check:generated` — the wrapper AGENTS.md prescribes
+before every spec push — exited at the reconcile stage on `main`, running zero gates.
+
+Not hypothetical: twice in three days. #4177 added `check:variant-docs` and `main`
+stayed red for local wrappers until #4194 happened to collide with the same wall
+(#4203); #4232 added `check:strictness-ledger`, caught while wiring this very step
+and classified in the same change — it audits the hand-written strictness ledger
+against the code it describes, so NO_GENERATOR.
+
+The step lives in lint.yml's "TypeScript Type Check" job rather than ci.yml's
+`check-generated` job because the latter is gated on a `generated` paths filter that
+does not watch `packages/spec/package.json` — the one file every offending PR must
+touch; both offenders skipped that job entirely. The typecheck job is unfiltered and
+required, so the meta-gate cannot go dormant — the same reasoning that already placed
+`check:docs`, `check:skill-refs` and `check:react-blocks` there.

.github/workflows/lint.yml

@@ -223,6 +223,23 @@ jobs:
       - name: Type check (@objectstack/spec)
         run: pnpm --filter @objectstack/spec exec tsc --noEmit
 
+      # Meta-gate (#4203): the `check:generated` aggregate reconciles its
+      # GATED/NO_GENERATOR ledger against packages/spec/package.json on every run,
+      # in both directions, so a `check:`/`gen:` script nobody classified fails
+      # instead of quietly dropping out of coverage. But the reconciliation only
+      # ran where the aggregate ran — locally. CI runs the gates as individual
+      # steps, so a PR adding an unclassified script kept every CI gate green
+      # while the wrapper AGENTS.md prescribes exited red on `main`, running zero
+      # gates. Twice in three days: #4177 (`check:variant-docs`, fixed only when
+      # #4194 collided with the same wall) and #4232 (`check:strictness-ledger`,
+      # caught while wiring this step). Not in ci.yml's `check-generated` job:
+      # its `generated` paths filter does not watch packages/spec/package.json —
+      # the one file every offending PR must touch — so both offenders skipped
+      # that job entirely. This job is unfiltered and required, so the meta-gate
+      # cannot go dormant. Reads package.json only; no build, sub-second.
+      - name: "Check every check:/gen: script is classified in the check:generated ledger"
+        run: pnpm --filter @objectstack/spec check:generated --reconcile-only
+
       # Generated-docs gate: content/docs/references/** is generated from the spec
       # by `gen:schema && gen:docs` and committed. Nothing regenerated it in CI, so
       # it drifted silently — #3076 added RowCrudActionOverride to the spec and the

AGENTS.md

@@ -265,7 +265,10 @@ The script carries its own ledger of gate → generator and **reconciles it agai
 `package.json` on every run**, in both directions. A new `check:`/`gen:` script that
 nobody classified fails the run rather than quietly dropping out of coverage — the
 failure mode a hardcoded list here would have had. (It caught its own `package.json`
-entry on the very first run.)
+entry on the very first run.) CI runs the same reconciliation on every PR
+(`--reconcile-only`, in lint.yml's required typecheck job), so an unclassified script
+fails its own PR instead of landing on `main` and turning this wrapper red for
+everyone else — which happened twice before the CI step existed (#4203, #4232).
 
 ⚠️ **`check:api-surface` reads the built `dist/*.d.ts`, not `src/`.** A stale `dist`
 makes it report exports as **removed** — "N breaking (removed/narrowed)" — when nothing

packages/spec/scripts/check-generated.ts

@@ -22,6 +22,7 @@
  * Usage:
  *   pnpm --filter @objectstack/spec check:generated          # report every stale artifact
  *   pnpm --filter @objectstack/spec check:generated --fix    # + regenerate exactly those
+ *   pnpm --filter @objectstack/spec check:generated --reconcile-only   # ledger audit only, no gates (CI)
  */
 
 import { execSync } from 'node:child_process';
@@ -67,6 +68,15 @@ const NO_GENERATOR: ReadonlyArray<{ check: string; why: string }> = [
   // failing on `main` itself. The doc it checks against is hand-written, so there
   // is no generator to name.
   { check: 'check:variant-docs', why: 'audits that each schema variant appears in its hand-written doc — no artifact' },
+  // The #4177 story again, one day after #4203 closed it: #4232 added this script
+  // and nothing in CI runs this reconciliation, so `main` went red for every local
+  // wrapper run a second time. Caught while wiring `--reconcile-only` into
+  // lint.yml's unfiltered job — the fix for exactly this class. The ledger it
+  // audits is a hand-maintained doc (docs/audits/), so there is no generator.
+  {
+    check: 'check:strictness-ledger',
+    why: 'audits the hand-written strictness ledger against the code it describes — no artifact',
+  },
   // The odd one out: it audits the source's TYPES, but reads them from the BUILT
   // `dist/*.d.ts` — the surface a consumer's import actually resolves to, which
   // is the only place the defect is visible (#4171). So the `readsDist` caveat
@@ -146,9 +156,32 @@ function run(script: string): { ok: boolean; output: string } {
 }
 
 const fix = process.argv.includes('--fix');
+// CI mode (#4203): reconcile and stop — no gates. The reconciliation above only
+// ever ran where this aggregate ran, which was locally: CI runs the gates as
+// individual steps, so an unclassified `check:`/`gen:` script kept every CI gate
+// green while this wrapper exited red on `main` before running a single gate.
+// Twice in three days — #4177 (fixed only by colliding with #4194) and #4232
+// (caught wiring this flag in). ci.yml's `check-generated` job cannot host the
+// fix: its `generated` paths filter does not watch packages/spec/package.json,
+// the one file every offending PR must touch — both offenders skipped that job
+// entirely. So lint.yml's unfiltered, required "TypeScript Type Check" job runs
+// this mode instead. Reads package.json and the arrays above; no build, <1s.
+const reconcileOnly = process.argv.includes('--reconcile-only');
 const scripts = JSON.parse(readFileSync(join(pkgRoot, 'package.json'), 'utf8')).scripts ?? {};
 reconcileLedger(scripts);
 
+if (reconcileOnly) {
+  const checks = Object.keys(scripts).filter((n) => n.startsWith('check:')).length;
+  const gens = Object.keys(scripts).filter((n) => n.startsWith('gen:')).length;
+  console.log(
+    `✓ check:generated ledger reconciles with package.json: ${checks} check: + ${gens} gen: scripts, ` +
+      `all classified (${GATED.length} gated, ${NO_GENERATOR.length} source audits, ` +
+      `${UNGATED_GENERATORS.length} ungated generators, 1 aggregate).\n` +
+      `  --reconcile-only: no gates were run — this verifies coverage, not artifacts.`,
+  );
+  process.exit(0);
+}
+
 console.log(`Checking ${GATED.length} generated artifacts (every gate runs — the first failure does not stop the rest).\n`);
 
 const stale: typeof GATED[number][] = [];