ci(spec): gate generated reference docs, and un-dormant the sibling generated gates

content/docs/references/** is generated from packages/spec and committed, but no CI
job ever regenerated and diffed it — `grep -rlE "gen:docs|gen:schema" .github/workflows/`
returned nothing. So the public reference docs drifted silently while main stayed
green: #3076 added RowCrudActionOverride to the spec and the docs never learned the
type existed. The parent commit regenerates; this one stops it recurring.

build-docs.ts --check
  Follows the sibling convention (check:skill-docs / check:api-surface / …). Every
  write now goes through emit() and every wiped folder through manageDir(), so both
  modes run identical generation logic and differ only in the final disposition —
  write, or compare. That shared path is what makes the gate trustworthy: it cannot
  pass on output a real run would not produce. Verified output-identical to the
  previous generator (byte-for-byte across all 258 files).

  Catches all three drift classes, each proven to fail before being trusted:
  stale content (the #3076 case), a missing page, and a page left behind by a type
  removed from spec. The last two are why this is not `git diff --exit-code`: that
  misses untracked files. json-schema/ is gitignored, so `check:docs` runs gen:schema
  first and a run that finds no schemas errors rather than passing vacuously.

Placement
  In lint.yml's "TypeScript Type Check", not ci.yml's "Build Docs". Build Docs is
  gated on a `docs` paths-filter that excludes packages/spec/**, so it skips exactly
  the spec-only PRs that cause this drift — #3076 was one. The typecheck job has no
  paths filter and is a required status check, so the gate cannot go dormant. It reads
  src/ + json-schema/ via tsx and needs no build, so it runs before the workspace
  build and fails in ~2s.

Same hole found in the sibling job (fixed here, both currently in sync but unguarded):
  - check:spec-changes / check:upgrade-guide read the ADR-0087 registries and the
    protocol version, but ran under a filter listing only skills/** — a protocol bump
    or new migration could not trigger the job that verifies them.
  - the filter watched content/docs/guides/skills.mdx, a path #2584 moved; hand-edits
    to the real generated block in content/docs/ai/skills-reference.mdx went unchecked.
  Job renamed check-skill-docs → check-generated: it has guarded the protocol
  artifacts for a while, and a name that hid them is part of why their paths went
  missing from the filter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Author: os-zhuang

.github/workflows/ci.yml

@@ -24,7 +24,7 @@ jobs:
     outputs:
       docs: ${{ steps.changes.outputs.docs }}
       core: ${{ steps.changes.outputs.core }}
-      skilldocs: ${{ steps.changes.outputs.skilldocs }}
+      generated: ${{ steps.changes.outputs.generated }}
     steps:
       - name: Checkout repository
         uses: actions/checkout@v7
@@ -38,10 +38,30 @@ jobs:
               - 'content/**'
               - 'pnpm-lock.yaml'
               - '.github/workflows/ci.yml'
-            skilldocs:
+            # Inputs AND outputs of every generated artifact checked by the
+            # `check-generated` job. A path missing here makes that gate dormant on
+            # exactly the PRs that can break it, so keep the two in lockstep.
+            generated:
+              # check:skill-docs — catalog source is skills/*/SKILL.md; the derived
+              # listings are skills/README.md + the reference page below. (That page
+              # was content/docs/guides/skills.mdx until #2584 moved it; this filter
+              # kept watching the old path, so hand-edits to the generated block went
+              # unchecked from then until now.)
               - 'skills/**'
-              - 'content/docs/guides/skills.mdx'
+              - 'content/docs/ai/skills-reference.mdx'
               - 'packages/spec/scripts/build-skill-docs.ts'
+              # check:spec-changes / check:upgrade-guide — derived from the ADR-0087
+              # registries and the protocol version, none of which the skills paths
+              # above cover. Both gates were dormant on spec-only PRs: a protocol bump
+              # or a new migration could not trigger the job that verifies them.
+              - 'packages/spec/scripts/build-spec-changes.ts'
+              - 'packages/spec/scripts/build-upgrade-guide.ts'
+              - 'packages/spec/src/migrations/**'
+              - 'packages/spec/src/conversions/**'
+              - 'packages/spec/src/kernel/protocol-version.ts'
+              - 'packages/spec/api-surface.json'
+              - 'packages/spec/spec-changes.json'
+              - 'docs/protocol-upgrade-guide.md'
               - '.github/workflows/ci.yml'
             core:
               - 'packages/**'
@@ -354,10 +374,18 @@ jobs:
       - name: Build Docs
         run: pnpm --filter @objectstack/docs build
 
-  check-skill-docs:
-    name: Check Skill Docs
+  # Generated artifacts that are committed to the tree: regenerate, fail on drift.
+  # Named for the class, not for skill docs alone — it has checked the ADR-0087
+  # protocol artifacts for a while, and a name that hid them is part of why their
+  # paths went missing from the filter above.
+  #
+  # The generated reference docs (content/docs/references/**) are deliberately NOT
+  # here: they need to gate spec-only PRs, and this whole job is filter-gated, so
+  # they live in lint.yml's required, unfiltered "TypeScript Type Check" job.
+  check-generated:
+    name: Check Generated Artifacts
     needs: filter
-    if: needs.filter.outputs.skilldocs == 'true'
+    if: needs.filter.outputs.generated == 'true'
     runs-on: ubuntu-latest
     permissions:
       contents: read

.github/workflows/lint.yml

@@ -119,6 +119,21 @@ jobs:
       - name: Type check (@objectstack/spec)
         run: pnpm --filter @objectstack/spec exec tsc --noEmit
 
+      # Generated-docs gate: content/docs/references/** is generated from the spec
+      # by `gen:schema && gen:docs` and committed. Nothing regenerated it in CI, so
+      # it drifted silently — #3076 added RowCrudActionOverride to the spec and the
+      # public reference docs never learned the type existed. Regenerates and fails
+      # on any difference.
+      #
+      # Deliberately lives in this job, not in ci.yml's "Build Docs": that job is
+      # gated on a `docs` paths-filter that does not include packages/spec/**, so it
+      # skips exactly the spec-only PRs that cause this drift (#3076 was one). This
+      # job has no paths filter and is a required status check, so the gate cannot go
+      # dormant. It reads src/ + json-schema/ via tsx and needs no build, so it runs
+      # before the workspace build and fails in ~2s.
+      - name: Check generated reference docs are in sync with the spec
+        run: pnpm --filter @objectstack/spec check:docs
+
       # Example apps are AI-authoring reference templates; a red typecheck is a
       # bad signal to copy from. tsup transpiles them without a full typecheck,
       # so build alone will not catch type drift — typecheck them explicitly.

packages/spec/package.json

@@ -188,6 +188,7 @@
     "gen:schema": "OS_EAGER_SCHEMAS=1 tsx scripts/build-schemas.ts",
     "gen:openapi": "tsx scripts/build-openapi.ts",
     "gen:docs": "tsx scripts/build-docs.ts",
+    "check:docs": "pnpm gen:schema && tsx scripts/build-docs.ts --check",
     "gen:skill-refs": "tsx scripts/build-skill-references.ts",
     "gen:skill-docs": "tsx scripts/build-skill-docs.ts",
     "check:skill-docs": "tsx scripts/build-skill-docs.ts --check",

packages/spec/scripts/build-docs.ts

@@ -11,6 +11,10 @@
 // Hand-written documentation lives in the module folders under content/docs/
 // (data-modeling/, automation/, permissions/, ui/, api/, ai/, plugins/, kernel/, ...).
 // See DX_ROADMAP.md and .cursorrules for details.
+//
+// Usage:
+//   tsx scripts/build-docs.ts            # write
+//   tsx scripts/build-docs.ts --check    # verify in sync (CI); exit 1 on drift
 
 import fs from 'fs';
 import path from 'path';
@@ -20,6 +24,107 @@ const SRC_DIR = path.resolve(__dirname, '../src');
 // Output directly to references folder (flattened)
 // ⚠️  Everything inside category sub-folders is auto-generated and disposable.
 const DOCS_ROOT = path.resolve(__dirname, '../../../content/docs/references');
+const REPO_ROOT = path.resolve(__dirname, '../../..');
+
+const CHECK = process.argv.includes('--check');
+
+// ── Output sink ──────────────────────────────────────────────────────────────
+// Every generated file goes through emit(), every wholesale-regenerated folder
+// through manageDir(). Nothing touches the output tree until flush(), so the
+// two modes run byte-for-byte identical generation logic and differ only in the
+// final disposition — write to disk, or compare against it. That shared path is
+// what makes --check trustworthy: it cannot pass on output a real run wouldn't
+// produce, because it *is* the real run minus the writes.
+
+/** Absolute path → intended content. */
+const emitted = new Map<string, string>();
+/** Absolute dirs regenerated wholesale — anything on disk here that we didn't
+ *  emit is stale, and a real run would delete it. */
+const managedDirs = new Set<string>();
+
+function emit(filePath: string, content: string): void {
+  emitted.set(path.resolve(filePath), content);
+}
+
+function manageDir(dir: string): void {
+  managedDirs.add(path.resolve(dir));
+}
+
+/** Files this run generated, for the read-after-write lookups below. */
+function wasEmitted(filePath: string): boolean {
+  return emitted.has(path.resolve(filePath));
+}
+
+function walk(dir: string): string[] {
+  if (!fs.existsSync(dir)) return [];
+  return fs.readdirSync(dir, { withFileTypes: true }).flatMap(e => {
+    const full = path.join(dir, e.name);
+    return e.isDirectory() ? walk(full) : [full];
+  });
+}
+
+const rel = (p: string) => path.relative(REPO_ROOT, p);
+
+/** Write the emitted tree, or (in --check) report how it differs from disk. */
+function flush(): void {
+  if (!CHECK) {
+    for (const dir of managedDirs) {
+      if (fs.existsSync(dir)) fs.rmSync(dir, { recursive: true, force: true });
+    }
+    for (const [file, content] of emitted) {
+      fs.mkdirSync(path.dirname(file), { recursive: true });
+      fs.writeFileSync(file, content);
+      console.log(`✓ Generated ${rel(file)}`);
+    }
+    console.log(`\n✅ Generated ${emitted.size} files`);
+    return;
+  }
+
+  // A run with no schemas to read emits almost nothing, and "nothing differs"
+  // would read as success — the gate would pass while checking no pages at all.
+  // json-schema/ is gitignored, so this is one forgotten `gen:schema` away on a
+  // fresh checkout. Fail loudly instead of greenly.
+  if (managedDirs.size === 0) {
+    console.error(
+      `\n✗ No JSON schemas found under ${rel(SCHEMA_DIR)} — nothing to check against.\n` +
+        `  Run \`pnpm --filter @objectstack/spec gen:schema\` first (\`check:docs\` does this for you).\n`,
+    );
+    process.exit(1);
+  }
+
+  const changed: string[] = [];
+  const added: string[] = [];
+  for (const [file, content] of emitted) {
+    if (!fs.existsSync(file)) added.push(rel(file));
+    else if (fs.readFileSync(file, 'utf-8') !== content) changed.push(rel(file));
+  }
+  // A managed folder is regenerated in full, so an on-disk file we didn't emit
+  // is one a real run would delete — e.g. the page of a type removed from spec.
+  const stale = [...managedDirs]
+    .flatMap(walk)
+    .filter(f => !wasEmitted(f))
+    .map(rel);
+
+  const drift = [
+    ...added.map(f => `  + ${f} (missing — spec adds it)`),
+    ...changed.map(f => `  ~ ${f} (out of date)`),
+    ...stale.map(f => `  - ${f} (stale — spec no longer defines it)`),
+  ];
+
+  if (drift.length === 0) {
+    console.log(`✅ ${emitted.size} reference files in sync with packages/spec`);
+    return;
+  }
+
+  console.error(
+    `\n✗ content/docs/references/ is out of date with packages/spec:\n\n` +
+      drift.join('\n') +
+      `\n\nThese files are GENERATED — do not hand-edit them. Regenerate and commit:\n\n` +
+      `  pnpm --filter @objectstack/spec gen:schema && pnpm --filter @objectstack/spec gen:docs\n` +
+      `  git add content/docs/references\n`,
+  );
+  process.exit(1);
+}
 
 // Dynamically discover categories from src directory
 const getCategoryTitle = (dir: string) => {
@@ -329,9 +434,7 @@ Object.keys(CATEGORIES).forEach(category => {
     }
     return;
   }
-  if (fs.existsSync(dir)) {
-    fs.rmSync(dir, { recursive: true, force: true });
-  }
+  manageDir(dir);
 });
 
 const generatedFiles: string[] = [];
@@ -368,24 +471,21 @@ Object.keys(CATEGORIES).forEach(category => {
     zodFileSchemas.get(zodFile)!.push({ name: schemaName, content });
   });
 
-  // Create Category Directory
   const categoryDir = path.join(DOCS_ROOT, category);
-  if (!fs.existsSync(categoryDir)) fs.mkdirSync(categoryDir, { recursive: true });
 
   // Generate file
   zodFileSchemas.forEach((schemas, zodFile) => {
     const fileName = `${zodFile}.mdx`;
     const mdx = generateZodFileMarkdown(zodFile, schemas, category);
-    fs.writeFileSync(path.join(categoryDir, fileName), mdx);
-    console.log(`✓ Generated ${category}/${fileName}`);
+    emit(path.join(categoryDir, fileName), mdx);
   });
-  
+
   // Generate Category Meta
   const meta = {
     title: CATEGORIES[category],
     pages: Array.from(zodFileSchemas.keys()).sort()
   };
-  fs.writeFileSync(path.join(categoryDir, 'meta.json'), JSON.stringify(meta, null, 2));
+  emit(path.join(categoryDir, 'meta.json'), JSON.stringify(meta, null, 2));
 });
 
 // 2.5 Generate Category Overviews (index.mdx in each folder)
@@ -408,24 +508,21 @@ Object.entries(CATEGORIES).forEach(([category, title]) => {
       // Flow edge schema carry CEL-expression transforms) — generates no page,
       // so carding it would be a dangling 404 link. This aligns the index with
       // `meta.json`, which already lists only generated pages.
-      if (!fs.existsSync(path.join(DOCS_ROOT, category, `${zodFile}.mdx`))) return;
+      //
+      // Asks the sink, not the disk: this run's own output is the authority on
+      // what pages exist. (Equivalent on disk, since the folder was just wiped
+      // and rewritten — but it stays correct under --check, where nothing is
+      // written and the stale files are still lying around.)
+      if (!wasEmitted(path.join(DOCS_ROOT, category, `${zodFile}.mdx`))) return;
       const fileTitle = zodFile.split('-').map(w => w.charAt(0).toUpperCase() + w.slice(1)).join(' ');
       // Link relative to the category folder (where index.mdx lives)
       mdx += `  <Card href="/docs/references/${category}/${zodFile}" title="${fileTitle}" description="Source: packages/spec/src/${category}/${zodFile}.zod.ts" />\n`;
   });
   mdx += `</Cards>\n`;
 
-  // Write as index.mdx inside the category folder? 
-  // If we do that, accessing /docs/references/ai works.
-  // BUT 'index' must be in 'meta.json' pages? No, index is implicit usually.
-  
-  // However, Fumadocs often treats folder/index.mdx as the page for the folder.
-  // Ensure directory exists before writing
-  const categoryDir = path.join(DOCS_ROOT, category);
-  if (!fs.existsSync(categoryDir)) fs.mkdirSync(categoryDir, { recursive: true });
-  
-  fs.writeFileSync(path.join(categoryDir, 'index.mdx'), mdx);
-  console.log(`✓ Generated ${category}/index.mdx`);
+  // Fumadocs treats folder/index.mdx as the page for the folder, so this is what
+  // makes /docs/references/<category> resolve.
+  emit(path.join(DOCS_ROOT, category, 'index.mdx'), mdx);
 });
 
 // 3. Update root meta.json
@@ -437,7 +534,9 @@ const categoryDirs = Object.keys(CATEGORIES)
   })
   .sort();
 
-// Collect other root files (if any exist, like implementation-status.mdx)
+// Collect other root files (if any exist, like implementation-status.mdx).
+// Root-level .mdx is hand-written and never generated, so this reads the disk in
+// both modes — it is an input to the sidebar, not part of the emitted tree.
 const rootFiles = fs.readdirSync(DOCS_ROOT)
   .filter(f => f.endsWith('.mdx') && !f.startsWith('index')) // Exclude index.mdx if it exists?
   .map(f => f.replace('.mdx', ''))
@@ -455,7 +554,7 @@ const meta = {
   // NOT be a root tab — the whole docs tree renders as one sidebar.
   pages: pages
 };
-fs.writeFileSync(path.join(DOCS_ROOT, 'meta.json'), JSON.stringify(meta, null, 2));
-console.log(`✓ Updated root meta.json`);
+emit(path.join(DOCS_ROOT, 'meta.json'), JSON.stringify(meta, null, 2));
 
-console.log('Done!');
+// 4. Disposition: write the tree, or report drift against it.
+flush();