Skip to content

Commit 04660e8

Browse files
committed
build: merge driver for generator-owned spec artifacts (#4675)
`packages/spec`'s checked-in artifacts are sorted arrays and append-only ledgers. Two PRs each adding a few lines is a set union — semantically composable — but git reports a text conflict a human must resolve. Measured over one afternoon (2026-08-02): four merges, nine conflicts across those files, and NOT ONE was a real semantic conflict. Every correct resolution was the same three steps — discard both sides, re-run the generator, re-run the gates — at a dozen-plus minutes each, on a `main` moving fast enough that the rerun could itself go stale. `.gitattributes` now routes those paths to `merge=os-regen`. ## The driver deliberately does not regenerate The obvious implementation — "on conflict, run the generator" — is wrong, and measurably so. Git invokes merge drivers WHILE merging, in index order, and the worktree still holds pre-merge sources at that moment. Verified directly: `packages/spec/spec-changes.json` sorts before `packages/spec/src/...`, and a driver firing there sees a `migrations/registry.ts` with the incoming side's retirements missing. It would write a confidently wrong artifact. That is strictly worse than the conflict it replaces. A conflict marker is a visible error; a plausible generated file is an invisible one — and this repo already has the scar: on #4687 a `gen:api-surface` against an incomplete `dist` silently dropped an unrelated `./studio` export and ratcheted a baseline exemption in to cover the hole. Nothing failed; it was caught by diffing generated files against `main`. So the driver defers: it resolves the path (no markers, exit 0) and records it in `$GIT_DIR/os-regen-pending`. A `pre-commit` hook then refuses the commit until those artifacts check clean — regeneration happens on the fully-merged tree, the only state in which it is correct, and cannot be forgotten. The hook verifies and clears; it never regenerates, because blanket regeneration rewrites artifacts whose staleness nobody saw (the same reason `check:generated` refuses it). A marker cannot get stuck: it clears the moment the artifacts are current. ## The dist trap, made unsurvivable where it writes `check:generated --fix` now REFUSES `gen:api-surface` when `dist` is older than `src`, rather than printing advice a reader can skip. On a stale dist that generator does not fail — it emits a plausible surface missing every export added since the last build. `--fix` is the one path that writes, so it is the one place the trap cannot be survived. The staleness rule is shared with the pre-commit half rather than copied, because the direction two copies drift in is the one that writes a wrong artifact. ## Excluded on purpose Shrink-only ratchets (`docs-import-surface.baseline.json`, `dual-source-exports.baseline.json`), the hand-written migrations/conversions registries, and `variant-docs.json` stay on text merge. Recomputing a shrink-only ratchet can WIDEN it, laundering a new exemption in as merge noise. `NOT_DRIVER_MANAGED` records the reason per path. ## Verification `pnpm check:merge-driver` reconciles `.gitattributes` against the one table in both directions, pins that `.githooks/pre-commit` is mode 100755 in the index (git silently IGNORES a non-executable hook — caught exactly that way here, with two e2e commits sailing past an installed-but-inert hook), and proves the driver end to end against real git. Beyond the self-tests, the whole loop was exercised on real files and real generators: two branches each appending a retirement, then merged. Three conflicts became one (the hand-written registry), both generated files came out marker-free and recorded, committing without regenerating was BLOCKED with the exact commands, and after regenerating the commit went through with the marker self-cleared — with both sides' entries present, which is the set union the text merge could not express. Registration is per clone via `prepare`; an unregistered clone falls back to git's default text merge — pre-#4675 behaviour, not breakage. Closes #4675 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NKcGqCYCCpMkB5UW8jNPXx
1 parent 742cebb commit 04660e8

10 files changed

Lines changed: 741 additions & 0 deletions

File tree

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
"@objectstack/spec": patch
3+
---
4+
5+
build: 为生成物加 `merge=os-regen` 合并驱动,把「集合运算被打成文本冲突」的返工消掉 (#4675)
6+
7+
`packages/spec` 的生成物是排序数组与追加式登记表。两个 PR 各增删几行,语义上是集合并与集合差、完全可组合,git 却按三路文本合并报成需要人工解决的冲突 —— 2026-08-02 一个下午实测四次合并、九处冲突,**没有一次是真正的语义冲突**,每次的正确解法都是「丢掉两边、重新生成、重跑门禁」。
8+
9+
`.gitattributes` 现在把这些路径交给 `scripts/git-merge-regen.mjs`
10+
11+
**驱动不做重算。** git 是在合并**过程中**按索引顺序调用 merge driver 的,那一刻工作区里还是合并前的源码:`packages/spec/spec-changes.json` 排在 `packages/spec/src/...` 之前,所以在驱动里跑生成器会读到缺了对方那半边改动的 `migrations/registry.ts`,写出一个自信而错误的产物 —— 比它取代的那个冲突更糟,因为冲突标记是可见的错误,而看起来合理的生成文件不是。改为**推迟**:驱动解析路径(不做文本合并、不留标记)并记入 `$GIT_DIR/os-regen-pending`,`pre-commit` 在产物重新生成之前拒绝提交。重算因此发生在合并后的完整树上 —— 唯一正确的时刻。
12+
13+
`check:generated --fix` 现在在 `dist``src` 旧时**拒绝**运行 `gen:api-surface`,而不再只是警告。陈旧 dist 下该生成器不会失败,它会写出一份缺失了上次构建以来所有新导出的、看似合理的 surface,并让 `gen:docs` 顺手为这个缺口棘轮一条基线豁免(#4687 实际发生过,只靠与 `main` 对比生成物才发现)。`--fix` 是唯一会**写入**的路径,所以是这个陷阱唯一不可幸存的地方。
14+
15+
只减不增的棘轮(`docs-import-surface.baseline.json``dual-source-exports.baseline.json`)与手写登记表刻意排除在外:重算一个只减不增的棘轮可能**放宽**它,等于把一条新豁免当作合并噪音洗进来。这些冲突仍然留给人看,逐条理由见 `scripts/regen-artifacts.mjs``NOT_DRIVER_MANAGED`
16+
17+
驱动按 clone 注册(`pnpm install``prepare` 完成)。没注册的 clone 回退到 git 默认文本合并 —— 即 #4675 之前的行为,不是故障。`pnpm check:merge-driver` 双向核对 `.gitattributes` 与该表,并对真实 git 做端到端验证。

.gitattributes

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
# Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
#
3+
# Merge semantics for generator-owned artifacts (#4675).
4+
#
5+
# These files are sorted arrays and append-only ledgers derived from source. When
6+
# two PRs each add or drop a few lines the result is a set union — fully
7+
# composable — but a three-way TEXT merge reports it as a conflict a human must
8+
# resolve by hand. The correct resolution is always the same: discard both sides
9+
# and re-run the generator. `authorable-surface.json` alone is a 8k-line sorted
10+
# array, so any two PRs landing near each other collide.
11+
#
12+
# `merge=os-regen` hands those paths to `scripts/git-merge-regen.mjs`, which does
13+
# NOT text-merge them. See that file for why it also does not regenerate them
14+
# in place (git runs merge drivers BEFORE the sources are merged, so anything
15+
# computed there describes a half-merged tree).
16+
#
17+
# The driver is registered per clone by `scripts/setup-git-hooks.mjs`, which
18+
# `pnpm install` runs. A clone WITHOUT it registered falls back to git's default
19+
# text merge — i.e. exactly today's behaviour — so committing this file cannot
20+
# regress anyone.
21+
#
22+
# The single source of truth for this list is `scripts/regen-artifacts.mjs`;
23+
# `node scripts/git-merge-regen.mjs --self-test` reconciles the two in both
24+
# directions. Add a path there, not only here.
25+
#
26+
# Deliberately absent: docs-import-surface.baseline.json and
27+
# dual-source-exports.baseline.json (shrink-only ratchets — recomputing can
28+
# WIDEN them), variant-docs.json and the migrations/conversions registries
29+
# (hand-written). Those conflicts are for a human. See NOT_DRIVER_MANAGED.
30+
31+
packages/spec/spec-changes.json merge=os-regen
32+
packages/spec/authorable-surface.json merge=os-regen
33+
packages/spec/json-schema.manifest.json merge=os-regen
34+
packages/spec/api-surface.json merge=os-regen
35+
packages/spec/api-surface-signatures.json merge=os-regen
36+
docs/protocol-upgrade-guide.md merge=os-regen
37+
content/docs/references/** merge=os-regen

.githooks/pre-commit

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
#!/bin/sh
2+
# Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
3+
#
4+
# Registered by `scripts/setup-git-hooks.mjs` via `core.hooksPath=.githooks`,
5+
# which `pnpm install` runs. Cheap by construction: with no pending marker it
6+
# exits before doing any work, which is every commit that did not just merge a
7+
# generator-owned artifact (#4675).
8+
9+
if [ -z "$OS_SKIP_REGEN_CHECK" ]; then
10+
node "$(git rev-parse --show-toplevel)/scripts/check-regen-pending.mjs" || exit 1
11+
fi

AGENTS.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,6 +228,42 @@ Even inside your own worktree, operate defensively:
228228
commit itself — that second CI round is where joint breakage surfaces, and
229229
the guards in `scripts/check-*.mjs` exist largely because this class of
230230
breakage is invisible to `git merge`.
231+
11. **Generated artifacts don't text-merge — a driver defers them and
232+
`pre-commit` collects the debt.** §10's "never trust git's textual merge of a
233+
generated file" is now mechanical (#4675). `.gitattributes` routes the
234+
generator-owned artifacts (`spec-changes.json`, `authorable-surface.json`,
235+
`api-surface*.json`, `json-schema.manifest.json`,
236+
`docs/protocol-upgrade-guide.md`, `content/docs/references/**`) to
237+
`merge=os-regen`, so a merge that used to stop on conflicts across all of
238+
them now stops only on the hand-written files that actually need you.
239+
240+
The driver does **not** regenerate. Git runs merge drivers *while* it merges,
241+
in index order, so the worktree still holds pre-merge sources — a generator
242+
run there would describe a half-merged tree and write a confidently wrong
243+
artifact, which is strictly worse than the conflict it replaced. Instead it
244+
records each path in `$GIT_DIR/os-regen-pending`, and `pre-commit` refuses the
245+
commit until those artifacts check clean. So the sequence after a merge is
246+
unchanged from §9 — rebuild, then `check:generated --fix` — you just cannot
247+
forget it.
248+
249+
Two things worth knowing:
250+
- **Registration is per clone.** `pnpm install` does it (`prepare`
251+
`scripts/setup-git-hooks.mjs`). A clone where that never ran falls back to
252+
git's default text merge — pre-#4675 behaviour, not breakage — so nothing
253+
depends on every machine being set up.
254+
- **The ratchets are deliberately excluded**
255+
(`docs-import-surface.baseline.json`, `dual-source-exports.baseline.json`,
256+
the hand-written `migrations`/`conversions` registries, `variant-docs.json`).
257+
Recomputing a shrink-only ratchet can *widen* it, which would launder a new
258+
exemption in as merge noise. Those conflicts are yours to read. See
259+
`NOT_DRIVER_MANAGED` in `scripts/regen-artifacts.mjs` for why, per path.
260+
261+
Related: `check:generated --fix` now **refuses** to run `gen:api-surface` on a
262+
stale `dist` rather than warning about it (§9's trap, made unsurvivable on the
263+
one path that writes).
264+
265+
`pnpm check:merge-driver` reconciles `.gitattributes` against that table in
266+
both directions and proves the driver end to end against real git.
231267

232268
---
233269

package.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,8 @@
1515
"typecheck": "turbo run typecheck",
1616
"clean": "turbo run clean && rm -rf dist",
1717
"setup": "pnpm install && pnpm --filter @objectstack/spec build",
18+
"prepare": "node scripts/setup-git-hooks.mjs",
19+
"check:merge-driver": "node scripts/git-merge-regen.mjs --self-test && node scripts/check-regen-pending.mjs --self-test",
1820
"version": "changeset version && node scripts/sync-protocol-version.mjs && node scripts/sync-template-versions.mjs",
1921
"release": "pnpm run build && bash scripts/build-console.sh && bash scripts/release-publish.sh",
2022
"docs:dev": "pnpm --filter @objectstack/docs dev",

packages/spec/scripts/check-generated.ts

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,11 @@ import { readFileSync } from 'node:fs';
3030
import { dirname, join } from 'node:path';
3131
import { fileURLToPath } from 'node:url';
3232

33+
// One staleness rule, shared with the merge driver's pre-commit half (#4675) —
34+
// two copies of "is dist older than src" would drift, and the direction they
35+
// drift in is the one that writes a wrong artifact.
36+
import { distIsStale } from '../../../scripts/check-regen-pending.mjs';
37+
3338
const pkgRoot = join(dirname(fileURLToPath(import.meta.url)), '..');
3439

3540
/**
@@ -232,6 +237,24 @@ if (!fix) {
232237
console.log(`\n--fix: regenerating the ${stale.length} stale artifact(s). Review the diff before committing.\n`);
233238
let failed = 0;
234239
for (const s of stale) {
240+
// The `readsDist` warning above is advice a reader can ignore; here it must
241+
// become a refusal. `gen:api-surface` on a stale dist does not fail — it
242+
// writes a plausible surface with every export added since the last build
243+
// missing, and `gen:docs` then ratchets a baseline exemption in to cover the
244+
// hole. That landed unnoticed on #4687 and was caught only by diffing the
245+
// generated files against `main`. --fix is the one path that WRITES, so it is
246+
// the one place the trap is unsurvivable: a visible conflict is recoverable,
247+
// a confidently wrong artifact is not (#4675).
248+
if (s.readsDist && distIsStale()) {
249+
failed++;
250+
console.log(` ✗ ${s.gen} — REFUSED`);
251+
console.error(
252+
` packages/spec/dist is missing or older than packages/spec/src.\n`
253+
+ ` Regenerating now would write a surface describing a build that no longer exists.\n`
254+
+ ` pnpm --filter @objectstack/spec build && pnpm --filter @objectstack/spec ${s.gen}`,
255+
);
256+
continue;
257+
}
235258
const { ok, output } = run(s.gen);
236259
console.log(` ${ok ? '✓' : '✗'} ${s.gen}`);
237260
if (!ok) {

scripts/check-regen-pending.mjs

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
#!/usr/bin/env node
2+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
3+
4+
/**
5+
* The other half of the `merge=os-regen` driver (#4675): make the deferred
6+
* regeneration **mandatory** instead of remembered.
7+
*
8+
* The driver resolves generator-owned artifacts without text-merging them and
9+
* records each one in `$GIT_DIR/os-regen-pending`. It cannot regenerate them
10+
* itself — git runs merge drivers before the sources are merged, so anything
11+
* computed there describes a half-merged tree (see `git-merge-regen.mjs`). This
12+
* runs from `pre-commit`, where the merged tree finally exists, and refuses the
13+
* commit while any pending artifact is still stale.
14+
*
15+
* It **verifies, then clears** — it does not regenerate. Blanket regeneration
16+
* from a hook would rewrite artifacts whose staleness nobody saw, which is the
17+
* signal-destroying behaviour `check:generated` already refuses for the same
18+
* reason. And a marker cannot get stuck: the moment the artifacts check clean,
19+
* whether you regenerated them or the merge simply did not change them, the
20+
* marker is removed and the commit proceeds.
21+
*
22+
* ## The dist trap
23+
*
24+
* `gen:api-surface` reads the BUILT `dist/*.d.ts`. On a stale dist it does not
25+
* fail — it emits a plausible surface missing every export added since the last
26+
* build. So for `readsDist` artifacts this refuses to even run the gate unless
27+
* the build is newer than the sources, because a phantom "breaking removal" has
28+
* cost real triage time before (#4687, and the trap is recorded in AGENTS.md).
29+
*
30+
* Usage:
31+
* node scripts/check-regen-pending.mjs # pre-commit
32+
* node scripts/check-regen-pending.mjs --self-test # no repo state touched
33+
*/
34+
35+
import { execFileSync, execSync } from 'node:child_process';
36+
import { existsSync, readFileSync, readdirSync, rmSync, statSync } from 'node:fs';
37+
import { dirname, join, resolve } from 'node:path';
38+
import { fileURLToPath } from 'node:url';
39+
40+
import { PENDING_MARKER, entryForPath } from './regen-artifacts.mjs';
41+
42+
const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), '..');
43+
const SPEC_DIR = join(REPO_ROOT, 'packages/spec');
44+
45+
/** Newest mtime under `dir` for files matching `pred`, or 0 when there are none. */
46+
function newestMtime(dir, pred, depth = 0) {
47+
if (depth > 12 || !existsSync(dir)) return 0;
48+
let newest = 0;
49+
for (const e of readdirSync(dir, { withFileTypes: true })) {
50+
if (e.name === 'node_modules' || e.name.startsWith('.')) continue;
51+
const p = join(dir, e.name);
52+
if (e.isDirectory()) newest = Math.max(newest, newestMtime(p, pred, depth + 1));
53+
else if (pred(e.name)) newest = Math.max(newest, statSync(p).mtimeMs);
54+
}
55+
return newest;
56+
}
57+
58+
/**
59+
* Is `packages/spec/dist` older than the sources it claims to describe? Missing
60+
* counts as stale. Deliberately conservative: a false "stale" costs a build, a
61+
* false "fresh" costs a silently wrong artifact.
62+
*/
63+
export function distIsStale(specDir = SPEC_DIR) {
64+
const dist = newestMtime(join(specDir, 'dist'), (n) => n.endsWith('.d.ts'));
65+
if (!dist) return true;
66+
return newestMtime(join(specDir, 'src'), (n) => n.endsWith('.ts')) > dist;
67+
}
68+
69+
function markerPath() {
70+
const gitDir = execFileSync('git', ['rev-parse', '--absolute-git-dir'], { encoding: 'utf8' }).trim();
71+
return join(gitDir, PENDING_MARKER);
72+
}
73+
74+
function readPending(marker) {
75+
if (!existsSync(marker)) return [];
76+
return [...new Set(readFileSync(marker, 'utf8').split('\n').map((l) => l.trim()).filter(Boolean))];
77+
}
78+
79+
function runCheck(script) {
80+
try {
81+
execSync(`pnpm -s ${script}`, { cwd: SPEC_DIR, stdio: ['ignore', 'pipe', 'pipe'] });
82+
return { ok: true, output: '' };
83+
} catch (err) {
84+
return { ok: false, output: `${err?.stdout?.toString() ?? ''}${err?.stderr?.toString() ?? ''}`.trim() };
85+
}
86+
}
87+
88+
function main() {
89+
const marker = markerPath();
90+
const pending = readPending(marker);
91+
if (!pending.length) return 0;
92+
93+
const entries = pending.map((p) => ({ path: p, entry: entryForPath(p) })).filter((x) => x.entry);
94+
const unknown = pending.filter((p) => !entryForPath(p));
95+
96+
console.error(
97+
`\nos-regen: ${pending.length} generated artifact(s) were merged WITHOUT a text merge and must be `
98+
+ `regenerated from the merged tree before this commit.\n`,
99+
);
100+
101+
// Group by gate: `gen:schema` owns two artifacts, so running it twice is waste.
102+
const byCheck = new Map();
103+
for (const { path, entry } of entries) {
104+
const g = byCheck.get(entry.check) ?? { entry, paths: [] };
105+
g.paths.push(path);
106+
byCheck.set(entry.check, g);
107+
}
108+
109+
let blocked = 0;
110+
for (const [check, { entry, paths }] of byCheck) {
111+
if (entry.readsDist && distIsStale()) {
112+
blocked++;
113+
console.error(
114+
` ✗ ${paths.join(', ')}\n`
115+
+ ` ${check} reads packages/spec/dist, which is older than src — NOT running it.\n`
116+
+ ` On a stale dist this gate reports phantom removals and the generator WRITES them.\n`
117+
+ ` pnpm --filter @objectstack/spec build && pnpm --filter @objectstack/spec ${entry.gen}`,
118+
);
119+
continue;
120+
}
121+
const { ok, output } = runCheck(check);
122+
if (ok) {
123+
console.error(` ✓ ${paths.join(', ')} — current`);
124+
continue;
125+
}
126+
blocked++;
127+
const detail = output.split('\n').filter(Boolean).slice(0, 3).map((l) => ` ${l}`).join('\n');
128+
console.error(` ✗ ${paths.join(', ')} — stale\n${detail ? `${detail}\n` : ''}`
129+
+ ` pnpm --filter @objectstack/spec ${entry.gen}`);
130+
}
131+
132+
for (const p of unknown) {
133+
blocked++;
134+
console.error(` ✗ ${p} — recorded as pending but absent from scripts/regen-artifacts.mjs (cannot verify)`);
135+
}
136+
137+
if (blocked) {
138+
console.error(
139+
`\nRegenerate the ${blocked} stale artifact(s) above, \`git add\` them, and commit again.\n`
140+
+ ' This check clears itself the moment they are current — nothing to reset by hand.\n'
141+
+ ' Bypass with --no-verify only if you intend CI to catch it: every one of these has a\n'
142+
+ ' required gate on the PR.\n',
143+
);
144+
return 1;
145+
}
146+
147+
rmSync(marker, { force: true });
148+
console.error('os-regen: all deferred artifacts are current — marker cleared.\n');
149+
return 0;
150+
}
151+
152+
// `check:generated --fix` imports `distIsStale` from here, so nothing may run on
153+
// import — only when this file IS the entry point.
154+
const invokedDirectly = process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url);
155+
156+
if (invokedDirectly) {
157+
if (process.argv.includes('--self-test')) {
158+
// Touches no repo state: the interesting logic is the staleness rule, and its
159+
// dangerous direction is "says fresh when stale".
160+
const ok = distIsStale(join(REPO_ROOT, 'scripts')) === true;
161+
console.log(`${ok ? '✓' : '✗'} a directory with no dist/ reads as STALE (conservative default)`);
162+
console.log(ok ? '\n✓ check-regen-pending self-test passed.' : '\n✗ self-test failed.');
163+
process.exit(ok ? 0 : 1);
164+
}
165+
process.exit(main());
166+
}

0 commit comments

Comments
 (0)