SCHEMAS.md merge driver breaks rebases due to worktree side-effect

[aspiers]

Summary

The custom merge driver registered for SCHEMAS.md works correctly for one-shot merges but breaks multi-step rebases by leaving the worktree dirty between rebase steps. This is a known limitation (see commit 7379cb1: "Although it turns out this doesn't work with rebases anyway.") that has not been fixed.

Repro

Rebase any branch with lexicon changes onto a main that also contains lexicon changes:

git fetch origin
git rebase origin/main

Result, at the first commit that touches SCHEMAS.md:

Rebasing (2/16)
✅ Generated /data/projects/hypercerts-lexicon/SCHEMAS.md
error: Your local changes to the following files would be overwritten by merge:
        SCHEMAS.md
Please commit your changes or stash them before you merge.
Aborting

Root cause

Current driver (from scripts/setup-merge-driver.sh):

npm run gen-schemas-md --silent && cp -- SCHEMAS.md "%A"

A merge driver is a function: given (base, ours, theirs), write the resolution to %A. Side-effects on the worktree are forbidden.

npm run gen-schemas-md always writes to ./SCHEMAS.md in the worktree, then cp copies that into %A. The first half is the side-effect.

• In a one-shot merge the dirty worktree is harmless: git stages the resolution from %A, the worktree gets cleaned up by the merge finalization.
• In a multi-step rebase git wants the worktree to match each next replayed commit's tree. The driver's leftover SCHEMAS.md modification doesn't match → your local changes would be overwritten → abort.

Workaround

Bypass the driver for the duration of the rebase (use the trivial identity merge for SCHEMAS.md), then regenerate at the end:

git -c merge.schemas.driver=true rebase origin/main
npm run gen-schemas-md
git add SCHEMAS.md
git commit -m "chore: regenerate SCHEMAS.md after rebase"

This is what I had to do to land PR #219.

Suggested fix

Teach the driver to write only to %A, never to the worktree. Either:

1. Add an --outfile flag to scripts/generate-schemas.js and call npm run gen-schemas-md -- --outfile "%A" from the driver — no cp, no worktree mutation.
2. Or in the driver, snapshot/restore: cp SCHEMAS.md SCHEMAS.md.bak && npm run gen-schemas-md --silent && cp -- SCHEMAS.md "%A" && mv SCHEMAS.md.bak SCHEMAS.md. Uglier but no script change.

Option 1 is cleaner.

Until then

Add an inline comment to scripts/setup-merge-driver.sh noting the rebase limitation and the -c merge.schemas.driver=true workaround, so future contributors (and AI agents) don't rediscover it from scratch.

References

• 7379cb1 — first acknowledgement: "Although it turns out this doesn't work with rebases anyway."
• 91500b6 — replaced git add -u -- %A with cp SCHEMAS.md %A to fix the index-lock error (merge drivers can't call git themselves).
• c769bd1 — quoted/escaped paths in the cp invocation.
• PR [HYPER-181] add cryptographic signature support to record lexicons (resubmission of #170) #219 — first instance where the rebase incompatibility actually blocked work.