feat(pgpm): pgpm diff — identity-keyed semantic diff + migration generation

[pyramation]

Summary

Part C of constructive-planning#1340 (Phase 5 of #1329): pgpm diff <A> <B> compares two schema sources semantically and can emit the delta as a deployable pgpm module.

pgpm diff <A> <B> [--emit-migration <dir>] [--pkg <name>] [--granularity ...] [--naming ...] [--json] [--verify]

Each side may be a pgpm module dir (flattened in plan order via loadModuleSource), a raw .sql file, or a live database (postgres:// DSN or db:<name>, read via pg_dump --schema-only --no-owner). Both sides are normalized into an identity-keyed object set (identityOf-derived keys), so the diff is dial-invariant: the same schema authored atomic vs consolidated vs differently named/partitioned diffs as empty (tested explicitly).

The diff core stays in @pgpmjs/transform (diffChangeSets/diffSchemas in semantic-diff-driver.ts, landed with #1569) rather than a new @pgpmjs/diff package — it already owns restructureChanges, granularity, and the statement graph the emission pipeline needs; a separate package would just re-export those. Source ingestion (loadDiffSideFromDisk, sqlToDiffChanges, stripDumpPreamble, deltaChangesToRows) lives in @pgpmjs/export next to loadModuleSource and the catalog-equivalence helpers; the CLI is a thin wrapper.

Semantics:

• added / removed / modified per object (AST-normalized: location fields stripped, statement order within an object ignored)
• tables sub-diff column-by-column and constraint-by-constraint → ALTER TABLE ADD/DROP/ALTER COLUMN, ADD/DROP CONSTRAINT (never a rebuild); functions/views/policies/etc. modified → DROP + CREATE (never CREATE OR REPLACE)
• non-derivable alters (column default/not-null changes, drops without a name) → warning + -- TODO (not derivable) comment; removed+added similar objects are never guessed as renames
• --emit-migration: topo-sorted pgpm module (creates in dependency order, drops in reverse), spec-derived paths, graph-derived requires, generated revert/verify per change
• --verify: oracle mode — deploy A + emitted migration into a scratch DB, assert catalog equivalence with B deployed fresh (reuses snapshotCatalog/diffCatalogSnapshots from pgpm transform --check). Column comparison is order-insensitive (withoutColumnOrder): a drop+add migration can never reproduce a fresh deploy's physical column ordinals.

Two fixes in the existing driver surfaced by the e2e:

 diffTable():
-  add extras, then drop removed extras
+  drop removed extras first — a changed constraint that keeps its name must
+  DROP CONSTRAINT before the replacement ADD CONSTRAINT
+  removed AT_EnableRowSecurity/AT_ForceRowSecurity extras invert to their
+  Disable/NoForce pair instead of warning (so reverts disable RLS)

@pgpmjs/transform also bumps @pgsql/scripts to ^18.2.0 (installed 18.0.1 predated invertStatement/existenceCheck).

Tests

• pgpm/export/__tests__/diff-source.test.ts: side-kind resolution, dump-preamble stripping, module/sql loading, dial-invariance (atomic module vs consolidated module vs flat .sql → empty diff), delta→row conversion incl. alteration-path collisions.
• pgpm/transform/__tests__/semantic-diff-driver.test.ts: constraint drop-before-add ordering, RLS enable/disable inversion, non-derivable warnings (plus existing per-kind add/remove/modify, column sub-diffs, normalization, reverse-topo drops).
• pgpm/cli/__tests__/diff-e2e.test.ts (live Postgres): v1→v2 module (new table, new column, dropped column, changed function body, new policy, dropped index, changed constraint) → --emit-migration → deploy v1+migration → catalog equivalence with v2 → migration's own verify passes → revert returns to v1; plus --verify oracle run and module-vs-.sql parity.

Note: pgpm/export has pre-existing failures on main (export-utils/export-meta/export-flow/cross-flow-parity, schema-prefix renames) unrelated to this PR — verified by running the same suite on a clean main checkout.

Link to Devin session: https://app.devin.ai/sessions/798445577ac8487abf80e28f9ff7a916
Requested by: @pyramation

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	@​pgsql/​scripts@​18.2.0

View full report