Skip to content

feat(pgpm): pgpm diff — identity-keyed semantic diff + migration generation - #1572

Merged
pyramation merged 2 commits into
mainfrom
feat/pgpm-diff
Aug 1, 2026
Merged

feat(pgpm): pgpm diff — identity-keyed semantic diff + migration generation#1572
pyramation merged 2 commits into
mainfrom
feat/pgpm-diff

Conversation

@pyramation

Copy link
Copy Markdown
Contributor

Summary

Part C of constructive-planning#1340 (Phase 5 of #1329): pgpm diff <A> <B> compares two schema sources semantically and can emit the delta as a deployable pgpm module.

pgpm diff <A> <B> [--emit-migration <dir>] [--pkg <name>] [--granularity ...] [--naming ...] [--json] [--verify]

Each side may be a pgpm module dir (flattened in plan order via loadModuleSource), a raw .sql file, or a live database (postgres:// DSN or db:<name>, read via pg_dump --schema-only --no-owner). Both sides are normalized into an identity-keyed object set (identityOf-derived keys), so the diff is dial-invariant: the same schema authored atomic vs consolidated vs differently named/partitioned diffs as empty (tested explicitly).

The diff core stays in @pgpmjs/transform (diffChangeSets/diffSchemas in semantic-diff-driver.ts, landed with #1569) rather than a new @pgpmjs/diff package — it already owns restructureChanges, granularity, and the statement graph the emission pipeline needs; a separate package would just re-export those. Source ingestion (loadDiffSideFromDisk, sqlToDiffChanges, stripDumpPreamble, deltaChangesToRows) lives in @pgpmjs/export next to loadModuleSource and the catalog-equivalence helpers; the CLI is a thin wrapper.

Semantics:

  • added / removed / modified per object (AST-normalized: location fields stripped, statement order within an object ignored)
  • tables sub-diff column-by-column and constraint-by-constraint → ALTER TABLE ADD/DROP/ALTER COLUMN, ADD/DROP CONSTRAINT (never a rebuild); functions/views/policies/etc. modified → DROP + CREATE (never CREATE OR REPLACE)
  • non-derivable alters (column default/not-null changes, drops without a name) → warning + -- TODO (not derivable) comment; removed+added similar objects are never guessed as renames
  • --emit-migration: topo-sorted pgpm module (creates in dependency order, drops in reverse), spec-derived paths, graph-derived requires, generated revert/verify per change
  • --verify: oracle mode — deploy A + emitted migration into a scratch DB, assert catalog equivalence with B deployed fresh (reuses snapshotCatalog/diffCatalogSnapshots from pgpm transform --check). Column comparison is order-insensitive (withoutColumnOrder): a drop+add migration can never reproduce a fresh deploy's physical column ordinals.

Two fixes in the existing driver surfaced by the e2e:

 diffTable():
-  add extras, then drop removed extras
+  drop removed extras first — a changed constraint that keeps its name must
+  DROP CONSTRAINT before the replacement ADD CONSTRAINT
+  removed AT_EnableRowSecurity/AT_ForceRowSecurity extras invert to their
+  Disable/NoForce pair instead of warning (so reverts disable RLS)

@pgpmjs/transform also bumps @pgsql/scripts to ^18.2.0 (installed 18.0.1 predated invertStatement/existenceCheck).

Tests

  • pgpm/export/__tests__/diff-source.test.ts: side-kind resolution, dump-preamble stripping, module/sql loading, dial-invariance (atomic module vs consolidated module vs flat .sql → empty diff), delta→row conversion incl. alteration-path collisions.
  • pgpm/transform/__tests__/semantic-diff-driver.test.ts: constraint drop-before-add ordering, RLS enable/disable inversion, non-derivable warnings (plus existing per-kind add/remove/modify, column sub-diffs, normalization, reverse-topo drops).
  • pgpm/cli/__tests__/diff-e2e.test.ts (live Postgres): v1→v2 module (new table, new column, dropped column, changed function body, new policy, dropped index, changed constraint) → --emit-migration → deploy v1+migration → catalog equivalence with v2 → migration's own verify passes → revert returns to v1; plus --verify oracle run and module-vs-.sql parity.

Note: pgpm/export has pre-existing failures on main (export-utils/export-meta/export-flow/cross-flow-parity, schema-prefix renames) unrelated to this PR — verified by running the same suite on a clean main checkout.

Link to Devin session: https://app.devin.ai/sessions/798445577ac8487abf80e28f9ff7a916
Requested by: @pyramation

@pyramation pyramation self-assigned this Jul 31, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@socket-security

socket-security Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Added@​pgsql/​scripts@​18.2.07710010090100

View full report

@pyramation
pyramation merged commit 45b9191 into main Aug 1, 2026
16 checks passed
@pyramation
pyramation deleted the feat/pgpm-diff branch August 1, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant