feat(pgpm): pgpm import — pgpm-itize an arbitrary SQL dump (Part A of #1340)

[pyramation]

Summary

Part A of constructive-planning#1340 (three-dial roadmap: constructive-planning#1329): a file/dump front door into the dials substrate. pgpm import turns a real pg_dump output (or a directory of .sql files, concatenated in sorted filename order) into a complete deployable pgpm module — module dir, .control, pgpm.plan, deploy/revert/verify trees with spec-derived paths, graph-derived requires, and generated revert/verify.

pgpm import <dump.sql | dir-of-sql-files> --pkg <module-name>
  [--granularity atomic|object|consolidated]   # default: object
  [--naming directory|flat] [--out <dir>] [--with-data]
  [--partition <config.json>] [--write] [--dry-run]

This branch merges two implementations: the initial line-regex sanitizing driver and the spec-complete driver, keeping the spec behavior (--pkg, default object granularity, COPY blocks as units, riders preserved and attached to hosts, misc/ fallback, directory input, --with-data) and folding in --partition support from the first.

Pipeline

@pgpmjs/export:

• dump-source.ts — loadDumpSource/preprocessDumpText: strips psql backslash commands (outside dollar-quoted bodies), extracts COPY ... FROM stdin blocks atomically through the terminating \., and copyBlockToInsert converts COPY rows to multi-row INSERTs (pgpm's deploy driver doesn't stream COPY stdin).
• import.ts — importDumpRows: classify (classifyStatements) → restructure through the shared restructureExportRows seam → rows. Dump-specific handling:
  • pg_dump preamble (SET ..., set_config(...)) skipped and counted; COMMENT ON/GRANT/ownership kept as riders attached to their host object; late ALTER TABLE ONLY ... ADD CONSTRAINT folds at object/consolidated.
  • CREATE EXTENSION → .control requires (pgpm owns extension creation).
  • Data (COPY/INSERT/setval) skipped by default with a warning; --with-data emits seed fixture changes with FK-ordered seed deps.
  • ALTER SEQUENCE ... OWNED BY table.col rides the table change (the sequence deploys before its table exists otherwise); serial nextval('seq') defaults get explicit sequence deps (plain-string refs the statement graph can't see); rows are topo-sorted so the plan deploys in order.
  • Unclassifiable statements land in a misc/statements change with a warning — never dropped silently. Summary printed: N statements → M changes, K warnings.
• linkTextualDeps — post-partition fixup: re-links rider/serial deps that only exist in statement text after partitionUnits regroups statements, pruning cycle-forming back-edges so partitioned plans stay deployable.

pgpm/cli:

• commands/import.ts — the command; --partition <config> reuses partitionExportRows (same config shape as pgpm transform/pgpm export --partition) to split the import into multiple packages with cross-package requires.
• utils/emit-package.ts — module emission (writePackage/writeControlFile/checkOverwrite) factored out of transform.ts and shared by import/transform, per the feat(pgpm): pgpm transform — re-dial an existing module/workspace; wire --partition into pgpm export #1569 convergence note.

Tests

• Unit (export): preamble filtering, COPY block parsing (incl. dollar-quoted bodies, unterminated COPY), COPY→INSERT NULL/escape handling, rider attachment, late-FK folding, extension requires, misc fallback, data skip/--with-data, seed FK ordering, collision paths, flat naming (32 tests).
• e2e (cli, live Postgres): committed representative pg_dump fixture (preamble, 2 schemas, FKs, serial + standalone sequences, function, trigger, RLS policy, grants, comments, extension, COPY data). For each granularity: import → deploy into a fresh DB → catalog equivalence with the source (snapshotCatalog/diffCatalogSnapshots) → pgpm verify → pgpm revert leaves the DB clean. Plus --dry-run, --with-data (seeds land, sequence state continues), and --partition (two packages, cross-package requires, deploys to an equivalent catalog).

Refs: constructive-planning#1340 (Part A), constructive-planning#1329; emit path shared with #1569 (pgpm transform).

Link to Devin session: https://app.devin.ai/sessions/f54bc87f17d748b6a143280e0ace5ba6
Requested by: @pyramation

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring