|
| 1 | +# Packaging: restructuring and separating pgpm packages |
| 2 | + |
| 3 | +How pgpm derives modular packages from one canonical substrate: statement |
| 4 | +facts and a typed dependency graph. Consult this when asked to split a |
| 5 | +database into multiple pgpm packages, cherry-pick objects into a package, |
| 6 | +restructure change granularity, or understand how change paths and |
| 7 | +`requires` are derived. |
| 8 | + |
| 9 | +Everything here lives in `@pgpmjs/transform` (`pgpm/transform`), with paths |
| 10 | +rendered by `@pgpmjs/naming-spec` (`pgpm/naming-spec`). The underlying facts, |
| 11 | +graph, and identity primitives come from `@pgsql/transform` (npm, from the |
| 12 | +pgsql-parser repo). Tracking issue: constructive-planning#1329. |
| 13 | + |
| 14 | +## The substrate: facts, graph, identity |
| 15 | + |
| 16 | +One extractor and one edge taxonomy feed every projection: |
| 17 | + |
| 18 | +```ts |
| 19 | +import { classifyStatements, buildStatementGraph, identityOf } from '@pgsql/transform'; |
| 20 | + |
| 21 | +const facts = classifyStatements(sql); // per-statement: kind, creates, references, |
| 22 | + // bodyReferences, fkTargets, roles, dynamicSql |
| 23 | +const graph = buildStatementGraph(facts); // typed edges (hard | fk | late), producer |
| 24 | + // index, SCC components, stable topo order |
| 25 | +const identity = identityOf(facts[i]); // (kind, schema, name, table?) or null |
| 26 | +``` |
| 27 | + |
| 28 | +- `hard` — must exist at creation time; `fk` — foreign-key target; `late` — |
| 29 | + body reference resolved at run time (does not constrain deploy order). |
| 30 | +- `identityOf` returns `null` for statements that create nothing (grants, |
| 31 | + comments) — they ride with the object they attach to. |
| 32 | +- `@pgpmjs/slice` consumes this same substrate (its `extractSqlFacts` and |
| 33 | + object graph are thin adapters over it). |
| 34 | + |
| 35 | +## Derived paths (naming spec) |
| 36 | + |
| 37 | +Paths are projections of identity, never authored and never identity: |
| 38 | + |
| 39 | +```ts |
| 40 | +import { pathFor } from '@pgpmjs/naming-spec'; |
| 41 | + |
| 42 | +pathFor({ kind: 'function', schema: 'app', name: 'user_count' }); |
| 43 | +// 'schemas/app/procedures/user_count/procedure' (directory style, default) |
| 44 | +// flat style drops trailing kind tokens: 'schemas/app/procedures/user_count' |
| 45 | +``` |
| 46 | + |
| 47 | +Re-alterations of the same object get `<parent>/alterations/alt0000000042` |
| 48 | +via `alterationPathFor` (mirrors `db_deps.next_alteration` in |
| 49 | +constructive-db). |
| 50 | + |
| 51 | +## Granularity: restructure change shape |
| 52 | + |
| 53 | +`restructureChanges(changes, { granularity })` flattens a module's deploy |
| 54 | +scripts, restructures the SQL, and re-slices into one change per object with |
| 55 | +dependencies recomputed from the statement graph: |
| 56 | + |
| 57 | +- `atomic` — bare CREATE TABLE plus one ALTER per column/constraint (the |
| 58 | + machine-emitted shape). |
| 59 | +- `object` — each table fully baked; cross-object statements (FKs, indexes, |
| 60 | + triggers, policies) stay separate. |
| 61 | +- `consolidated` — additionally inlines FKs proven safe by the graph. |
| 62 | + |
| 63 | +Wired into the CLI: `pgpm export --granularity consolidated` (see |
| 64 | +`pgpm/export/src/restructure.ts`). Requires `loadModule()` from |
| 65 | +`plpgsql-parser` before use. |
| 66 | + |
| 67 | +## Package separation: partitionUnits |
| 68 | + |
| 69 | +`partitionUnits(sqlOrChanges, config)` projects one deploy surface into a |
| 70 | +set of packages. Membership is a declarative config; SQL is never |
| 71 | +re-authored: |
| 72 | + |
| 73 | +```ts |
| 74 | +import { partitionUnits } from '@pgpmjs/transform'; |
| 75 | + |
| 76 | +const result = partitionUnits(sql, { |
| 77 | + rules: [ |
| 78 | + // a whole schema (and everything in it) |
| 79 | + { package: 'pkg-billing', select: [{ schema: 'billing' }], closure: true }, |
| 80 | + // one specific table |
| 81 | + { package: 'pkg-users', select: [{ kind: 'table', schema: 'app', name: 'users' }] }, |
| 82 | + // specific procedures by name |
| 83 | + { package: 'pkg-fns', select: [{ kind: 'function', schema: 'app', name: ['user_count', 'signup'] }] }, |
| 84 | + // security surface: all policies plus split-out grants |
| 85 | + { package: 'pkg-security', select: [{ kind: 'policy' }, { statementKind: 'grant' }] }, |
| 86 | + // exact cherry-pick by derived path |
| 87 | + { package: 'pkg-x', select: [{ path: 'schemas/app/tables/users/table' }] } |
| 88 | + ], |
| 89 | + defaultPackage: 'pkg-app', |
| 90 | + splitRiders: ['grant'] |
| 91 | +}); |
| 92 | +// → { packages, assignments, closureIncluded, warnings } |
| 93 | +``` |
| 94 | + |
| 95 | +Selector semantics: |
| 96 | + |
| 97 | +- Fields AND within a selector; arrays OR within a field; selectors OR |
| 98 | + within a rule. Rules are tried in order; first match wins; unmatched units |
| 99 | + land in `defaultPackage`. |
| 100 | +- Selectable kinds: `schema`, `table`, `view`, `sequence`, `type`, |
| 101 | + `function`, `index`, `trigger`, `policy`, `constraint`, `seed_dml` — |
| 102 | + anything `identityOf` can name. Table-scoped kinds also match on `table`. |
| 103 | +- `path` is the canonical single-object cherry-pick (paths are unique). |
| 104 | + |
| 105 | +Behavior: |
| 106 | + |
| 107 | +- **Riders** (grants, comments) ride with the object they attach to by |
| 108 | + default. `splitRiders: ['grant']` peels each grant into its own selectable |
| 109 | + unit named `<hostPath>/grants/<role>`, with an explicit dependency on its |
| 110 | + host — so the security surface can ship as its own package. |
| 111 | +- **`closure: true`** drags the rule's transitive dependency closure |
| 112 | + (hard + fk edges) out of the default package, but never steals units |
| 113 | + another rule explicitly claimed — those become cross-package deps. |
| 114 | +- **Requires are derived**: same-package deps render as `requires: <path>`, |
| 115 | + cross-package as `requires: <pkg>:<path>`; each `PartitionedPackage` also |
| 116 | + reports package-level `requires`. Objects require their schema's change; |
| 117 | + table-scoped objects require their table's change. |
| 118 | +- **Cross-package cycles are a hard error** (`PartitionCycleError` lists the |
| 119 | + package cycle and one offending edge per hop). Re-draw the boundaries or |
| 120 | + merge the packages. |
| 121 | +- Units executing dynamic SQL produce warnings — their edges are incomplete, |
| 122 | + so the graph alone cannot prove the assignment safe. |
| 123 | + |
| 124 | +The emit shape (`PartitionedPackage { name, changes, requires }`) is |
| 125 | +structurally typed on the change seam — no `@pgpmjs/bundle` or |
| 126 | +`@pgpmjs/core` dependency; callers materialize packages (write `pgpm.plan` + |
| 127 | +deploy trees, feed a bundle, ...). |
| 128 | + |
| 129 | +## Related |
| 130 | + |
| 131 | +- `references/pgpm-export.md` — exporting a live database to pgpm packages |
| 132 | +- `pgpm-migration-bundle` skill — the portable bundle artifact; bundle |
| 133 | + transpile/split operate on the same module seams |
| 134 | +- `@pgpmjs/slice` (`pgpm/slice`) — subsystem contract/cascade analysis over |
| 135 | + the same graph substrate |
0 commit comments