Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 190 additions & 0 deletions rules/pgfence-postgres-migration-safety.mdc
Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
---
description: Postgres migration safety rules. Apply when generating, reviewing, or editing schema migrations and DDL such as ALTER TABLE, CREATE INDEX, ADD CONSTRAINT, DROP TABLE, ALTER TYPE. Helps prevent migrations that lock production tables, rewrite large tables under ACCESS EXCLUSIVE, or break replication.
globs: ["**/migrations/**/*.sql", "**/migrations/**/migration.sql", "**/migrations/**/*.ts", "**/migrations/**/*.js", "**/db/migrate/**/*.rb", "**/db/migrate/**/*.sql"]
alwaysApply: false
---
# Postgres migration safety

Rules for writing online-safe Postgres migrations. Sourced from the pgfence
analyzer (https://pgfence.com). Run `npx @flvmnt/pgfence analyze <files>` for
the authoritative verdict on any specific migration.

## Always prepend to every migration

```sql
SET lock_timeout = '2s';
SET statement_timeout = '5min';
SET idle_in_transaction_session_timeout = '30s';
SET application_name = 'migrate:<short-name>';
```

`lock_timeout` is the single most important safety knob: without it, a blocked
ALTER queues behind running transactions and blocks every later query on the
table.

## Lock mode reference

| Lock mode | What it blocks |
|---|---|
| ACCESS SHARE | Nothing |
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Correct the lock conflict table

When agents use this as the lock-mode reference for production DDL, this row understates conflicts: an ACCESS SHARE lock held by ordinary reads conflicts with ACCESS EXCLUSIVE, so it does block operations that need that lock rather than blocking “Nothing”. Because the table is meant to guide migration safety decisions, this should either use PostgreSQL's actual conflict matrix or avoid the inaccurate shorthand.

Useful? React with 👍 / 👎.

| ROW SHARE | ROW EXCLUSIVE+ |
| ROW EXCLUSIVE | SHARE+ |
| SHARE UPDATE EXCLUSIVE | SHARE UPDATE EXCLUSIVE+ |
| SHARE | Writes (reads allowed) |
| SHARE ROW EXCLUSIVE | Most writes |
| EXCLUSIVE | Everything except ACCESS SHARE |
| ACCESS EXCLUSIVE | Everything, reads included |
Comment on lines +27 to +36
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

PostgreSQL explicit locking conflict matrix for table-level lock modes (ACCESS SHARE, ROW SHARE, ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE). Please provide the official docs table and a concise conflict summary for each mode.

💡 Result:

In PostgreSQL, all explicit table-level locks are governed by a conflict matrix that defines which modes can be held concurrently on the same table. The official documentation (Table 13.2) provides the definitive compatibility matrix [1][2]. PostgreSQL Table-Level Lock Conflict Matrix The following table shows the conflicts between lock modes. An 'X' indicates that the two lock modes conflict. | Requested Lock | AS | RS | RE | SUE | S | SRE | E | AE | |:--- |:---: |:---: |:---: |:---: |:---: |:---: |:---: |:---: | | ACCESS SHARE (AS) | | | | | | | | X | | ROW SHARE (RS) | | | | | | | X | X | | ROW EXCLUSIVE (RE) | | | | | X | X | X | X | | SHARE UPDATE EXCL. (SUE) | | | | X | X | X | X | X | | SHARE (S) | | | X | X | | X | X | X | | SHARE ROW EXCL. (SRE) | | | X | X | X | X | X | X | | EXCLUSIVE (E) | | X | X | X | X | X | X | X | | ACCESS EXCLUSIVE (AE) | X | X | X | X | X | X | X | X | Note: A transaction never conflicts with itself [1][2]. Concise Summary of Lock Modes 1. ACCESS SHARE: Conflicts only with ACCESS EXCLUSIVE. Acquired by SELECT statements. 2. ROW SHARE: Conflicts with EXCLUSIVE and ACCESS EXCLUSIVE. Acquired by SELECT FOR UPDATE and SELECT FOR SHARE. 3. ROW EXCLUSIVE: Conflicts with SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE. Acquired by INSERT, UPDATE, and DELETE. 4. SHARE UPDATE EXCLUSIVE: Conflicts with itself, SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE. Acquired by VACUUM, ANALYZE, and CREATE INDEX CONCURRENTLY. 5. SHARE: Conflicts with ROW EXCLUSIVE, SHARE UPDATE EXCLUSIVE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE. Acquired by CREATE INDEX (non-concurrent). 6. SHARE ROW EXCLUSIVE: Conflicts with all modes except ACCESS SHARE. Acquired by various DDL commands like CREATE TRIGGER. 7. EXCLUSIVE: Conflicts with all modes except ACCESS SHARE. Acquired by REFRESH MATERIALIZED VIEW CONCURRENTLY. 8. ACCESS EXCLUSIVE: Conflicts with all lock modes (including itself). Acquired by DROP, TRUNCATE, REINDEX, VACUUM FULL, and LOCK TABLE. All listed modes are table-level locks, even if the name contains the word row; these names are historical [1][2][3]. Non-conflicting lock modes can be held concurrently by many transactions [1][2].

Citations:


Fix lock conflict matrix for table-level locks (lines 27-36) to match PostgreSQL compatibility rules.

Several entries don’t reflect PostgreSQL’s table-lock conflict matrix and can lead to incorrect migration-safety gating:

  • ACCESS SHARE: conflicts only with ACCESS EXCLUSIVE (not “Nothing”).
  • ROW SHARE: conflicts only with EXCLUSIVE and ACCESS EXCLUSIVE (not ROW EXCLUSIVE+).
  • ROW EXCLUSIVE: conflicts with SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS EXCLUSIVE (but is compatible with SHARE UPDATE EXCLUSIVE).
  • SHARE / SHARE ROW EXCLUSIVE: the current “writes” / “most writes” wording is inaccurate; SHARE ROW EXCLUSIVE conflicts with all modes except ACCESS SHARE.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rules/pgfence-postgres-migration-safety.mdc` around lines 27 - 36, The lock
conflict matrix rows for table-level locks are incorrect; update the table
entries for ACCESS SHARE, ROW SHARE, ROW EXCLUSIVE, SHARE, and SHARE ROW
EXCLUSIVE to match PostgreSQL semantics: change ACCESS SHARE to indicate it
conflicts only with ACCESS EXCLUSIVE; change ROW SHARE to indicate it conflicts
with EXCLUSIVE and ACCESS EXCLUSIVE only; change ROW EXCLUSIVE to indicate it
conflicts with SHARE, SHARE ROW EXCLUSIVE, EXCLUSIVE, and ACCESS EXCLUSIVE but
is compatible with SHARE UPDATE EXCLUSIVE; replace the vague “writes”/“most
writes” descriptions for SHARE and SHARE ROW EXCLUSIVE with explicit conflict
sets (SHARE conflicts with ROW EXCLUSIVE, SHARE ROW EXCLUSIVE, EXCLUSIVE, ACCESS
EXCLUSIVE; SHARE ROW EXCLUSIVE conflicts with everything except ACCESS SHARE) so
the table enumerates precise conflicting modes by name.


## DDL footguns (refuse to emit these patterns without the safe rewrite)

### Table rewrites under ACCESS EXCLUSIVE

- `ALTER COLUMN ... TYPE` (most narrowing type changes rewrite the table)
- `ADD COLUMN ... DEFAULT <volatile>` such as `DEFAULT now()` or `DEFAULT uuid_generate_v4()`
- `ADD COLUMN ... NOT NULL` without a `DEFAULT` on a non-empty table
- `VACUUM FULL`, `CLUSTER`
- `ALTER TABLE ... SET LOGGED` / `SET UNLOGGED`

Safe rewrite: expand + backfill + contract. Add the column nullable, backfill
in batches with `FOR UPDATE SKIP LOCKED`, then enforce NOT NULL via
`CHECK ... NOT VALID` + `VALIDATE CONSTRAINT`.

Some widening changes are metadata-only (for example `varchar(50)` to
`varchar(255)`, or `varchar` to `text`). Confirm with `pgfence explain` before
rewriting every type change as expand/contract.

### Constraints

Most `ADD CONSTRAINT` forms take ACCESS EXCLUSIVE on the target table. Foreign
keys are the exception and take SHARE ROW EXCLUSIVE on both tables.

Always prefer:

```sql
ALTER TABLE t ADD CONSTRAINT c ... NOT VALID;
ALTER TABLE t VALIDATE CONSTRAINT c;
```

The `NOT VALID` step takes the brief ACCESS EXCLUSIVE; the validation scan
runs under SHARE UPDATE EXCLUSIVE and does not block writes.

For unique constraints:

```sql
CREATE UNIQUE INDEX CONCURRENTLY uq_t_col ON t (col);
ALTER TABLE t ADD CONSTRAINT uq_t_col UNIQUE USING INDEX uq_t_col;
```

### Indexes

- Never emit `CREATE INDEX` without `CONCURRENTLY` on a non-empty table.
- `DROP INDEX CONCURRENTLY` over `DROP INDEX`.
- `REINDEX INDEX CONCURRENTLY` (PG12+) over `REINDEX INDEX`.

`CREATE INDEX CONCURRENTLY` cannot run inside a transaction. If the migration
tool wraps every file in `BEGIN/COMMIT`, opt out for these statements.

### Production footguns that often slip through review

- `CLUSTER table USING idx`: full table rewrite under ACCESS EXCLUSIVE. Use
`pg_repack` instead.
- `ALTER TABLE t REPLICA IDENTITY FULL`: every UPDATE/DELETE writes the full
old row image to WAL. 10x to 100x WAL amplification. Saturates Debezium and
pglogical. Use the primary key (default) or a unique non-null index.
- `ALTER TABLE t ENABLE ROW LEVEL SECURITY` without prior policies: affected
non-owner roles see no rows. The application may appear to lose its data.
- `ALTER TABLE t DISABLE ROW LEVEL SECURITY`: silently exposes every row
previously gated by policy.
- `DROP SCHEMA s CASCADE`: drops every table, view, function, type, and
sequence in the schema. Irreversible.
- `DROP DATABASE`: irreversible. Belongs in an ops runbook, never a migration.
- `CREATE TYPE x AS ENUM (...)`: Postgres has no `ALTER TYPE x DROP VALUE`.
If a value may ever need to be removed, use a lookup table or a `CHECK`
constraint instead.

### Enum changes

- `ALTER TYPE ... ADD VALUE` on PG12+: instant, EXCLUSIVE on the type only.
Safe.
- `ALTER TYPE ... DROP VALUE`: does not exist in Postgres. Plan accordingly.

### Refresh materialized view

- `REFRESH MATERIALIZED VIEW CONCURRENTLY`: EXCLUSIVE on the matview. Allows
reads, blocks writes. Requires a unique index on the matview.
- `REFRESH MATERIALIZED VIEW` (non-concurrent): ACCESS EXCLUSIVE. Blocks reads
and writes.

### Destructive

- `DROP TABLE`, `TRUNCATE`: ACCESS EXCLUSIVE. Schedule a separate release.
- `DELETE` with no `WHERE` (or `WHERE TRUE`): writes a row version for every
row and generates massive WAL. Batch instead.
- `DROP COLUMN`: ACCESS EXCLUSIVE. The data is not actually removed; the
column is hidden. Reads and writes block while metadata updates.

## Safe rewrite recipes

### Add a NOT NULL column with a default

```sql
-- Migration 1: nullable, fast metadata-only on PG11+
ALTER TABLE t ADD COLUMN IF NOT EXISTS col text;

-- Migration 2: batched backfill (separate file, no transaction wrapper)
DO $$
Comment on lines +134 to +135
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Split the backfill into separate transactions

When this recipe is used on a large table, the DO block runs the entire loop as one transaction even if the migration tool is not wrapping the file, so every row lock/dead tuple and all WAL are retained until the loop finishes and a timeout rolls back the whole backfill. That defeats the stated batched-backfill safety; this should be shown as one committed batch per invocation/statement or a procedure that explicitly commits between batches.

Useful? React with 👍 / 👎.

DECLARE updated int := 1;
BEGIN
WHILE updated > 0 LOOP
WITH batch AS (
SELECT ctid FROM t
WHERE col IS NULL
LIMIT 1000
FOR UPDATE SKIP LOCKED
)
UPDATE t SET col = '<default>'
FROM batch
WHERE t.ctid = batch.ctid;
GET DIAGNOSTICS updated = ROW_COUNT;
END LOOP;
END $$;

-- Migration 3: validated NOT NULL
ALTER TABLE t ADD CONSTRAINT t_col_nn CHECK (col IS NOT NULL) NOT VALID;
ALTER TABLE t VALIDATE CONSTRAINT t_col_nn;
ALTER TABLE t ALTER COLUMN col SET NOT NULL;
ALTER TABLE t DROP CONSTRAINT t_col_nn;
```

### Add a foreign key

```sql
ALTER TABLE child ADD CONSTRAINT fk_child_parent
FOREIGN KEY (parent_id) REFERENCES parent (id) NOT VALID;
ALTER TABLE child VALIDATE CONSTRAINT fk_child_parent;
```

### Add a unique constraint

```sql
CREATE UNIQUE INDEX CONCURRENTLY IF NOT EXISTS uq_t_col ON t (col);
ALTER TABLE t ADD CONSTRAINT uq_t_col UNIQUE USING INDEX uq_t_col;
```

### Create an index

```sql
CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_t_col ON t (col);
```

Outside any transaction wrapper.

## When in doubt

```bash
npx @flvmnt/pgfence analyze migrations/*.sql
npx @flvmnt/pgfence explain "ALTER TABLE t ADD COLUMN x int NOT NULL"
```

Returns the lock mode, what it blocks, and the safe rewrite recipe for any
DDL statement.
Loading