Skip to content

Commit 0b3f432

Browse files
maths22maths22
authored
Support replicating index creation (#3)
* Support replicating index creation (and use parsed queries for transformation) * Improved drop handling with pg_toast * Add older versions of deparse * Fix multi expected for old psql * Make PG 11 and 12 happy * Fix deb build * Update debian changelog
1 parent f175f52 commit 0b3f432

59 files changed

Lines changed: 94682 additions & 1499 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ jobs:
88
fail-fast: false
99
matrix:
1010
pg_version: ["11", "12", "13", "14", "15", "16", "17", "18"]
11-
from_version: ["2.2", "2.3"]
11+
from_version: ["2.3", "2.4"]
1212
steps:
1313
- uses: actions/checkout@v6
1414
- name: Remove default PostgreSQL

‎Makefile‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,19 @@ DATA = pgl_ddl_deploy--1.0.sql pgl_ddl_deploy--1.0--1.1.sql \
1010
pgl_ddl_deploy--2.0.sql pgl_ddl_deploy--2.0--2.1.sql \
1111
pgl_ddl_deploy--2.1.sql pgl_ddl_deploy--2.1--2.2.sql \
1212
pgl_ddl_deploy--2.2.sql pgl_ddl_deploy--2.2--2.3.sql \
13-
pgl_ddl_deploy--2.3.sql
14-
MODULES = pgl_ddl_deploy ddl_deparse
13+
pgl_ddl_deploy--2.3.sql pgl_ddl_deploy--2.3--2.4.sql \
14+
pgl_ddl_deploy--2.4.sql
15+
MODULES = ddl_deparse
16+
MODULE_big = pgl_ddl_deploy
17+
OBJS = postgres_deparse.11.o \
18+
postgres_deparse.12.o \
19+
postgres_deparse.13.o \
20+
postgres_deparse.14.o \
21+
postgres_deparse.15.o \
22+
postgres_deparse.16.o \
23+
postgres_deparse.17.o \
24+
postgres_deparse.18.o \
25+
pgl_ddl_deploy.o
1526

1627
REGRESS := 01_create_ext 02_setup 03_add_configs 04_deploy 04_deploy_update \
1728
05_allowed 06_multi 07_edges 08_ignored \
@@ -52,7 +63,8 @@ REGRESS := 01_create_ext 02_setup 03_add_configs 04_deploy 04_deploy_update \
5263
54_new_setup \
5364
55_raise_message \
5465
56_1_6_features \
55-
57_native_features
66+
57_2_4_features \
67+
99_native_features
5668
PG_CONFIG = pg_config
5769
PGXS := $(shell $(PG_CONFIG) --pgxs)
5870
include $(PGXS)

‎README.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,9 +39,10 @@ https://innovation.enova.com/pursuing-postgres-ddl-replication/
3939

4040
# <a name="release_notes"></a>Release Notes
4141

42-
### Release 2.3
42+
### Release 2.4
4343
Summary of changes:
44-
* Support for Postgres 17 & 18
44+
* Support for replicating index DDL
45+
* Prevent DDL that internally touches toast tables from being silently dropped
4546

4647
### Release 2.2
4748
Summary of changes:
@@ -356,6 +357,11 @@ SQL statement with a single node `parsetree`) will be eligible for propagation.
356357
be maintained by DDL replication. Thus only `ALTER TABLE`
357358
statements are permitted here. This option is incompatible with
358359
`include_schema_regex`.
360+
- `include_indexes`: if true, will replicate `CREATE INDEX`/`ALTER INDEX`/`DROP INDEX`
361+
statements. This can be undesirable if the replica is intended to have different
362+
configuration from the primary. Also, indexes cannot be created with `CREATE INDEX CONCURRENTLY`
363+
and will therefore potentially block operations on the table, which can be problematic
364+
if the replica is in active read usage.
359365
- `queue_subscriber_failures`: if true, DDL will be allowed to fail on subscriber
360366
without breaking replication, and queued for retry using function
361367
`pgl_ddl_deploy.retry_all_subscriber_logs()`. This is useful for example if you

‎ddl_deparse.c‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,7 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS)
104104
strtype = "ADD OIDS";
105105
break;
106106
case AT_AddOidsRecurse:
107-
strtype = "ADD OIDS (and recurse)";
107+
strtype = "ADD OIDS";
108108
break;
109109
#endif
110110
#if PG_VERSION_NUM >= 120000 && PG_VERSION_NUM < 180000
@@ -119,19 +119,19 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS)
119119
#endif
120120
#if PG_VERSION_NUM < 160000
121121
case AT_AddColumnRecurse:
122-
strtype = "ADD COLUMN (and recurse)";
122+
strtype = "ADD COLUMN";
123123
break;
124124
case AT_DropColumnRecurse:
125-
strtype = "DROP COLUMN (and recurse)";
125+
strtype = "DROP COLUMN";
126126
break;
127127
case AT_AddConstraintRecurse:
128-
strtype = "ADD CONSTRAINT (and recurse)";
128+
strtype = "ADD CONSTRAINT";
129129
break;
130130
case AT_ValidateConstraintRecurse:
131-
strtype = "VALIDATE CONSTRAINT (and recurse)";
131+
strtype = "VALIDATE CONSTRAINT";
132132
break;
133133
case AT_DropConstraintRecurse:
134-
strtype = "DROP CONSTRAINT (and recurse)";
134+
strtype = "DROP CONSTRAINT";
135135
break;
136136
#endif
137137
#if PG_VERSION_NUM >= 170000

‎debian/changelog‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,10 @@
1+
pgl-ddl-deploy (2.4.0-1) unstable; urgency=medium
2+
3+
* Support for replicating index DDL
4+
* Prevent DDL that internally touches toast tables from being silently dropped
5+
6+
-- Jacob Burroughs <jburroughs@instructure.com> Wed, 17 Jun 2026 13:12:22 -0500
7+
18
pgl-ddl-deploy (2.3.0-1) unstable; urgency=medium
29

310
* Support for Postgres 16

‎expected/05_allowed.out‎

Lines changed: 264 additions & 264 deletions
Large diffs are not rendered by default.

‎expected/06_multi.out‎

Lines changed: 36 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -19,18 +19,18 @@ WARNING: Unhandled deployment logged in pgl_ddl_deploy.unhandled
1919
WARNING: Unhandled deployment logged in pgl_ddl_deploy.unhandled
2020
DROP TABLE
2121
SELECT set_name, ddl_sql_raw, ddl_sql_sent FROM pgl_ddl_deploy.events ORDER BY id DESC LIMIT 10;
22-
set_name | ddl_sql_raw | ddl_sql_sent
23-
----------+-----------------------------+-----------------------------
24-
test8 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
25-
test7 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
26-
test6 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
27-
test5 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
28-
test4 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
29-
test3 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
30-
test2 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
31-
test1 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
32-
test8 | DROP SCHEMA foobar CASCADE; | DROP SCHEMA foobar CASCADE;
33-
test7 | DROP SCHEMA foobar CASCADE; | DROP SCHEMA foobar CASCADE;
22+
set_name | ddl_sql_raw | ddl_sql_sent
23+
----------+---------------------------+---------------------------
24+
test8 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
25+
test7 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
26+
test6 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
27+
test5 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
28+
test4 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
29+
test3 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
30+
test2 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
31+
test1 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
32+
test8 | DROP SEQUENCE foobar.foo; | DROP SEQUENCE foobar.foo;
33+
test7 | DROP SEQUENCE foobar.foo; | DROP SEQUENCE foobar.foo;
3434
(10 rows)
3535

3636
SELECT set_name, ddl_sql_raw, command_tag, reason FROM pgl_ddl_deploy.unhandled ORDER BY id DESC LIMIT 10;
@@ -60,14 +60,14 @@ WARNING: Unhandled deployment logged in pgl_ddl_deploy.unhandled
6060
WARNING: Unhandled deployment logged in pgl_ddl_deploy.unhandled
6161
COMMIT
6262
SELECT set_name, ddl_sql_raw, ddl_sql_sent FROM pgl_ddl_deploy.events ORDER BY id DESC LIMIT 10;
63-
set_name | ddl_sql_raw | ddl_sql_sent
64-
----------+-------------------------------------------------------------+------------------------------------------------
65-
test7 | BEGIN; CREATE TABLE foobar.foo(id int primary key); COMMIT; | CREATE TABLE foobar.foo(id int primary key);
66-
test5 | BEGIN; CREATE TABLE foobar.foo(id int primary key); COMMIT; | CREATE TABLE foobar.foo(id int primary key);
67-
test3 | BEGIN; CREATE TABLE foobar.foo(id int primary key); COMMIT; | CREATE TABLE foobar.foo(id int primary key);
68-
test1 | BEGIN; CREATE TABLE foobar.foo(id int primary key); COMMIT; | CREATE TABLE foobar.foo(id int primary key);
69-
test3 | BEGIN; CREATE TABLE foo(id int primary key); COMMIT; | CREATE TABLE foo(id int primary key);
70-
test1 | BEGIN; CREATE TABLE foo(id int primary key); COMMIT; | CREATE TABLE foo(id int primary key);
63+
set_name | ddl_sql_raw | ddl_sql_sent
64+
----------+-------------------------------------------------------------+-----------------------------------------------
65+
test7 | BEGIN; CREATE TABLE foobar.foo(id int primary key); COMMIT; | CREATE TABLE foobar.foo (id int PRIMARY KEY);
66+
test5 | BEGIN; CREATE TABLE foobar.foo(id int primary key); COMMIT; | CREATE TABLE foobar.foo (id int PRIMARY KEY);
67+
test3 | BEGIN; CREATE TABLE foobar.foo(id int primary key); COMMIT; | CREATE TABLE foobar.foo (id int PRIMARY KEY);
68+
test1 | BEGIN; CREATE TABLE foobar.foo(id int primary key); COMMIT; | CREATE TABLE foobar.foo (id int PRIMARY KEY);
69+
test3 | BEGIN; CREATE TABLE foo(id int primary key); COMMIT; | CREATE TABLE foo (id int PRIMARY KEY);
70+
test1 | BEGIN; CREATE TABLE foo(id int primary key); COMMIT; | CREATE TABLE foo (id int PRIMARY KEY);
7171
test8 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
7272
test7 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
7373
test6 | CREATE SCHEMA foobar; | CREATE SCHEMA foobar;
@@ -106,14 +106,14 @@ WARNING: Unhandled deployment logged in pgl_ddl_deploy.unhandled
106106
WARNING: Unhandled deployment logged in pgl_ddl_deploy.unhandled
107107
DROP TABLE
108108
SELECT set_name, ddl_sql_raw, ddl_sql_sent FROM pgl_ddl_deploy.events ORDER BY id DESC LIMIT 10;
109-
set_name | ddl_sql_raw | ddl_sql_sent
110-
----------+-----------------------------------------------------------------------------+-----------------------------------------------------------------------------
111-
test7 | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE; | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE;
112-
test5 | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE; | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE;
113-
test3 | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE; | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE;
114-
test1 | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE; | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE;
115-
test3 | CREATE TABLE foo(id int primary key); DROP TABLE foo CASCADE; | CREATE TABLE foo(id int primary key); DROP TABLE foo CASCADE;
116-
test1 | CREATE TABLE foo(id int primary key); DROP TABLE foo CASCADE; | CREATE TABLE foo(id int primary key); DROP TABLE foo CASCADE;
109+
set_name | ddl_sql_raw | ddl_sql_sent
110+
----------+-----------------------------------------------------------------------------+------------------------------------------------------------------------------
111+
test7 | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE; | CREATE TABLE foobar.foo (id int PRIMARY KEY); DROP TABLE foobar.foo CASCADE;
112+
test5 | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE; | CREATE TABLE foobar.foo (id int PRIMARY KEY); DROP TABLE foobar.foo CASCADE;
113+
test3 | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE; | CREATE TABLE foobar.foo (id int PRIMARY KEY); DROP TABLE foobar.foo CASCADE;
114+
test1 | CREATE TABLE foobar.foo(id int primary key); DROP TABLE foobar.foo CASCADE; | CREATE TABLE foobar.foo (id int PRIMARY KEY); DROP TABLE foobar.foo CASCADE;
115+
test3 | CREATE TABLE foo(id int primary key); DROP TABLE foo CASCADE; | CREATE TABLE foo (id int PRIMARY KEY); DROP TABLE foo CASCADE;
116+
test1 | CREATE TABLE foo(id int primary key); DROP TABLE foo CASCADE; | CREATE TABLE foo (id int PRIMARY KEY); DROP TABLE foo CASCADE;
117117
test8 | DROP TABLE foobar.foo CASCADE; | DROP TABLE foobar.foo CASCADE;
118118
test7 | DROP TABLE foobar.foo CASCADE; | DROP TABLE foobar.foo CASCADE;
119119
test6 | DROP TABLE foobar.foo CASCADE; | DROP TABLE foobar.foo CASCADE;
@@ -147,18 +147,18 @@ WARNING: Unhandled deployment logged in pgl_ddl_deploy.unhandled
147147
WARNING: Unhandled deployment logged in pgl_ddl_deploy.unhandled
148148
DROP TABLE
149149
SELECT set_name, ddl_sql_raw, ddl_sql_sent FROM pgl_ddl_deploy.events ORDER BY id DESC LIMIT 10;
150-
set_name | ddl_sql_raw | ddl_sql_sent
151-
----------+----------------------------------------------+----------------------------------------------
150+
set_name | ddl_sql_raw | ddl_sql_sent
151+
----------+----------------------------------------------+-----------------------------------------------
152152
test4 | DROP TABLE foo, foobar.foo CASCADE; | DROP TABLE foo, foobar.foo CASCADE;
153153
test3 | DROP TABLE foo, foobar.foo CASCADE; | DROP TABLE foo, foobar.foo CASCADE;
154154
test2 | DROP TABLE foo, foobar.foo CASCADE; | DROP TABLE foo, foobar.foo CASCADE;
155155
test1 | DROP TABLE foo, foobar.foo CASCADE; | DROP TABLE foo, foobar.foo CASCADE;
156-
test8 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo(id int primary key);
157-
test7 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo(id int primary key);
158-
test6 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo(id int primary key);
159-
test5 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo(id int primary key);
160-
test4 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo(id int primary key);
161-
test3 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo(id int primary key);
156+
test8 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo (id int PRIMARY KEY);
157+
test7 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo (id int PRIMARY KEY);
158+
test6 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo (id int PRIMARY KEY);
159+
test5 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo (id int PRIMARY KEY);
160+
test4 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo (id int PRIMARY KEY);
161+
test3 | CREATE TABLE foobar.foo(id int primary key); | CREATE TABLE foobar.foo (id int PRIMARY KEY);
162162
(10 rows)
163163

164164
SELECT set_name, ddl_sql_raw, command_tag, reason FROM pgl_ddl_deploy.unhandled ORDER BY id DESC LIMIT 10;

0 commit comments

Comments
 (0)