Backport snapshot/restore fixes to v1.1.x (#906, #908, #909, #907, #910)#966
Merged
Conversation
## Problem Databases created on/upgraded through older PostgreSQL versions can carry leftover `public.plpgsql_call_handler()` and `public.plpgsql_validator(oid)` C-language function definitions that point at `$libdir/plpgsql`. These are normally provided by the `plpgsql` extension itself; restoring them as ordinary user functions on the target conflicts/errors during a snapshot restore. ## Fix `parseDump` now detects and skips these two legacy handler function definitions while preserving all genuine user-defined functions. ## Testing - Unit: `TestSnapshotGenerator_parseDumpSkipsLegacyPublicPLPGSQLHandlers` (confirms a normal `keep_me()` function survives) - Integration: `Test_SnapshotToPostgres_SkipsLegacyPLPGSQLHandlers` - `go test ./pkg/snapshot/generator/postgres/schema/pgdumprestore/...` --- Cherry-picked from [@danddanddand](https://github.com/danddanddand/pgstream) (`fix/skip-legacy-plpgsql-handlers`). --------- Co-authored-by: dand <daniel.dent@borderconnect.com> (cherry picked from commit 2cf967b)
## Problem pgstream dumps schema with materialized views created `WITH NO DATA`, and matview contents are not part of the data copy. As a result, restored materialized views were empty on the target. Additionally, a unique index on a matview cannot be built until the matview has data, and dependent matviews must be refreshed in dependency order. ## Fix - Track materialized views while parsing the schema dump. - Keep matview indexes with the view restore section (not the general index section). - Emit `REFRESH MATERIALIZED VIEW ... WITH DATA` statements (preserving `\connect` context and creation order) and run them after table data has been copied. ## Testing - Unit: `TestSnapshotGenerator_parseDumpMovesMaterializedViewIndexesToViews`, `...RefreshesMaterializedViewsInCreationOrder`, `...CreateSnapshotRefreshesMaterializedViewsAfterViews` - Integration: `Test_SnapshotToPostgres_MaterializedViewRefresh` - `go test ./pkg/snapshot/generator/postgres/schema/pgdumprestore/...` --- Cherry-picked from [@danddanddand](https://github.com/danddanddand/pgstream) (`fix/refresh-materialized-views`). --- **Update:** the refresh is now **opt-in** and disabled by default (refreshing can be expensive on large views). Enable it with: - YAML: `source.postgres.snapshot.schema.pgdump_pgrestore.refresh_materialized_views: true` - Env: `PGSTREAM_POSTGRES_SNAPSHOT_REFRESH_MATERIALIZED_VIEWS=true` --------- Co-authored-by: dand <daniel.dent@borderconnect.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> (cherry picked from commit 5f1d6e3)
) ## Problem The CTID page-range data snapshot uses `SELECT * FROM <table> WHERE ctid BETWEEN ...` and `SELECT MAX(ctid) FROM <table>`. For a legacy-inheritance parent table (`CREATE TABLE ... INHERITS`), these implicitly span the parent **plus all child heaps**, so CTIDs aren't unique to the parent's storage: page ranges overlap and child rows get copied through the parent (and again when the child is snapshotted on its own). ## Fix Add `ONLY` to the page-range and max-CTID queries so each snapshot is confined to the selected table's own heap, matching pgstream's per-table snapshot model. Note: this is a no-op for plain tables and leaf partitions, and does not affect declaratively partitioned parents (`relkind='p'`), which are already excluded from this path by the `relkind='r'` filter in `getTableInfo`. ## Testing - Unit: `TestSnapshotQueriesDoNotIncludeInheritedRows` - Integration: `Test_SnapshotToPostgres_SelectedParentTableDoesNotCopyInheritedRows` - `go test ./pkg/snapshot/generator/postgres/data/...` --- Cherry-picked from [@danddanddand](https://github.com/danddanddand/pgstream) (`fix/snapshot-only-table-pages`). --------- Co-authored-by: dand <daniel.dent@borderconnect.com> (cherry picked from commit bce8ea8)
## Problem `ALTER TABLE ... CLUSTER ON <index>` references an index. During snapshot restore it could be emitted before the index was created, causing the restore to fail. ## Fix Route `ALTER TABLE ... CLUSTER ON ...` statements into the indices/constraints restore phase so the referenced index always exists first. ## Testing - Unit: `TestSnapshotGenerator_parseDumpMovesClusterOnToConstraints` (asserts CLUSTER ON is ordered after CREATE INDEX) - Integration: `Test_SnapshotToPostgres_ClusteredIndex` - `go test ./pkg/snapshot/generator/postgres/schema/pgdumprestore/...` --- Cherry-picked from [@danddanddand](https://github.com/danddanddand/pgstream) (`fix/restore-cluster-after-indexes`). --------- Co-authored-by: dand <daniel.dent@borderconnect.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> (cherry picked from commit 1b090aa)
## Problem When an `int4range`/`int8range` value arrives as `pgtype.Range[any]`, `pgx` cannot encode it to the binary wire format for the target column (the bounds are untyped `any`), so inserts and bulk COPY fail. ## Fix Normalize `int4range` and `int8range` values from `pgtype.Range[any]` into typed `pgtype.Range[int32]` / `pgtype.Range[int64]` before insert/COPY handling (`tstzrange` already had handling). --- Cherry-picked from [@danddanddand](https://github.com/danddanddand/pgstream) (`fix/type-integer-range-values`). Co-authored-by: dand <daniel.dent@borderconnect.com> (cherry picked from commit 6b3bb8a)
|
Merging this branch will increase overall coverage
Coverage by fileChanged files (no unit tests)
Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code. Changed unit test files
|
tsg
approved these changes
Jul 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backports the following snapshot/restore fixes from
mainto thev1.1.xrelease branch.ONLYfor CTID snapshot queries to exclude inherited child rowsCLUSTER ONstatements after their referenced indexesint4range/int8rangevalues before insert/COPY