Fix postgres18 new volume path in docker images by masterashu · Pull Request #4280 · PeerDB-io/peerdb

masterashu · 2026-05-06T11:15:14Z

No description provided.

masterashu · 2026-05-06T11:17:08Z

This got created, with the recent postgres image bump in #4245.
The new version fails to start with following error

Error: in 18+, these Docker images are configured to store database data in a
format which is compatible with "pg_ctlcluster" (specifically, using
major-version-specific directory names). This better reflects how
PostgreSQL itself works, and how upgrades are to be performed.
See also docker-library/postgres#1259
Counter to that, there appears to be PostgreSQL data in:
/var/lib/postgresql/data (unused mount/volume)
This is usually the result of upgrading the Docker image without
upgrading the underlying database using "pg_upgrade" (which requires both
versions).
The suggested container configuration for 18+ is to place a single mount
at /var/lib/postgresql which will then place PostgreSQL data in a
subdirectory, allowing usage of "pg_upgrade --link" without mount point
boundary issues.
See docker-library/postgres#37 for a (long)
discussion around this process, and suggestions for how to do so.

This update fixes it

claude · 2026-05-06T11:18:05Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

claude · 2026-05-06T11:19:37Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

codecov · 2026-05-06T11:31:07Z

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
2229	2	2227	203

View the top 3 failed test(s) by shortest run time

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg

Stack Traces | 0.01s run time

=== RUN   TestApiPg
=== PAUSE TestApiPg
=== CONT  TestApiPg
--- FAIL: TestApiPg (0.01s)
2026/05/06 11:24:19 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestTableAdditionWithoutInitialLoad

Stack Traces | 0.13s run time

=== RUN   TestApiPg/TestTableAdditionWithoutInitialLoad
=== PAUSE TestApiPg/TestTableAdditionWithoutInitialLoad
=== CONT  TestApiPg/TestTableAdditionWithoutInitialLoad
2026/05/06 11:30:01 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_yh7r3kjk.\"qrepapi_api_yh7r3kjk\" ORDER BY id"
2026/05/06 11:30:01 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_yh7r3kjk.\"qrepapi_api_yh7r3kjk\" ORDER BY id"
2026/05/06 11:30:01 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_4955886156147576516 CURSOR FOR SELECT id,val FROM e2e_test_api_yh7r3kjk.\"qrepapi_api_yh7r3kjk\" ORDER BY id" args=[]
2026/05/06 11:30:01 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_yh7r3kjk.\"qrepapi_api_yh7r3kjk\" ORDER BY id" channelLen=0
2026/05/06 11:30:01 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4955886156147576516
2026/05/06 11:30:01 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4955886156147576516 records=2 bytes=19 channelLen=1
2026/05/06 11:30:01 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_yh7r3kjk.\"qrepapi_api_yh7r3kjk\" ORDER BY id" rows=2 bytes=19 channelLen=1
2026/05/06 11:30:01 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4955886156147576516
2026/05/06 11:30:01 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4955886156147576516 records=0 bytes=0 channelLen=0
2026/05/06 11:30:01 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_yh7r3kjk.\"qrepapi_api_yh7r3kjk\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/05/06 11:30:01 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/05/06 11:30:01 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_yh7r3kjk.\"qrepapi_api_yh7r3kjk\" ORDER BY id" rows=2 bytes=19 channelLen=0
2026/05/06 11:30:01 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2498: 
        	Error Trace:	.../flow/e2e/api_test.go:2498
        	Error:      	Received unexpected error:
        	            	failed to get workflow_id for flow added_tables_no_initial_load_api_v8hct65m: FATAL: terminating connection due to administrator command (SQLSTATE 57P01)
        	Test:       	TestApiPg/TestTableAdditionWithoutInitialLoad
    api_test.go:49: begin tearing down postgres schema api_v8hct65m
--- FAIL: TestApiPg/TestTableAdditionWithoutInitialLoad (0.13s)

github.com/PeerDB-io/peerdb/flow/e2e::TestApiPg/TestQRep

Stack Traces | 63.2s run time

=== RUN   TestApiPg/TestQRep
=== PAUSE TestApiPg/TestQRep
=== CONT  TestApiPg/TestQRep
2026/05/06 11:22:39 INFO Received AWS credentials from peer for connector: ci x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
2026/05/06 11:22:39 INFO Received AWS credentials from peer for connector: clickhouse x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN}
    api_test.go:2371: WaitFor qrep initial load 2026-05-06 11:22:39.698217018 +0000 UTC m=+146.993478959
2026/05/06 11:22:39 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id"
2026/05/06 11:22:39 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id"
2026/05/06 11:22:39 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_10340312727400484931 CURSOR FOR SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" args=[]
2026/05/06 11:22:39 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" channelLen=0
2026/05/06 11:22:39 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10340312727400484931
2026/05/06 11:22:39 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10340312727400484931 records=1 bytes=9 channelLen=0
2026/05/06 11:22:39 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" rows=1 bytes=9 channelLen=0
2026/05/06 11:22:39 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10340312727400484931
2026/05/06 11:22:39 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_10340312727400484931 records=0 bytes=0 channelLen=0
2026/05/06 11:22:39 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/05/06 11:22:39 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/05/06 11:22:39 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" rows=1 bytes=9 channelLen=0
    api_test.go:2371: code: 60, message: Unknown table expression identifier 'qrepapi_api_rewnqlaj' in scope SELECT id, val FROM qrepapi_api_rewnqlaj FINAL WHERE _peerdb_is_deleted = 0 ORDER BY 1 ASC SETTINGS use_query_cache = false
2026/05/06 11:22:39 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column,lost_column FROM e2e_test_bq_h592k526_20260506112159.\"test_lost_column_bug\" ORDER BY id"
2026/05/06 11:22:39 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column,lost_column FROM e2e_test_bq_h592k526_20260506112159.\"test_lost_column_bug\" ORDER BY id"
2026/05/06 11:22:39 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_15844419252775875505 CURSOR FOR SELECT id,good_column,lost_column FROM e2e_test_bq_h592k526_20260506112159.\"test_lost_column_bug\" ORDER BY id" args=[]
2026/05/06 11:22:39 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column,lost_column FROM e2e_test_bq_h592k526_20260506112159.\"test_lost_column_bug\" ORDER BY id" channelLen=0
2026/05/06 11:22:39 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_15844419252775875505
2026/05/06 11:22:39 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_15844419252775875505 records=2 bytes=23 channelLen=1
2026/05/06 11:22:39 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column,lost_column FROM e2e_test_bq_h592k526_20260506112159.\"test_lost_column_bug\" ORDER BY id" rows=2 bytes=23 channelLen=1
2026/05/06 11:22:39 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_15844419252775875505
2026/05/06 11:22:39 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_15844419252775875505 records=0 bytes=0 channelLen=0
2026/05/06 11:22:39 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column,lost_column FROM e2e_test_bq_h592k526_20260506112159.\"test_lost_column_bug\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/05/06 11:22:39 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/05/06 11:22:39 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,good_column,lost_column FROM e2e_test_bq_h592k526_20260506112159.\"test_lost_column_bug\" ORDER BY id" rows=2 bytes=23 channelLen=0
    api_test.go:2376: WaitFor insert post qrep initial load 2026-05-06 11:22:40.718999959 +0000 UTC m=+148.014261910
2026/05/06 11:22:40 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id"
2026/05/06 11:22:40 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id"
2026/05/06 11:22:40 INFO [pg_query_executor] declared cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursorQuery="DECLARE peerdb_cursor_4637257258015212399 CURSOR FOR SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" args=[]
2026/05/06 11:22:40 INFO [pg_query_executor] fetching rows start x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" channelLen=0
2026/05/06 11:22:40 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4637257258015212399
2026/05/06 11:22:40 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4637257258015212399 records=2 bytes=19 channelLen=1
2026/05/06 11:22:40 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" rows=2 bytes=19 channelLen=1
2026/05/06 11:22:40 INFO [pg_query_executor] fetching from cursor x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4637257258015212399
2026/05/06 11:22:40 INFO processed row stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart cursor=peerdb_cursor_4637257258015212399 records=0 bytes=0 channelLen=0
2026/05/06 11:22:40 INFO [pg_query_executor] fetched rows x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" rows=0 bytes=0 channelLen=0
2026/05/06 11:22:40 INFO [pg_query_executor] committing transaction x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart
2026/05/06 11:22:40 INFO [pg_query_executor] committed transaction for query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id,val FROM e2e_test_api_rewnqlaj.\"qrepapi_api_rewnqlaj\" ORDER BY id" rows=2 bytes=19 channelLen=0
    api_test.go:2376: q.NumRecords: 2
    api_test.go:2376: other.NumRecords: 1
2026/05/06 11:22:40 INFO Executing and processing query x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id, col_to_add FROM e2e_test_bq_2qydxmhq_20260506112159.\"test_ddl_drop_column\" ORDER BY id"
2026/05/06 11:22:40 INFO Executing and processing query stream x-peerdb-additional-metadata={Operation:FLOW_OPERATION_UNKNOWN} partitionId=testpart query="SELECT id, col_to_add FROM e2e_test_bq_2qydxmhq_20260506112159.\"test_ddl_drop_column\" ORDER BY id"
    api_test.go:2376: q.NumRecords: 2
    api_test.go:2376: other.NumRecords: 1
    api_test.go:2395: WaitFor finish 2026-05-06 11:22:42.744643234 +0000 UTC m=+150.039905185
    api_test.go:2395: UNEXPECTED TIMEOUT finish 2026-05-06 11:23:42.871884241 +0000 UTC m=+210.167146192
    api_test.go:49: begin tearing down postgres schema api_rewnqlaj
--- FAIL: TestApiPg/TestQRep (63.24s)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

github-actions · 2026-05-06T11:32:48Z

🔄 Flaky Test Detected

Analysis: The test failed due to a PostgreSQL connection being terminated by an administrator command (SQLSTATE 57P01) during an active test, and a teardown race condition where a replication slot was still held by another process (PID 15018) when cleanup tried to drop it — both are transient infrastructure issues, not code bugs.
Confidence: 0.92

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-05-06T11:35:10Z

🔄 Flaky Test Detected

Analysis: Test failed with SQLSTATE 57P01 ("terminating connection due to administrator command"), meaning the PostgreSQL server was abruptly shut down mid-test by the CI environment — a clear infrastructure flake, not a code regression.
Confidence: 0.93

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-05-06T11:50:29Z

🔄 Flaky Test Detected

Analysis: TestGenericBQ/Test_Partitioned_Table_Without_Publish_Via_Partition_Root timed out after 185s waiting for 13 rows to normalize into BigQuery — source Postgres consistently showed 10 rows but BigQuery destination returned 0 throughout, indicating an async replication lag under CI load rather than a code regression.
Confidence: 0.92

✅ Automatically retrying the workflow

View workflow run

github-actions · 2026-05-06T12:07:57Z

🔄 Flaky Test Detected

Analysis: The test TestCancelTableAddition_WithRemoval failed due to a PostgreSQL catalog connection being forcibly terminated by an administrator command (SQLSTATE 57P01 / admin_shutdown), which is a transient CI infrastructure issue unrelated to the code changes.
Confidence: 0.92

✅ Automatically retrying the workflow

View workflow run

pfcoperez

I got bitten by this in my local dev env! Thanks for fixing 👏

masterashu added 2 commits May 6, 2026 16:44

Fix postgres18 new volume path in docker images

393a630

Fix postgres18 new volume path in docker images

2eb00ff

masterashu requested a review from pfcoperez May 6, 2026 11:17

masterashu requested a review from ilidemi May 6, 2026 11:29

masterashu requested a review from serprex May 6, 2026 11:44

pfcoperez approved these changes May 6, 2026

View reviewed changes

masterashu merged commit f090ccc into main May 6, 2026
15 of 17 checks passed

masterashu deleted the masterashu-patch-1 branch May 6, 2026 13:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix postgres18 new volume path in docker images#4280

Fix postgres18 new volume path in docker images#4280
masterashu merged 2 commits into
mainfrom
masterashu-patch-1

masterashu commented May 6, 2026

Uh oh!

masterashu commented May 6, 2026

Uh oh!

claude Bot commented May 6, 2026

Uh oh!

claude Bot commented May 6, 2026

Uh oh!

codecov Bot commented May 6, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented May 6, 2026

Uh oh!

github-actions Bot commented May 6, 2026

Uh oh!

github-actions Bot commented May 6, 2026

Uh oh!

github-actions Bot commented May 6, 2026

Uh oh!

pfcoperez left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

masterashu commented May 6, 2026

Uh oh!

masterashu commented May 6, 2026

Uh oh!

claude Bot commented May 6, 2026

Code review

Uh oh!

claude Bot commented May 6, 2026

Code review

Uh oh!

codecov Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 2 Tests Failed:

Uh oh!

github-actions Bot commented May 6, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented May 6, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented May 6, 2026

🔄 Flaky Test Detected

Uh oh!

github-actions Bot commented May 6, 2026

🔄 Flaky Test Detected

Uh oh!

pfcoperez left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov Bot commented May 6, 2026 •

edited

Loading