Add pg_mooncake.enable_bgworker GUC to control bgworker registration#2
Open
NotHimmel wants to merge 6 commits into
Open
Add pg_mooncake.enable_bgworker GUC to control bgworker registration#2NotHimmel wants to merge 6 commits into
NotHimmel wants to merge 6 commits into
Conversation
- Dockerfile + .dockerignore + docker-compose.yml: build pg_mooncake on the IvorySQL 5.3 UBI8 base, bundling pg_duckdb + libduckdb, with shared_preload_libraries + duckdb GUCs wired into postgresql.conf.sample. - scripts/docker-build.sh: end-to-end helper to install docker, build the image, and smoke-test CREATE EXTENSION + mooncake.create_table. - pg_mooncake.control: rebrand comment for IvorySQL Oracle-compat distro. - .github/workflows/docker.yml: build/publish image on push/tag, PR smoke. - .github/workflows/regression.yml: PR/manual cargo pgrx regress inside the Dockerfile build stage; passes shared_preload_libraries via --postgresql-conf so pg_duckdb loads. - .gitignore: exclude developer-local docs/, alt-base Dockerfile, and the data-relocate helper that is machine-specific. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Postmaster-context bool GUC gating whether the moonlink background worker is registered at PG startup. When 'off', pg_mooncake.so still loads (DDL functions remain) but no bgworker spawns. Default 'on' preserves existing behaviour. Use cases: - Run pg_mooncake.so for the schema/function bridge only, without paying the cost of moonlink_service when mirror tables aren't used. - Disable bgworker quickly when troubleshooting stuck mirror state, without recompiling without the 'bgworker' Cargo feature or removing pg_mooncake from shared_preload_libraries. - Single switch downstream packagers can toggle. The GUC is Postmaster context because RegisterBackgroundWorker is only legal during postmaster startup (shared_preload_libraries load). The check runs immediately before BackgroundWorkerBuilder::load() and skips registration entirely when the GUC is off; no register_failed warning on subsequent LOAD attempts. Verified end-to-end on IvorySQL 5.3 (PG 18 base): - default (on): bgworker process appears, moonlink.sock created - pg_mooncake.enable_bgworker=off: bgworker absent, pg_duckdb-only queries still work, SHOW reports off - toggle back to on across PG restarts respects new value Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds /usr/local/bin/ivy-apply-env.sh + ivy-entrypoint-shim.sh and a
/docker-entrypoint-initdb.d/00-ivy-apply-env.sh hook. Together they
idempotently apply IVY_* environment variables as PG GUC settings on
every container start; first-time initdb is handled by the init.d hook
so subsequent restarts and first launches behave the same.
First env -> GUC mapping:
IVY_MOONCAKE_ENABLE_BGWORKER -> pg_mooncake.enable_bgworker
To add a new tunable, edit ivy-apply-env.sh and add another
'[ -n "${IVY_FOO:-}" ] && apply foo.bar "${IVY_FOO}"' line; no
Dockerfile or rebuild logic change needed.
Usage:
docker run -e IVY_MOONCAKE_ENABLE_BGWORKER=off ivy_mooncake:5.3-ubi8
Verified end-to-end:
- default (no env): pg_mooncake.enable_bgworker=on (compiled default)
- IVY_MOONCAKE_ENABLE_BGWORKER=off: conf populated, bgworker absent
- toggle on/off across container restarts on shared PGDATA volume:
postgresql.conf is overwritten (sed -i removes prior key) and the
new value takes effect on the next PG startup
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Returns a JSON object with three liveness signals: - guc_enabled: pg_mooncake.enable_bgworker setting value - socket_exists: whether $PGDATA/pg_mooncake/moonlink.sock exists - socket_listening: probe result of UnixStream::connect() to the socket Disambiguates "SHOW pg_mooncake.enable_bgworker = on" but mirror operations still fail: socket may exist as a stale leftover from a prior bgworker run that has since panicked, leaving 'socket_exists=true socket_listening=false' — the exact state behind the "mooncake_duckdb_cpp_init ... Connection refused (os error 111)" error. The probe runs in a worker thread with a 250ms hard ceiling so a hung socket cannot block the calling PG backend. Adds serde_json = "1" dependency. Verified end-to-end: - default (GUC on, bgworker healthy): all three true - GUC=off + restart: guc_enabled=false, socket_listening=false - toggle back to on: all true Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnose and force-clean orphan moonlink_slot_* replication slots and the moonlink_pub publication left behind when a mooncake table or extension is dropped without the bgworker cleaning up (bgworker dead, crash, or prior install). Complements the automatic DROP EXTENSION cleanup for the paths it cannot reach: retroactive orphans, DROP TABLE orphans, and bgworker-recreate race residue. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a disowned (non-member) sql_drop event trigger that fires on DROP EXTENSION pg_mooncake, drops leftover moonlink_slot_* replication slots and the moonlink_pub publication in the current database (best-effort, WARNING on failure), then self-removes. A member sql_drop trigger does not fire for the command that drops its own extension, so the trigger is created and immediately disowned via ALTER EXTENSION ... DROP. Cleanup is pure SQL and does not depend on the moonlink bgworker. Add tests/pg_regress/zz_drop_cleanup (sorts last, no recreate) verifying the trigger installs, is a non-member, fires on drop, and self-removes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dockerize ivy_mooncake for IvorySQL and add CI regression workflow
Add pg_mooncake.enable_bgworker GUC to control bgworker registration
docker: translate IVY_* env vars to postgresql.conf via entrypoint shim