Migrated pgqueue to go-pg

[djthorpe]

This pull request introduces significant refactoring and reorganization to the pgqueue (task queue) component of the project. The main changes involve moving the pgqueue command-line application into its own top-level directory, restructuring the codebase to align with new conventions, and updating documentation and build/test scripts accordingly. Additionally, a new exec module is added to the pgqueue manager for handling task execution. Below are the most important changes grouped by theme:

1. Refactoring and Codebase Reorganization

• The pgqueue CLI application has been moved from cmd/_pgqueue to cmd/pgqueue, with its implementation now delegating to a new package structure under pgqueue/cmd. This includes deleting the old CLI implementation (cmd/_pgqueue/main.go, cmd/_pgqueue/server.go, cmd/_pgqueue/version.go) and introducing a new entrypoint (cmd/pgqueue/main.go) that uses a shared server command framework. [1] [2] [3] [4] [5]

• The server command logic for pgqueue is now encapsulated in pgqueue/cmd/server.go, utilizing a shared command and flag structure for consistency with other components.

2. New Features and Internal Improvements

• A new exec module (pgqueue/manager/exec.go) has been added to manage task registration, removal, and execution for both queue and ticker tasks, including OpenTelemetry tracing support and robust error handling.

3. Documentation and Build/Test Updates

• Documentation links in README.md have been updated to reflect the new locations for the PostgreSQL Manager and Task Queue modules, and new sections have been added for clarity. [1] [2]

• The Makefile has been updated to include the new pgqueue directory in unit and coverage tests, ensuring all relevant code is tested.

4. Minor Code Cleanups

• Minor import aliasing and ordering fixes in pgmanager/cmd/server.go and pgmanager/manager/opt.go for consistency and clarity. [1] [2]

[github-actions]

Coverage for this change: 12.0%

[Copilot]

Pull request overview

This PR migrates the legacy pkg/queue implementation into a new top-level pgqueue module, including a new manager runtime loop, schema/SQL rework (with partition support), and a new cmd/pgqueue entrypoint, while updating docs and build/test targets accordingly.

Changes:

• Replaced pkg/_queue (manager/schema/http handlers/http client/CLI) with the new pgqueue/{schema,manager,cmd,test} layout.
• Added partition-aware SQL and manager helpers (create/list/drop partitions) plus a new task/ticker execution registry (exec) and runtime loop.
• Updated documentation links and Makefile test/coverage targets to include pgqueue and the new docs locations.

Reviewed changes

Copilot reviewed 78 out of 82 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
README.md	Updates documentation links and adds a Task Queue section pointing at pgqueue/README.md.
Makefile	Includes ./pgqueue/... in unit/coverage test targets.
cmd/_pgqueue/main.go	Removes legacy pgqueue CLI entrypoint.
cmd/_pgqueue/server.go	Removes legacy pgqueue server implementation.
cmd/_pgqueue/version.go	Removes legacy version metadata helper.
cmd/pgqueue/main.go	Adds new pgqueue CLI entrypoint using shared server command framework.
pgmanager/cmd/server.go	Minor import alias cleanup (mcp alias).
pgmanager/manager/opt.go	Minor import ordering/alias cleanup.
pgqueue/cmd/_namespace.go	Adds (ignored) legacy-style CLI namespace commands scaffold for reference/testing.
pgqueue/cmd/_queue.go	Adds (ignored) legacy-style CLI queue commands scaffold for reference/testing.
pgqueue/cmd/_task.go	Adds (ignored) legacy-style CLI task commands scaffold for reference/testing.
pgqueue/cmd/_ticker.go	Adds (ignored) legacy-style CLI ticker commands scaffold for reference/testing.
pgqueue/cmd/server.go	Adds new pgqueue run server command wiring manager + server with errgroup.
pgqueue/manager/exec.go	Adds execution registry for queue/ticker callbacks with OTEL spans and panic recovery.
pgqueue/manager/exec_test.go	Adds tests for task registration/removal and exec behavior.
pgqueue/manager/manager.go	Adds new pgqueue manager bootstrap (schema creation, query registration, maintenance/cleanup tickers).
pgqueue/manager/metrics.go	Adds OTEL metrics callback for queue task counts.
pgqueue/manager/opt.go	Adds manager options (schema/worker/tracer/meter).
pgqueue/manager/partition.go	Adds partition management helpers (create next, drop drained, CRUD).
pgqueue/manager/partition_test.go	Adds tests for partitions and sequence behavior.
pgqueue/manager/queue.go	Adds queue CRUD + callback registration integrated with exec registry.
pgqueue/manager/queue_test.go	Adds tests for queue CRUD and queue/task execution behaviors (incl concurrency).
pgqueue/manager/run.go	Adds main run loop: LISTEN/NOTIFY handling, periodic polling/backoff, draining results, graceful shutdown.
pgqueue/manager/task.go	Adds task CRUD/retain/release methods for new schema.
pgqueue/manager/task_test.go	Adds task creation tests.
pgqueue/manager/test_test.go	Adds TestMain wiring for pgqueue manager tests.
pgqueue/manager/ticker.go	Adds ticker CRUD + callback registration integrated with exec registry.
pgqueue/manager/ticker_test.go	Adds tests for ticker CRUD and next-ticker behavior.
pgqueue/schema/partition.go	Adds schema types/selectors/writers for partitions + bounds parsing.
pgqueue/schema/queries.sql	Adds new named SQL queries for queues/tasks/tickers/stats/partitions.
pgqueue/schema/queue.go	Refactors queue schema types and binds to new SQL shape (adds concurrency).
pgqueue/schema/schema.go	Adds embedded objects.sql/queries.sql plus new defaults/constants.
pgqueue/schema/ticker.go	Refactors ticker schema types/validation and binds to new SQL shape (last_at).
pgqueue/test/test.go	Adds shared manager-backed integration test harness for pgqueue.
pkg/_queue/README.md	Removes legacy queue module documentation (replaced by pgqueue).
pkg/_queue/doc.go	Removes legacy package docs.
pkg/_queue/handler.go	Removes legacy handler interfaces.
pkg/_queue/httphandler/doc.go	Removes legacy HTTP handler docs.
pkg/_queue/httphandler/frontend_excluded.go	Removes legacy frontend fallback handler.
pkg/_queue/httphandler/httphandler.go	Removes legacy HTTP handler registration/router helpers.
pkg/_queue/httphandler/metrics.go	Removes legacy Prometheus metrics handler.
pkg/_queue/httphandler/metrics_test.go	Removes legacy metrics handler tests.
pkg/_queue/httphandler/namespace.go	Removes legacy namespace handler.
pkg/_queue/httphandler/namespace_test.go	Removes legacy namespace handler tests.
pkg/_queue/httphandler/queue.go	Removes legacy queue handler.
pkg/_queue/httphandler/task.go	Removes legacy task handler.
pkg/_queue/httphandler/task_test.go	Removes legacy task handler tests.
pkg/_queue/httphandler/ticker.go	Removes legacy ticker handler.
pkg/_queue/httpclient/client.go	Removes legacy HTTP client.
pkg/_queue/httpclient/client_test.go	Removes legacy HTTP client tests.
pkg/_queue/httpclient/doc.go	Removes legacy HTTP client docs.
pkg/_queue/httpclient/namespace.go	Removes legacy namespace client.
pkg/_queue/httpclient/namespace_test.go	Removes legacy namespace client tests.
pkg/_queue/httpclient/opts.go	Removes legacy client option helpers.
pkg/_queue/httpclient/opts_test.go	Removes legacy client option tests.
pkg/_queue/httpclient/queue.go	Removes legacy queue client.
pkg/_queue/httpclient/queue_test.go	Removes legacy queue client tests.
pkg/_queue/httpclient/task.go	Removes legacy task client.
pkg/_queue/httpclient/task_test.go	Removes legacy task client tests.
pkg/_queue/httpclient/ticker.go	Removes legacy ticker client.
pkg/_queue/manager.go	Removes legacy manager.
pkg/_queue/manager_test.go	Removes legacy manager tests.
pkg/_queue/namespace.go	Removes legacy namespace methods.
pkg/_queue/namespace_test.go	Removes legacy namespace tests.
pkg/_queue/opts.go	Removes legacy manager option helpers.
pkg/_queue/queue.go	Removes legacy queue CRUD methods.
pkg/_queue/run.go	Removes legacy runtime loop/worker pool orchestration.
pkg/_queue/schema/globals.go	Removes legacy schema globals and stringify helper.
pkg/_queue/schema/namespace.go	Removes legacy namespace schema.
pkg/_queue/sql/queries.sql	Removes legacy SQL queries (replaced by pgqueue/schema/queries.sql).
pkg/_queue/sql/sql.go	Removes legacy SQL embedding wrapper.
pkg/_queue/task.go	Removes legacy task CRUD/loop implementation.
pkg/_queue/ticker.go	Removes legacy ticker CRUD/loop implementation.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.