feat(tracker-core): add PostgreSQL driver (subissue #1723) by josecelano · Pull Request #1724 · torrust/torrust-tracker

josecelano · 2026-05-01T08:13:15Z

PostgreSQL Driver for Torrust Tracker

Implements subissue 1525-08 of the persistence overhaul (#1525): add a PostgreSQL database driver to complement the existing SQLite3 and MySQL backends.

Spec: docs/issues/1723-1525-08-add-postgresql-driver.md

Closes #1723.

Summary

Adds full PostgreSQL support across the tracker's persistence layer:

Configuration package (new Driver::PostgreSQL variant)
Internal driver implementation with connection pooling and schema migrations
Automated schema migrations using sqlx::migrate!()
Driver tests with testcontainers
Container deployment configs and entry script support
E2E and benchmark runner integrations
CI compatibility matrix

Tasks

Task 1 — Add Driver::PostgreSQL to configuration package
Task 2 — Add Driver::PostgreSQL variant to internal driver enum and factory
Task 3 — Implement PostgreSQL driver (packages/tracker-core/src/databases/driver/postgres/mod.rs)
Task 4 — Add PostgreSQL migration files
Task 5 — Extend Cargo.toml with postgres feature and implement driver tests
Task 6 — Extend persistence benchmark runner with PostgreSQL support
Task 7 — Add --db-driver flag and PostgreSQL support to qBittorrent E2E runner
Task 8 — Extend .github/workflows/testing.yaml with PostgreSQL compatibility matrix
Task 9 — Add container configs, update entry script, and rename/add compose files

Status

Specification written and reviewed
User answers recorded
Implementation plan finalized
Code readiness analysis complete
Implementation in progress

…tation summary

Implements a full PostgreSQL driver for the tracker, covering: - Driver::PostgreSQL variant in configuration and tracker-core - Four SQL migration files under migrations/postgresql/ - SchemaMigrator, AuthKeyStore, TorrentMetricsStore and WhitelistStore trait implementations using $1/$2 placeholders and ON CONFLICT UPSERT - Factory wiring in setup.rs - Benchmark runner support (postgres.rs + BenchmarkResource::Postgres) - Container entry script support for postgresql driver - Default container config tracker.container.postgresql.toml Closes torrust#1723 Part of torrust#1525

codecov · 2026-05-01T08:46:26Z

Codecov Report

❌ Patch coverage is 9.06801% with 361 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.73%. Comparing base (4cb8853) to head (453dd48).
⚠️ Report is 23 commits behind head on develop.

Files with missing lines	Patch %	Lines
.../tracker-core/src/databases/driver/postgres/mod.rs	3.79%	152 Missing ⚠️
...stence_benchmark/driver_bench/database/postgres.rs	0.00%	51 Missing ⚠️
...re/src/databases/driver/postgres/auth_key_store.rs	0.00%	43 Missing ⚠️
...databases/driver/postgres/torrent_metrics_store.rs	0.00%	38 Missing ⚠️
...e/src/databases/driver/postgres/whitelist_store.rs	0.00%	26 Missing ⚠️
src/console/ci/qbittorrent_e2e/runner.rs	0.00%	18 Missing ⚠️
...nsole/ci/qbittorrent_e2e/tracker/config_builder.rs	0.00%	18 Missing ⚠️
...e/src/databases/driver/postgres/schema_migrator.rs	0.00%	6 Missing ⚠️
packages/tracker-core/src/databases/setup.rs	0.00%	4 Missing ⚠️
packages/tracker-core/src/databases/driver/mod.rs	0.00%	3 Missing ⚠️
... and 1 more

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1724      +/-   ##
===========================================
- Coverage    80.77%   79.73%   -1.05%     
===========================================
  Files          348      354       +6     
  Lines        25208    25597     +389     
  Branches     25208    25597     +389     
===========================================
+ Hits         20362    20410      +48     
- Misses        4616     4958     +342     
+ Partials       230      229       -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Rename compose.qbittorrent-e2e.yaml to compose.qbittorrent-e2e.sqlite3.yaml - Add compose.qbittorrent-e2e.mysql.yaml for MySQL backend E2E tests - Add compose.qbittorrent-e2e.postgresql.yaml for PostgreSQL backend E2E tests - Add --db-driver CLI argument to qBittorrent E2E runner for backend selection - Select compose file and generate tracker config based on chosen backend - Add MySQL and PostgreSQL qBittorrent E2E CI steps to testing.yaml - Update SQLite CI step to use renamed compose file path - Update qbt debug scripts and README to use renamed compose file

…nner

…line

Move database-compatibility-mysql and database-compatibility-postgres jobs from testing.yaml into a dedicated db-compatibility.yaml workflow. Motivation: the compat jobs only exercise the tracker-core package. Keeping them in a separate file makes the workflow portable if tracker-core is ever extracted to its own repository, and reduces the size of testing.yaml. The e2e job in testing.yaml now depends only on unit (the cross-workflow gate via database-compatibility was a soft ordering preference, not a hard requirement).

Add a new db-benchmarking.yaml workflow that runs the persistence_benchmark_runner binary against all three drivers on every push and pull request. Each job uses --ops 10 to keep runtime short while still exercising the full binary path: argument parsing, testcontainers startup, schema creation, query execution, and JSON report emission. A non-zero exit code (panic, missing container, broken SQL) fails the job immediately, catching runtime regressions that compilation alone cannot detect.

…rking

Move the cross-platform build job from testing.yaml into a new os-compatibility.yaml workflow. The build job has no dependency on any other job and is solely concerned with compilation portability across Ubuntu, macOS, and Windows. Keeping it separate follows the same principle applied to db-compatibility and db-benchmarking: one workflow, one concern. Also adds the os-compatibility badge to README.md.

Copilot

Pull request overview

Adds PostgreSQL as a first-class persistence backend for bittorrent-tracker-core, wiring it through configuration, driver setup, migrations, CI compatibility testing, qBittorrent E2E harness, and the persistence benchmark runner.

Changes:

Introduces a new async sqlx-backed PostgreSQL driver with embedded migrations and compatibility tests.
Extends configuration/factory wiring and adds PostgreSQL migration history aligned with existing backends.
Updates CI and qBittorrent E2E runner to support selecting sqlite3/mysql/postgresql, plus adds backend-specific compose files and container defaults.

Reviewed changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
`src/console/ci/qbittorrent_e2e/tracker/mod.rs`	Re-exports the new E2E-level `DatabaseDriver` type for runner integration.
`src/console/ci/qbittorrent_e2e/tracker/config_builder.rs`	Adds backend selection and default DSNs/paths for SQLite/MySQL/PostgreSQL E2E runs.
`src/console/ci/qbittorrent_e2e/runner.rs`	Adds `--db-driver` flag and backend-specific default compose file selection.
`src/console/ci/qbittorrent_e2e/mod.rs`	Updates module docs to include PostgreSQL as an E2E backend option.
`src/bin/qbittorrent_e2e_runner.rs`	Updates runner documentation for backend-specific compose stacks and flags.
`share/default/config/tracker.container.postgresql.toml`	Adds default container config template for PostgreSQL.
`share/container/entry_script_sh`	Adds `postgresql` branch to select the new default container config.
`packages/tracker-core/src/databases/setup.rs`	Wires PostgreSQL into database initialization and store construction.
`packages/tracker-core/src/databases/driver/postgres/mod.rs`	Implements PostgreSQL driver core + embedded migrator + compatibility tests.
`packages/tracker-core/src/databases/driver/postgres/schema_migrator.rs`	Implements migration apply/drop behavior for PostgreSQL.
`packages/tracker-core/src/databases/driver/postgres/torrent_metrics_store.rs`	Implements torrent metrics persistence for PostgreSQL.
`packages/tracker-core/src/databases/driver/postgres/whitelist_store.rs`	Implements whitelist persistence for PostgreSQL.
`packages/tracker-core/src/databases/driver/postgres/auth_key_store.rs`	Implements authentication key persistence for PostgreSQL.
`packages/tracker-core/src/databases/driver/mod.rs`	Adds `Driver::PostgreSQL` plus parsing/stringification and module export.
`packages/tracker-core/migrations/postgresql/20240730183000_torrust_tracker_create_all_tables.sql`	Adds PostgreSQL baseline schema migration.
`packages/tracker-core/migrations/postgresql/20240730183500_torrust_tracker_keys_valid_until_nullable.sql`	Aligns PostgreSQL `keys.valid_until` nullability with the migration history.
`packages/tracker-core/migrations/postgresql/20250527093000_torrust_tracker_new_torrent_aggregate_metrics_table.sql`	Adds PostgreSQL migration for aggregate metrics table.
`packages/tracker-core/migrations/postgresql/20260409120000_torrust_tracker_widen_download_counters.sql`	Widens PostgreSQL download counters to `BIGINT`.
`packages/tracker-core/Cargo.toml`	Enables `sqlx` postgres feature for the core crate.
`packages/configuration/src/v2_0_0/database.rs`	Adds `Driver::PostgreSQL`, documents DSN format, and masks PostgreSQL passwords.
`packages/tracker-core/src/bin/persistence_benchmark/reporting.rs`	Updates report metadata normalization to include PostgreSQL runs.
`packages/tracker-core/src/bin/persistence_benchmark/driver_bench/database/mod.rs`	Adds PostgreSQL benchmark resource wiring and cleanup.
`packages/tracker-core/src/bin/persistence_benchmark/driver_bench/database/postgres.rs`	Adds PostgreSQL testcontainer-based benchmark DB initialization.
`docs/issues/1723-1525-08-add-postgresql-driver.md`	Adds/updates the PostgreSQL driver subissue spec and implementation notes.
`docs/issues/1721-1525-07-align-rust-and-db-types.md`	Fixes the spec link to the renamed PostgreSQL subissue doc.
`docs/issues/1525-overhaul-persistence.md`	Updates the EPIC doc to point to the renamed PostgreSQL subissue spec.
`contrib/dev-tools/debugging/qbt/check-qbittorrent-e2e-compose.sh`	Updates the debug script to the new SQLite compose filename.
`contrib/dev-tools/debugging/qbt/README.md`	Updates debugging docs to reference the new compose filename.
`compose.qbittorrent-e2e.sqlite3.yaml`	Adds dedicated SQLite qBittorrent E2E compose stack.
`compose.qbittorrent-e2e.mysql.yaml`	Adds dedicated MySQL qBittorrent E2E compose stack.
`compose.qbittorrent-e2e.postgresql.yaml`	Adds dedicated PostgreSQL qBittorrent E2E compose stack.
`.github/workflows/testing.yaml`	Adds PostgreSQL compatibility job and runs qBittorrent E2E for all three backends.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Replace the four-job chain (format -> check -> unit -> e2e) with two flat jobs: test-nightly and test-stable. Each job runs checkout, toolchain setup, Node.js, Rust cache, dependency fetch, linter install, and tool install once, then executes all stages sequentially. Motivation: the previous structure recompiled the entire workspace multiple times per push across format, check, unit, and e2e jobs, each starting from a cold runner. Collapsing into two jobs eliminates the redundant compilations while keeping nightly and stable coverage separate. A cargo fetch step is added after cache restore in both jobs to make dependency download time visible in the job timeline. Differences from the old structure: - fmt check runs only under nightly (rustfmt nightly is the canonical formatter for this repo) - qBittorrent E2E tests (all three backends) run only under stable, matching the previous if: matrix.toolchain == 'stable' guard - test-nightly timeout: 45 min; test-stable timeout: 90 min

Move 10 closed issue spec files from docs/issues/ to the new docs/issues/closed/ buffer folder: torrust#523 internal-linting-tool torrust#1697 ai-agent-configuration torrust#1703 1525-01-persistence-test-coverage torrust#1706 1525-02-qbittorrent-e2e torrust#1710 1525-03-persistence-benchmarking torrust#1713 1525-04-split-persistence-traits torrust#1715 1525-04b-migrate-consumers-to-narrow-traits torrust#1717 1525-05-migrate-sqlite-and-mysql-to-sqlx torrust#1719 1525-06-introduce-schema-migrations torrust#1721 1525-07-align-rust-and-db-types Add docs/issues/closed/README.md explaining the two-stage lifecycle (archive then delete) for closed spec files. Update the cleanup-completed-issues skill (v1.1) to reflect the new process: move to closed/ first, delete permanently only when no longer referenced by active work.

Three issues flagged by the Copilot reviewer on PR torrust#1724: 1. Add PostgreSQL 14 to the db-compatibility CI matrix (.github/workflows/db-compatibility.yaml). The issue spec (1723-1525-08) lists 14 15 16 17 as the required version set; only 15-17 were present. 2. Add unit test for Driver::PostgreSQL in persistence_benchmark/reporting.rs. build_report handles PostgreSQL in the same match arm as MySQL but had no test coverage, leaving report-metadata regressions undetected. 3. Align create_legacy_pre_v4_schema to use BIGINT NOT NULL for keys.valid_until. Migration 1 creates the column as BIGINT NOT NULL; the helper used INTEGER NOT NULL, causing a type mismatch after migrator idempotency runs because there is no migration that widens that column.

Replace duplicated test-nightly and test-stable jobs with a single matrix-driven test job. Preserve behavior with matrix flags: - run formatting checks only for nightly - run qBittorrent E2E tests only for stable - keep toolchain-specific timeout values This removes duplicated setup/test steps while keeping the same CI coverage and execution policy.

Use --db-driver sqlite3 for the SQLite qBittorrent E2E step so it follows the same invocation pattern as MySQL and PostgreSQL. Behavior is unchanged because the runner maps sqlite3 to the same default compose file (compose.qbittorrent-e2e.sqlite3.yaml).

Refactor testing workflow to optimize runtime while preserving docker cache reuse in e2e execution: - rename matrix job from test to unit - keep nightly/stable checks in unit (fmt/lint/docs/unit tests) - merge tracker e2e and qBittorrent e2e into a single docker-e2e job so tracker image build and docker layers are reused in the same runner - run qBittorrent scenarios sequentially for sqlite3/mysql/postgresql - set docker-e2e timeout to 90 minutes

josecelano · 2026-05-01T12:44:59Z

ACK 453dd48

josecelano added 2 commits May 1, 2026 07:48

docs(issues): prefix 1525-08 spec with issue number

b06ee0f

docs(issues): update 1723-1525-08 spec with user answers and implemen…

cd665bf

…tation summary

josecelano had a problem deploying to coverage May 1, 2026 08:13 — with GitHub Actions Failure

josecelano self-assigned this May 1, 2026

josecelano temporarily deployed to coverage May 1, 2026 08:30 — with GitHub Actions Inactive

josecelano added 2 commits May 1, 2026 09:57

fix(tracker-core): correct postgres key timestamp column

15af1e0

ci(testing): add postgres compatibility job

54210f3

josecelano temporarily deployed to coverage May 1, 2026 09:01 — with GitHub Actions Inactive

josecelano temporarily deployed to coverage May 1, 2026 09:45 — with GitHub Actions Inactive

josecelano added 6 commits May 1, 2026 11:10

fix(tracker-core): wait for single postgres ready log in benchmark ru…

e0d0a87

…nner

docs(tracker-core): add 2026-05-01 benchmark run with PostgreSQL base…

aee2efb

…line

ci(container): isolate compose build paths from storage

248df3d

chore(repo): remove legacy compose file references

b0a654e

docs(containers): document PostgreSQL runtime configuration

3ef0707

docs(issues): mark task 9 complete in 1723 spec

3c9264c

josecelano requested a review from Copilot May 1, 2026 10:50

Copilot started reviewing on behalf of josecelano May 1, 2026 10:51 View session

josecelano added 4 commits May 1, 2026 11:51

docs(readme): add workflow badges for db-compatibility and db-benchma…

87d458c

…rking

Copilot AI reviewed May 1, 2026

View reviewed changes

Comment thread .github/workflows/testing.yaml Outdated

Comment thread packages/tracker-core/src/bin/persistence_benchmark/reporting.rs

Comment thread packages/tracker-core/src/databases/driver/postgres/mod.rs Outdated

josecelano added 2 commits May 1, 2026 12:03

josecelano temporarily deployed to coverage May 1, 2026 11:09 — with GitHub Actions Inactive

josecelano temporarily deployed to coverage May 1, 2026 11:18 — with GitHub Actions Inactive

josecelano temporarily deployed to coverage May 1, 2026 11:26 — with GitHub Actions Inactive

josecelano marked this pull request as ready for review May 1, 2026 11:26

josecelano requested a review from a team as a code owner May 1, 2026 11:26

josecelano temporarily deployed to coverage May 1, 2026 11:36 — with GitHub Actions Inactive

josecelano temporarily deployed to coverage May 1, 2026 12:44 — with GitHub Actions Inactive

josecelano merged commit b5051ef into torrust:develop May 1, 2026
28 of 33 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(tracker-core): add PostgreSQL driver (subissue #1723)#1724

feat(tracker-core): add PostgreSQL driver (subissue #1723)#1724
josecelano merged 22 commits intotorrust:developfrom
josecelano:1723-1525-08-add-postgresql-driver

josecelano commented May 1, 2026 •

edited

Loading

Uh oh!

codecov Bot commented May 1, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

josecelano commented May 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

josecelano commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PostgreSQL Driver for Torrust Tracker

Summary

Tasks

Status

Uh oh!

codecov Bot commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

josecelano commented May 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

josecelano commented May 1, 2026 •

edited

Loading

codecov Bot commented May 1, 2026 •

edited

Loading