|
| 1 | +# PR #1684 Review: PostgreSQL Driver Support |
| 2 | + |
| 3 | +## Summary |
| 4 | + |
| 5 | +This review covers both the original PR #1684 and its follow-up rework in PR #1695 |
| 6 | +(`DamnCrab:codex/pg-adaptation-rework`). PR #1684 was rejected due to a merge-blocking counter |
| 7 | +overflow bug and an architectural concern about per-query OS thread spawning. The rework in PR #1695 |
| 8 | +addresses all four issues raised in that review. This document records the original findings and |
| 9 | +documents how each was resolved. |
| 10 | + |
| 11 | +--- |
| 12 | + |
| 13 | +## Issues Found in PR #1684 |
| 14 | + |
| 15 | +### 1. Counter Overflow Risk — was MERGE BLOCKER — ✅ Resolved in PR #1695 |
| 16 | + |
| 17 | +**Original finding:** The PR #1684 PostgreSQL driver stored download counters in signed 32-bit |
| 18 | +`INTEGER` columns and converted through `i32` with `.unwrap()` calls. Any counter exceeding |
| 19 | +2,147,483,647 would panic at runtime. |
| 20 | + |
| 21 | +**Root cause:** The `r2d2_postgres` crate does not auto-downcast integer types, so the code had to |
| 22 | +explicitly specify `i32`. Combined with a 32-bit `INTEGER` column, this created a hard overflow at |
| 23 | +2 billion downloads. |
| 24 | + |
| 25 | +**Resolution in rework:** The rework migrated to `sqlx`, which supports async-native PostgreSQL |
| 26 | +access and handles type mapping directly. Counters are now stored and read as `i64`, with explicit |
| 27 | +error-propagating conversions via `.map_err()` instead of `.unwrap()`. Relevant locations in |
| 28 | +[packages/tracker-core/src/databases/driver/postgres.rs](../../packages/tracker-core/src/databases/driver/postgres.rs): |
| 29 | + |
| 30 | +- Line 73: `fn decode_counter_i64(&self, value: i64) -> Result<...>` — converts `i64` to |
| 31 | + `NumberOfDownloads` with error propagation |
| 32 | +- Line 77: `fn encode_counter(&self, value: NumberOfDownloads) -> Result<i64, Error>` — converts |
| 33 | + `NumberOfDownloads` to `i64` with error propagation |
| 34 | +- Line 153: counter columns read as `i64` via `row.try_get("completed")` |
| 35 | + |
| 36 | +No panicking `.unwrap()` calls remain on counter conversion paths. |
| 37 | + |
| 38 | +### 2. Per-Query OS Thread Spawning — was Medium Priority — ✅ Resolved in PR #1695 |
| 39 | + |
| 40 | +**Original finding:** PR #1684 spawned and joined a fresh OS thread for every database operation |
| 41 | +to work around a nested Tokio runtime conflict introduced by the `r2d2_postgres` / sync `postgres` |
| 42 | +crate. This was not free: each query created an OS thread, allocated a stack, and paid context |
| 43 | +switch overhead. Under high load this would have been measurable. |
| 44 | + |
| 45 | +**Resolution in rework:** The rework replaced `r2d2_postgres` with `sqlx`, which is async-native. |
| 46 | +The driver now runs directly on the existing Tokio runtime with no thread spawning. The connection |
| 47 | +pool is managed by `sqlx`'s built-in async pool, making the PostgreSQL driver consistent with how |
| 48 | +the SQLite and MySQL drivers will be migrated in subissue #1525-03. |
| 49 | + |
| 50 | +### 3. Missing PostgreSQL Configuration File — was Medium Priority — ✅ Resolved in PR #1695 |
| 51 | + |
| 52 | +**Original finding:** No default PostgreSQL configuration file was provided in the deployment |
| 53 | +directory, unlike the existing `tracker.container.mysql.toml`. |
| 54 | + |
| 55 | +**Resolution in rework:** |
| 56 | +`share/default/config/tracker.container.postgresql.toml` was added, parallel to the MySQL |
| 57 | +container configuration. |
| 58 | + |
| 59 | +### 4. Missing Database Migrations — was Medium Priority — ✅ Resolved in PR #1695 |
| 60 | + |
| 61 | +**Original finding:** No migration infrastructure existed. Adding a new backend with schema |
| 62 | +requirements without migrations made schema evolution risky. |
| 63 | + |
| 64 | +**Resolution in rework:** A full migrations directory was introduced at |
| 65 | +`packages/tracker-core/migrations/` with subdirectories for each backend |
| 66 | +(`mysql/`, `postgresql/`, `sqlite/`). Migrations are applied automatically when a database-backed |
| 67 | +store is first used. Timestamps are shared across backends to keep schema evolution aligned. |
| 68 | + |
| 69 | +--- |
| 70 | + |
| 71 | +## Validation Results (PR #1684) |
| 72 | + |
| 73 | +✅ **Passed locally:** |
| 74 | + |
| 75 | +- `cargo test -p bittorrent-tracker-core databases::error::tests` |
| 76 | +- `TORRUST_TRACKER_CORE_RUN_POSTGRES_DRIVER_TEST=true cargo test -p bittorrent-tracker-core run_postgres_driver_tests` |
| 77 | +- `cargo test -p torrust-tracker-configuration database` |
| 78 | + |
| 79 | +✅ **Integration test:** PostgreSQL driver ran successfully against a real container in |
| 80 | +testcontainers. |
| 81 | + |
| 82 | +✅ **Configuration handling:** PostgreSQL URL masking and schema registration work correctly. |
| 83 | + |
| 84 | +--- |
| 85 | + |
| 86 | +## Positive Aspects (PR #1684) |
| 87 | + |
| 88 | +- Configuration and driver wiring were clean |
| 89 | +- Error handling integration (`GenericConnectionError` variant) was appropriate |
| 90 | +- Tests were comprehensive with both container and local database options |
| 91 | +- Documentation was clear about runtime constraints |
| 92 | + |
| 93 | +--- |
| 94 | + |
| 95 | +## Original Recommendation |
| 96 | + |
| 97 | +PR #1684 was rejected. The main reason was architectural: the per-query OS thread spawning model |
| 98 | +was below the minimum performance bar for merge, independent of the overflow bug. |
| 99 | + |
| 100 | +The recommendation was to revisit PostgreSQL support alongside or after the persistence redesign |
| 101 | +tracked in [issue #1525](https://github.com/torrust/torrust-tracker/issues/1525). PR #1695 follows |
| 102 | +exactly that direction: it reworks the full persistence layer and delivers PostgreSQL on top of the |
| 103 | +new async `sqlx` substrate in a single coherent redesign. |
| 104 | + |
| 105 | +--- |
| 106 | + |
| 107 | +## Remaining Issues in PR #1695 |
| 108 | + |
| 109 | +Running `linter all` against the rework branch reveals failures that must be addressed before |
| 110 | +PR #1695 can merge. |
| 111 | + |
| 112 | +### Clippy errors (37) |
| 113 | + |
| 114 | +Affected files (all in `packages/tracker-core`): |
| 115 | + |
| 116 | +- `src/databases/driver/mysql.rs` — `unused_self`, `needless_pass_by_value`, casting `u64` to |
| 117 | + `usize` |
| 118 | +- `src/databases/driver/postgres.rs` — same categories |
| 119 | +- `src/databases/driver/sqlite.rs` — same categories |
| 120 | +- `src/databases/mod.rs` — `all fields have the same postfix: keys` |
| 121 | +- `src/torrent/services.rs` — `useless conversion to the same type: u64` |
| 122 | +- `packages/torrent-repository-benchmarking/tests/repository/mod.rs` — casting truncation |
| 123 | + |
| 124 | +The project's `.cargo/config.toml` sets `-D warnings` globally, so these are hard errors that |
| 125 | +prevent compilation. |
| 126 | + |
| 127 | +### Spell-checking failures |
| 128 | + |
| 129 | +New words introduced by the rework are not yet in `project-words.txt`: |
| 130 | +`sqlx`, `Sqlx`, `isready`, `getpid`, `qbittorrent`, `urandom`, `savepath`. |
0 commit comments