docs(1525-03): align persistence benchmarking spec with issue #1710#1711
Conversation
…l approach Revise the spec to reflect the simplified DB-driver-level benchmarking approach: - Benchmark Database trait methods directly, not through HTTP API - Remove Docker Compose and image-swapping complexity - Place binary in packages/tracker-core (not workspace root) - Report count, best, median, worst per operation (no p95 or ops/sec) - Single --ops default (10) for fast local runs under 3 minutes - Run once per driver/version; git diff of committed reports is the before/after comparison - Document in docs/benchmarking.md
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #1711 +/- ##
===========================================
- Coverage 82.55% 81.44% -1.12%
===========================================
Files 323 340 +17
Lines 24334 24960 +626
Branches 24334 24960 +626
===========================================
+ Hits 20089 20328 +239
- Misses 3987 4370 +383
- Partials 258 262 +4 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Add the persistence_benchmark_runner binary for tracker-core with CLI options for driver, db version, and operation count. Implement benchmark orchestration and per-operation timing for torrent, whitelist, and auth key database operations across sqlite3 and mysql backends. Add JSON report generation with timing statistics and metadata, plus utility modules for metrics, types, sampling, and git revision capture. Update tracker-core dependencies/binary target, extend database driver parsing helpers, and document issue 1710 benchmarking implementation details. Closes torrust#1710, part of torrust#1525
Add the 2026-04-28 baseline benchmarking docs, machine profile, and raw sqlite/mysql JSON results under tracker-core docs. Update cspell ignore patterns for benchmark machine artifacts as a lint follow-up.
There was a problem hiding this comment.
Pull request overview
This PR introduces a new persistence benchmarking runner (driver-level, not HTTP-level) in bittorrent-tracker-core, adds baseline benchmark artifacts/docs, and updates the issue spec source-of-truth document for subissue #1525-03 (issue #1710).
Changes:
- Add a new
persistence_benchmark_runnerbinary plus supporting modules to benchmarkDatabasetrait operations forsqlite3andmysql(via testcontainers). - Extend
Driverwith stable string identifiers andFromStrparsing for CLI usage. - Commit baseline benchmark outputs and documentation under
packages/tracker-core/docs/benchmarking/, and replace the old spec withdocs/issues/1710-1525-03-persistence-benchmarking.md.
Reviewed changes
Copilot reviewed 29 out of 31 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/tracker-core/src/databases/driver/mod.rs | Add Driver::as_str() and FromStr to support CLI/report identifiers. |
| packages/tracker-core/src/bin/persistence_benchmark_runner.rs | New binary entrypoint + extensive runner documentation. |
| packages/tracker-core/src/bin/persistence_benchmark/** | Runner implementation: CLI parsing, DB setup (sqlite/mysql), sampling, metrics, report serialization. |
| packages/tracker-core/docs/benchmarking/** | Add baseline run artifacts (JSON), run report, and machine profile + README describing workflow. |
| packages/tracker-core/Cargo.toml | Add dependencies needed for the benchmark runner (anyhow, clap, testcontainers). |
| docs/issues/1710-1525-03-persistence-benchmarking.md | New spec document replacing the old 1525-03 spec. |
| docs/issues/1525-03-persistence-benchmarking.md | Remove the old HTTP/compose-based spec. |
| cspell.json | Ignore machine profile text files for spelling checks. |
| .gitignore | Ignore .benchmarks/ output directory. |
| Cargo.lock | Lockfile updates for new dependencies. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| let database = initialize_database(&config); | ||
|
|
There was a problem hiding this comment.
initialize_database(&config) can panic because it calls driver::build(...).expect(...), and driver::build calls create_database_tables().expect(...). For MySQL containers this is flaky because the server may not be ready immediately after start(), so the benchmark can abort before your retry logic runs. Consider adding a non-panicking initializer in the library (e.g. try_initialize_database) or otherwise retry the initialization itself until the DB is ready, rather than retrying table creation after initialization.
| let database = initialize_database(&config); | |
| let mut database = None; | |
| let max_attempts = 10; | |
| for attempt in 1..=max_attempts { | |
| match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| initialize_database(&config))) { | |
| Ok(initialized_database) => { | |
| database = Some(initialized_database); | |
| break; | |
| } | |
| Err(_) if attempt < max_attempts => { | |
| tokio::time::sleep(std::time::Duration::from_millis(500)).await; | |
| } | |
| Err(_) => { | |
| return Err(anyhow::anyhow!( | |
| "failed to initialize mysql database after {max_attempts} attempts" | |
| )); | |
| } | |
| } | |
| } | |
| let database = database.context("mysql database initialization did not succeed")?; |
| @@ -39,12 +41,12 @@ torrust-tracker-located-error = { version = "3.0.0-develop", path = "../located- | |||
| torrust-tracker-metrics = { version = "3.0.0-develop", path = "../metrics" } | |||
| torrust-tracker-primitives = { version = "3.0.0-develop", path = "../primitives" } | |||
| torrust-tracker-swarm-coordination-registry = { version = "3.0.0-develop", path = "../swarm-coordination-registry" } | |||
| testcontainers = "0" | |||
| tracing = "0" | |||
There was a problem hiding this comment.
anyhow, clap, and especially testcontainers are added as unconditional library dependencies, but they are only used by the developer-only benchmark binary. This increases compile times and dependency surface for downstream users of bittorrent-tracker-core. Consider making these dependencies optional behind a dedicated feature (e.g. persistence-benchmark) and requiring that feature for the persistence_benchmark_runner binary.
- Separate fixture setup from timed section in measure_operation by switching to a two-closure (setup, operation) signature so recorded durations reflect only the database call. - Move database handle into Option<Arc<Box<dyn Database>>> so it is explicitly dropped before the SQLite file is removed in Drop. - Preserve the last error from create_database_tables_with_retry instead of discarding it, making container startup failures easier to diagnose. - Align docs/issues/1710-1525-03-persistence-benchmarking.md with the implementation: ops default 100, stdout-only JSON output, and correct artifact paths under packages/tracker-core/docs/benchmarking.
|
ACK 5647890 |
This draft PR updates the persistence benchmarking spec to match the approved issue #1710 proposal.
Summary:
Context: