Skip to content

docs(1525-03): align persistence benchmarking spec with issue #1710#1711

Merged
josecelano merged 5 commits into
torrust:developfrom
josecelano:1710-add-persistence-benchmarking
Apr 29, 2026
Merged

docs(1525-03): align persistence benchmarking spec with issue #1710#1711
josecelano merged 5 commits into
torrust:developfrom
josecelano:1710-add-persistence-benchmarking

Conversation

@josecelano
Copy link
Copy Markdown
Member

This draft PR updates the persistence benchmarking spec to match the approved issue #1710 proposal.

Summary:

  • Rename the issue spec file to include the GitHub issue prefix.
  • Replace the original HTTP/Compose benchmarking design with the DB-driver-level approach.
  • Set the new source of truth to docs/issues/1710-1525-03-persistence-benchmarking.md.

Context:

…l approach

Revise the spec to reflect the simplified DB-driver-level benchmarking approach:
- Benchmark Database trait methods directly, not through HTTP API
- Remove Docker Compose and image-swapping complexity
- Place binary in packages/tracker-core (not workspace root)
- Report count, best, median, worst per operation (no p95 or ops/sec)
- Single --ops default (10) for fast local runs under 3 minutes
- Run once per driver/version; git diff of committed reports is the before/after comparison
- Document in docs/benchmarking.md
@josecelano josecelano self-assigned this Apr 28, 2026
@josecelano josecelano linked an issue Apr 28, 2026 that may be closed by this pull request
7 tasks
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

Codecov Report

❌ Patch coverage is 38.65815% with 384 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.44%. Comparing base (f6d4544) to head (5647890).
⚠️ Report is 6 commits behind head on develop.

Files with missing lines Patch % Lines
...tence_benchmark/driver_bench/operations/torrent.rs 0.00% 85 Missing ⚠️
...nce_benchmark/driver_bench/operations/whitelist.rs 0.00% 56 Missing ⚠️
...sistence_benchmark/driver_bench/operations/keys.rs 0.00% 54 Missing ⚠️
...persistence_benchmark/driver_bench/database/mod.rs 0.00% 36 Missing ⚠️
...acker-core/src/bin/persistence_benchmark/runner.rs 0.00% 31 Missing ⚠️
...rsistence_benchmark/driver_bench/database/mysql.rs 0.00% 27 Missing ⚠️
...bin/persistence_benchmark/driver_bench/sampling.rs 0.00% 25 Missing ⚠️
...rsistence_benchmark/driver_bench/operations/mod.rs 0.00% 21 Missing ⚠️
...sistence_benchmark/driver_bench/database/sqlite.rs 0.00% 15 Missing ⚠️
.../src/bin/persistence_benchmark/driver_bench/mod.rs 0.00% 11 Missing ⚠️
... and 4 more
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1711      +/-   ##
===========================================
- Coverage    82.55%   81.44%   -1.12%     
===========================================
  Files          323      340      +17     
  Lines        24334    24960     +626     
  Branches     24334    24960     +626     
===========================================
+ Hits         20089    20328     +239     
- Misses        3987     4370     +383     
- Partials       258      262       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Add the persistence_benchmark_runner binary for tracker-core with CLI options for driver, db version, and operation count.

Implement benchmark orchestration and per-operation timing for torrent, whitelist, and auth key database operations across sqlite3 and mysql backends.

Add JSON report generation with timing statistics and metadata, plus utility modules for metrics, types, sampling, and git revision capture.

Update tracker-core dependencies/binary target, extend database driver parsing helpers, and document issue 1710 benchmarking implementation details.

Closes torrust#1710, part of torrust#1525
Add the 2026-04-28 baseline benchmarking docs, machine profile, and raw sqlite/mysql JSON results under tracker-core docs.

Update cspell ignore patterns for benchmark machine artifacts as a lint follow-up.
@josecelano josecelano marked this pull request as ready for review April 28, 2026 19:19
@josecelano josecelano requested a review from Copilot April 28, 2026 19:21
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new persistence benchmarking runner (driver-level, not HTTP-level) in bittorrent-tracker-core, adds baseline benchmark artifacts/docs, and updates the issue spec source-of-truth document for subissue #1525-03 (issue #1710).

Changes:

  • Add a new persistence_benchmark_runner binary plus supporting modules to benchmark Database trait operations for sqlite3 and mysql (via testcontainers).
  • Extend Driver with stable string identifiers and FromStr parsing for CLI usage.
  • Commit baseline benchmark outputs and documentation under packages/tracker-core/docs/benchmarking/, and replace the old spec with docs/issues/1710-1525-03-persistence-benchmarking.md.

Reviewed changes

Copilot reviewed 29 out of 31 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
packages/tracker-core/src/databases/driver/mod.rs Add Driver::as_str() and FromStr to support CLI/report identifiers.
packages/tracker-core/src/bin/persistence_benchmark_runner.rs New binary entrypoint + extensive runner documentation.
packages/tracker-core/src/bin/persistence_benchmark/** Runner implementation: CLI parsing, DB setup (sqlite/mysql), sampling, metrics, report serialization.
packages/tracker-core/docs/benchmarking/** Add baseline run artifacts (JSON), run report, and machine profile + README describing workflow.
packages/tracker-core/Cargo.toml Add dependencies needed for the benchmark runner (anyhow, clap, testcontainers).
docs/issues/1710-1525-03-persistence-benchmarking.md New spec document replacing the old 1525-03 spec.
docs/issues/1525-03-persistence-benchmarking.md Remove the old HTTP/compose-based spec.
cspell.json Ignore machine profile text files for spelling checks.
.gitignore Ignore .benchmarks/ output directory.
Cargo.lock Lockfile updates for new dependencies.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +33 to +34
let database = initialize_database(&config);

Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

initialize_database(&config) can panic because it calls driver::build(...).expect(...), and driver::build calls create_database_tables().expect(...). For MySQL containers this is flaky because the server may not be ready immediately after start(), so the benchmark can abort before your retry logic runs. Consider adding a non-panicking initializer in the library (e.g. try_initialize_database) or otherwise retry the initialization itself until the DB is ready, rather than retrying table creation after initialization.

Suggested change
let database = initialize_database(&config);
let mut database = None;
let max_attempts = 10;
for attempt in 1..=max_attempts {
match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| initialize_database(&config))) {
Ok(initialized_database) => {
database = Some(initialized_database);
break;
}
Err(_) if attempt < max_attempts => {
tokio::time::sleep(std::time::Duration::from_millis(500)).await;
}
Err(_) => {
return Err(anyhow::anyhow!(
"failed to initialize mysql database after {max_attempts} attempts"
));
}
}
}
let database = database.context("mysql database initialization did not succeed")?;

Copilot uses AI. Check for mistakes.
Comment thread packages/tracker-core/src/bin/persistence_benchmark/runner.rs
Comment thread packages/tracker-core/src/bin/persistence_benchmark/runner.rs
Comment thread packages/tracker-core/src/bin/persistence_benchmark/driver_bench/sampling.rs Outdated
Comment thread docs/issues/1710-1525-03-persistence-benchmarking.md Outdated
Comment thread docs/issues/1710-1525-03-persistence-benchmarking.md Outdated
Comment thread docs/issues/1710-1525-03-persistence-benchmarking.md Outdated
Comment thread packages/tracker-core/src/bin/persistence_benchmark/driver_bench/database/mod.rs Outdated
Comment on lines 20 to 45
@@ -39,12 +41,12 @@ torrust-tracker-located-error = { version = "3.0.0-develop", path = "../located-
torrust-tracker-metrics = { version = "3.0.0-develop", path = "../metrics" }
torrust-tracker-primitives = { version = "3.0.0-develop", path = "../primitives" }
torrust-tracker-swarm-coordination-registry = { version = "3.0.0-develop", path = "../swarm-coordination-registry" }
testcontainers = "0"
tracing = "0"
Copy link

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

anyhow, clap, and especially testcontainers are added as unconditional library dependencies, but they are only used by the developer-only benchmark binary. This increases compile times and dependency surface for downstream users of bittorrent-tracker-core. Consider making these dependencies optional behind a dedicated feature (e.g. persistence-benchmark) and requiring that feature for the persistence_benchmark_runner binary.

Copilot uses AI. Check for mistakes.
- Separate fixture setup from timed section in measure_operation
  by switching to a two-closure (setup, operation) signature so
  recorded durations reflect only the database call.
- Move database handle into Option<Arc<Box<dyn Database>>> so it is
  explicitly dropped before the SQLite file is removed in Drop.
- Preserve the last error from create_database_tables_with_retry
  instead of discarding it, making container startup failures easier
  to diagnose.
- Align docs/issues/1710-1525-03-persistence-benchmarking.md with
  the implementation: ops default 100, stdout-only JSON output, and
  correct artifact paths under packages/tracker-core/docs/benchmarking.
@josecelano
Copy link
Copy Markdown
Member Author

ACK 5647890

@josecelano josecelano merged commit 899fc82 into torrust:develop Apr 29, 2026
28 of 33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(tracker-core): add persistence benchmarking (#1525-03)

2 participants