docs(issues): add #1771 spec and enforce manual verification workflow#3
Closed
josecelano wants to merge 470 commits into
Closed
docs(issues): add #1771 spec and enforce manual verification workflow#3josecelano wants to merge 470 commits into
josecelano wants to merge 470 commits into
Conversation
- Fix stale "(both backends)" wording in the subissue 1525-06 spec — Task 1 only edits the SQLite migration; MySQL migration 1 must remain untouched per the immutability rule. - Build the SQLite test pool through `SqliteConnectOptions::filename` (mirroring the production driver) so non-UTF-8 and Windows paths are handled correctly; drop the brittle `to_str().unwrap()` URL-formatting in the test helper. Refs torrust#1719 (subissue 1525-06), PR torrust#1720.
…ons (subissue 1525-06) d6c2f65 refactor(tracker-core): address Copilot PR review feedback (Jose Celano) 897ca7e docs(tracker-core): clarify migration authoring guidelines (Jose Celano) e4bd575 feat(tracker-core): bootstrap legacy schemas into sqlx migration history (Jose Celano) fd3d7e8 feat(tracker-core): apply schema migrations automatically on startup (Jose Celano) 54677ee feat(tracker-core): scaffold sqlx schema migrations framework (Jose Celano) eb634ac docs(issues): resolve open questions and split 1525-06 plan into 3 phases (Jose Celano) 84eb3ac docs(issues): add current-code-analysis findings to 1525-06 spec (Jose Celano) 0864ded docs(issues): rename 1525-06 spec to include issue number prefix torrust#1719 (Jose Celano) Pull request description: Implements subissue 1525-06 of the persistence overhaul (torrust#1525): adopt automatic schema migrations using `sqlx::migrate!`. Spec: [docs/issues/1719-1525-06-introduce-schema-migrations.md](docs/issues/1719-1525-06-introduce-schema-migrations.md) Closes torrust#1719. ## Tasks - [x] Task 1 — Fix the SQLite-only `#` comment in migration 1 so the embedded migrator can parse it. - [x] Task 2 — Scaffolding: enable the sqlx `macros` feature, add per-driver `MIGRATOR` statics, add `Error::MigrationError` + `From` impl. - [x] Task 3 — Fresh-install path: wire `MIGRATOR.run()` into `create_database_tables()`, switch `drop_database_tables()` to `DROP TABLE IF EXISTS` (incl. `_sqlx_migrations`), rewrite `migrations/README.md`, add idempotency tests. - [x] Task 4 — Legacy bootstrap path: detect existing pre-v4 schemas without `_sqlx_migrations` and seed the migration history so the embedded migrator skips them on subsequent runs. Tests on both backends. Both fresh installs and pre-v4 in-place upgrades work end-to-end. The legacy-bootstrap layer is documented as a single removable unit (see the rust-doc on `bootstrap_legacy_schema` in each driver) so it can be dropped cleanly when pre-v4 support is no longer required. ACKs for top commit: josecelano: ACK d6c2f65 Tree-SHA512: 5918478feab23ce2fed6afec5635d61566d333e6bb00155f24e3dd7f417d1291c5ac93fc111603163474d3fd25be551538acfce4f54b44ee659727ebbdd32e74
Rename docs/issues/1525-07-align-rust-and-db-types.md to docs/issues/1721-1525-07-align-rust-and-db-types.md now that GitHub issue torrust#1721 has been opened for this subissue. Update the reference in the EPIC spec (docs/issues/1525-overhaul-persistence.md) accordingly.
… to bigint (subissue 1525-07) d67efdd docs(issues): align 1525-08 references with 1721 scope (Jose Celano) fa09390 fix(tracker-core): decode mysql metadata type bytes in tests (Jose Celano) 32b7e33 feat(tracker-core): widen mysql download counters to bigint (Jose Celano) a5936d1 docs(issues): update 1721-1525-07 scope to db-only widening (Jose Celano) 8f0c9ef docs(issues): rename 1525-07 spec to include GitHub issue number (Jose Celano) Pull request description: Implements subissue 1525-07 of the persistence overhaul (torrust#1525): align Rust and database types by widening only MySQL download-counter columns to `BIGINT` while keeping `NumberOfDownloads = u32`. Spec: [docs/issues/1721-1525-07-align-rust-and-db-types.md](docs/issues/1721-1525-07-align-rust-and-db-types.md) Closes torrust#1721. ## Scope - Add migration 4 for MySQL to widen: - `torrents.completed` -> `BIGINT NOT NULL DEFAULT 0` - `torrent_aggregate_metrics.value` -> `BIGINT NOT NULL DEFAULT 0` - Add migration 4 for SQLite as a no-op to keep migration history aligned. - Keep `NumberOfDownloads` as `u32` (no `u64` widening). - Update `SwarmMetadata.downloaded` and `downloads()` to use the `NumberOfDownloads` alias. ## Validation - MySQL db-compatibility test path now verifies migration outcomes: - migration history count is `4` - both target columns are `bigint` - round-trip works for `2_200_000_000_u32` (above `i32::MAX`) - SQLite idempotency test now verifies `_sqlx_migrations` count is `4`. - `cargo fmt --all` - `linter all` ACKs for top commit: josecelano: ACK d67efdd Tree-SHA512: 49c5228875a27840f31be059d83b3e65b6953f62b2d8a7f7927bccc4677238d4f811d443b41b8448ab41a62678d0355679635db8d6213a293b43c3a62e231f86
Implements a full PostgreSQL driver for the tracker, covering: - Driver::PostgreSQL variant in configuration and tracker-core - Four SQL migration files under migrations/postgresql/ - SchemaMigrator, AuthKeyStore, TorrentMetricsStore and WhitelistStore trait implementations using $1/$2 placeholders and ON CONFLICT UPSERT - Factory wiring in setup.rs - Benchmark runner support (postgres.rs + BenchmarkResource::Postgres) - Container entry script support for postgresql driver - Default container config tracker.container.postgresql.toml Closes torrust#1723 Part of torrust#1525
- Rename compose.qbittorrent-e2e.yaml to compose.qbittorrent-e2e.sqlite3.yaml - Add compose.qbittorrent-e2e.mysql.yaml for MySQL backend E2E tests - Add compose.qbittorrent-e2e.postgresql.yaml for PostgreSQL backend E2E tests - Add --db-driver CLI argument to qBittorrent E2E runner for backend selection - Select compose file and generate tracker config based on chosen backend - Add MySQL and PostgreSQL qBittorrent E2E CI steps to testing.yaml - Update SQLite CI step to use renamed compose file path - Update qbt debug scripts and README to use renamed compose file
Move database-compatibility-mysql and database-compatibility-postgres jobs from testing.yaml into a dedicated db-compatibility.yaml workflow. Motivation: the compat jobs only exercise the tracker-core package. Keeping them in a separate file makes the workflow portable if tracker-core is ever extracted to its own repository, and reduces the size of testing.yaml. The e2e job in testing.yaml now depends only on unit (the cross-workflow gate via database-compatibility was a soft ordering preference, not a hard requirement).
Add a new db-benchmarking.yaml workflow that runs the persistence_benchmark_runner binary against all three drivers on every push and pull request. Each job uses --ops 10 to keep runtime short while still exercising the full binary path: argument parsing, testcontainers startup, schema creation, query execution, and JSON report emission. A non-zero exit code (panic, missing container, broken SQL) fails the job immediately, catching runtime regressions that compilation alone cannot detect.
Move the cross-platform build job from testing.yaml into a new os-compatibility.yaml workflow. The build job has no dependency on any other job and is solely concerned with compilation portability across Ubuntu, macOS, and Windows. Keeping it separate follows the same principle applied to db-compatibility and db-benchmarking: one workflow, one concern. Also adds the os-compatibility badge to README.md.
Replace the four-job chain (format -> check -> unit -> e2e) with two flat jobs: test-nightly and test-stable. Each job runs checkout, toolchain setup, Node.js, Rust cache, dependency fetch, linter install, and tool install once, then executes all stages sequentially. Motivation: the previous structure recompiled the entire workspace multiple times per push across format, check, unit, and e2e jobs, each starting from a cold runner. Collapsing into two jobs eliminates the redundant compilations while keeping nightly and stable coverage separate. A cargo fetch step is added after cache restore in both jobs to make dependency download time visible in the job timeline. Differences from the old structure: - fmt check runs only under nightly (rustfmt nightly is the canonical formatter for this repo) - qBittorrent E2E tests (all three backends) run only under stable, matching the previous if: matrix.toolchain == 'stable' guard - test-nightly timeout: 45 min; test-stable timeout: 90 min
Move 10 closed issue spec files from docs/issues/ to the new docs/issues/closed/ buffer folder: torrust#523 internal-linting-tool torrust#1697 ai-agent-configuration torrust#1703 1525-01-persistence-test-coverage torrust#1706 1525-02-qbittorrent-e2e torrust#1710 1525-03-persistence-benchmarking torrust#1713 1525-04-split-persistence-traits torrust#1715 1525-04b-migrate-consumers-to-narrow-traits torrust#1717 1525-05-migrate-sqlite-and-mysql-to-sqlx torrust#1719 1525-06-introduce-schema-migrations torrust#1721 1525-07-align-rust-and-db-types Add docs/issues/closed/README.md explaining the two-stage lifecycle (archive then delete) for closed spec files. Update the cleanup-completed-issues skill (v1.1) to reflect the new process: move to closed/ first, delete permanently only when no longer referenced by active work.
Three issues flagged by the Copilot reviewer on PR torrust#1724: 1. Add PostgreSQL 14 to the db-compatibility CI matrix (.github/workflows/db-compatibility.yaml). The issue spec (1723-1525-08) lists 14 15 16 17 as the required version set; only 15-17 were present. 2. Add unit test for Driver::PostgreSQL in persistence_benchmark/reporting.rs. build_report handles PostgreSQL in the same match arm as MySQL but had no test coverage, leaving report-metadata regressions undetected. 3. Align create_legacy_pre_v4_schema to use BIGINT NOT NULL for keys.valid_until. Migration 1 creates the column as BIGINT NOT NULL; the helper used INTEGER NOT NULL, causing a type mismatch after migrator idempotency runs because there is no migration that widens that column.
Replace duplicated test-nightly and test-stable jobs with a single matrix-driven test job. Preserve behavior with matrix flags: - run formatting checks only for nightly - run qBittorrent E2E tests only for stable - keep toolchain-specific timeout values This removes duplicated setup/test steps while keeping the same CI coverage and execution policy.
Use --db-driver sqlite3 for the SQLite qBittorrent E2E step so it follows the same invocation pattern as MySQL and PostgreSQL. Behavior is unchanged because the runner maps sqlite3 to the same default compose file (compose.qbittorrent-e2e.sqlite3.yaml).
Refactor testing workflow to optimize runtime while preserving docker cache reuse in e2e execution: - rename matrix job from test to unit - keep nightly/stable checks in unit (fmt/lint/docs/unit tests) - merge tracker e2e and qBittorrent e2e into a single docker-e2e job so tracker image build and docker layers are reused in the same runner - run qBittorrent scenarios sequentially for sqlite3/mysql/postgresql - set docker-e2e timeout to 90 minutes
…roxy 3449801 docs(http): address copilot review suggestions on PR torrust#1766 (Jose Celano) 0235696 docs(agents): split task and PR review workflows (Jose Celano) 1e253f8 docs(issues): align 1736 workflow checkpoints before PR (Jose Celano) 6dd170e docs(http): document HTTP/3 reverse-proxy deployment (Jose Celano) efb8141 docs(workflow): reduce issue-doc duplication and add index readmes (Jose Celano) 29cce53 docs(issues): add HTTP/3 verification steps and fix torrust#1765 title (Jose Celano) 4cd6dc8 docs(issues): move HTTP/3 specs from drafts to open, assign torrust#1736 and torrust#1765 (Jose Celano) 752cb9f docs(issues): add draft specs for HTTP/3 proxy docs and native HTTP/3 readiness (Jose Celano) Pull request description: ## Summary Adds documentation for running HTTP/3 via a reverse proxy in front of the Torrust tracker's HTTP server, and introduces a follow-up issue spec for evaluating native HTTP/3 support. ### Changes - **`docs/containers.md`** — New section `### HTTP/3 at the edge with a reverse proxy` covering: - Why HTTP/3 terminates at the proxy (Axum/hyper do not yet support HTTP/3) - Caddy config example matching the format used in the official demo - Operational guidance and manual verification commands (`curl --http3`) - Link to the demo Caddy config for a real-world reference - **`docs/issues/open/1736-docs-http3-proxy.md`** — Spec for this issue (all tasks and ACs done) - **`docs/issues/open/1765-native-http3-readiness.md`** — Follow-up spec for native HTTP/3 evaluation (linked to GitHub issue torrust#1765, status: blocked pending hyperium/h3 stabilisation) - **`docs/issues/README.md`**, **`docs/issues/open/README.md`**, **`docs/issues/closed/README.md`**, **`docs/issues/drafts/README.md`** — Reduced duplication; added top-level index - **`AGENTS.md`** — Added policy torrust#6: documentation single-source-of-truth - **`.github/agents/`** — Split monolithic `reviewer.agent.md` into `task-reviewer.agent.md` (pre-PR acceptance validation) and `pr-reviewer.agent.md` (GitHub PR review only); updated `implementer.agent.md` to call `@task-reviewer` - **`.github/skills/dev/task-reviews/review-task/SKILL.md`** — New task-review skill - **`project-words.txt`** — Added `QUIC`, `hyperium`, `nghttp`, `ngtcp` ### Related issues Closes torrust#1736 Refs torrust#1765 ACKs for top commit: josecelano: ACK 3449801 Tree-SHA512: 66fe4e41f5ebcf9241c36732e360d8fb4dd1a22a33f05e14948bb89fcbbfe872225e0cb61ab305d82573896cf7a253662026c2fbf318ab4928b2841d393a2564
- Fix timeout_percent sample value: 33.3 -> 33 (integer, matches implementation) - Add --info-hash row to CLI Options table - Tick all completed Goals and Workflow Checkpoints - Add unit test: all latency fields null when every probe times out - Update refactor plan to mark items 1-4 as done
…tracker_checker f76c733 test(tracker-client): address copilot review feedback (Jose Celano) aece34b refactor(tracker-client): split tracker_checker integration tests by concern (Jose Celano) c5b0049 docs(planning): defer item 14 and close monitor udp refactor plan (Jose Celano) ec34d60 refactor(tracker-client): add MonitorStats conversion from Stats (Jose Celano) dbd452b refactor(tracker-client): extract run_probe_loop from run_monitor (Jose Celano) 617b1f5 fix(tracker-client): measure elapsed_ms after DNS resolution (Jose Celano) 1a650bf docs(tracker-client): document timeout_percent denominator includes error probes (Jose Celano) 36f5511 docs(tracker-client): document u64::MAX fallback is unreachable in elapsed_ms (Jose Celano) 7d404ba docs(tracker-client): clarify intent of both duration-check guards in run_monitor (Jose Celano) db747f6 docs(issue-1178): document last_ms null on timeout in AC3 and tick all ACs done (Jose Celano) 664a125 docs(issue-1178): correct Task 6 file reference to app.rs (Jose Celano) 7dac161 docs(tracker-client): document timeout-only nature of monitor_udp integration test (Jose Celano) df8c380 docs(issue-1178): fix spec accuracy and add all-null latency unit test (Jose Celano) d786a92 docs(planning): move agent-docs-refactor-plan to closed (Jose Celano) 7fb474b docs(planning): reorganise refactor-plans into drafts/open/closed subfolders (Jose Celano) 47d7092 docs(planning): add refactor plan template and create-refactor-plan skill (Jose Celano) 4d1493c docs(issue-1178): add post-implementation refactor plan (Jose Celano) 3067868 docs(issue-1178): add manual verification against old (down) demo tracker (Jose Celano) 2e38367 docs(issue-1178): add manual verification against demo tracker (Jose Celano) 5b0a786 feat(tracker-client): add tracker_checker monitor udp command (Jose Celano) 86a8246 docs(issue-1178): record maintainer feedback before implementation (Jose Celano) Pull request description: ## Summary - add `tracker_checker monitor udp` command in tracker client - emit NDJSON probe events to stderr and final JSON summary to stdout - add support for `--url`, `--interval`, `--timeout`, `--duration`, and `--info-hash` - implement timeout/error handling and monitor statistics (`total`, `timeouts`, `timeout_percent`, latency fields) - update issue/spec and refactor-plan docs, including explicit deferral of success-path mock UDP integration test to the planned tracker-client repository split - refactor `tracker_checker` integration tests into two concern modules: configuration and monitor ## Validation - `cargo test -p torrust-tracker-client --test tracker_checker` - `linter all` ## Notes - Item 14 from the post-implementation plan is intentionally deferred, documented in the issue and closed refactor plan. ## Usage Example ```sh cargo run -p torrust-tracker-client --bin tracker_checker -- monitor udp \ --url udp://udp1.torrust-tracker-demo.com:6969/announce \ --interval 10 \ --timeout 10 \ --duration 60 ``` The command emits per-probe NDJSON events to stderr and a final JSON summary to stdout. ## Example Output ```sh cargo run --quiet -p torrust-tracker-client --bin tracker_checker -- monitor udp --url udp://udp1.torrust-tracker-demo.com:6969/announce --interval 10 --timeout 10 --duration 60 2>/dev/null | jq ``` ```json { "udp_trackers": [ { "url": "udp://udp1.torrust-tracker-demo.com:6969/announce", "status": { "code": "ok", "message": "monitor completed", "stats": { "total": 6, "timeouts": 0, "timeout_percent": 0, "min_ms": 128, "max_ms": 145, "average_ms": 136, "last_ms": 137 } } } ] } ``` ACKs for top commit: josecelano: ACK f76c733 Tree-SHA512: e15d34835e788918136b0bb7e23c1827ec7c02794e6d84cb82300a3d26c64f6c96f934e5dd28a5cd1e0e2119aeebd0a4467c8e089ab503e68be90ae6660e5728
…rust#1769 e37e97e docs(issues): open specs for torrust#1768 and torrust#1769 (Jose Celano) Pull request description: ## Summary - move issue specs from drafts to open status for two new GitHub issues - assign metadata for issue torrust#1768 and issue torrust#1769 - update cross-links between both specs in docs/issues/open ## Included Specs - docs/issues/open/1768-refactor-update-dependencies-skill-automation.md - docs/issues/open/1769-refactor-pre-commit-checks-performance-and-verbosity.md ## Validation - linter all ## Related Issues - torrust#1768 - torrust#1769 ACKs for top commit: josecelano: ACK e37e97e Tree-SHA512: 02857f8df83c16074c8045189e8a00f32a765840274629341012e9a0897ead1721a16cdcef094b9f558c071429e984ebac530f6ac18f6e53e4ca37dba7f56ef8
Owner
Author
|
Closing this PR because the correct PR for review is opened against upstream: torrust#1772 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR intentionally contains only issue/spec workflow documentation updates so they can be reviewed and merged before implementation starts.
What changed
linter all, relevant tests, pre-push where applicable)Why
For complex issues, merging the issue spec first improves visibility and allows review of scope/acceptance criteria before code changes.
Manual verification was frequently omitted by agents; templates/skill now make it an explicit required step.
Scope of this PR
Documentation/spec process changes only. No production code changes.