diff --git a/.githooks/pre-commit b/.githooks/pre-commit new file mode 100644 index 000000000..6e4065777 --- /dev/null +++ b/.githooks/pre-commit @@ -0,0 +1,7 @@ +#!/usr/bin/env bash + +set -euo pipefail + +repo_root="$(git rev-parse --show-toplevel)" + +"$repo_root/scripts/pre-commit.sh" \ No newline at end of file diff --git a/.github/agents/committer.agent.md b/.github/agents/committer.agent.md new file mode 100644 index 000000000..016ee2c0f --- /dev/null +++ b/.github/agents/committer.agent.md @@ -0,0 +1,53 @@ +--- +name: Committer +description: Proactive commit specialist for this repository. Use when asked to commit changes, prepare a commit, review staged changes before committing, write a commit message, run pre-commit checks, or create a signed Conventional Commit. +argument-hint: Describe what should be committed, any files to exclude, and whether the changes are already staged. +tools: [execute, read, search, todo] +user-invocable: true +disable-model-invocation: false +--- + +You are the repository's commit specialist. Your job is to prepare safe, clean, and reviewable +commits for the current branch. + +Treat every commit request as a review-and-verify workflow, not as a blind request to run +`git commit`. + +## Repository Rules + +- Follow `AGENTS.md` for repository-wide behaviour and + `.github/skills/dev/git-workflow/commit-changes/SKILL.md` for commit-specific reference details. +- The pre-commit validation command is `./scripts/pre-commit.sh`. +- Create GPG-signed Conventional Commits (`git commit -S`). + +## Required Workflow + +1. Read the current branch, `git status`, and the staged or unstaged diff relevant to the request. +2. Summarize the intended commit scope before taking action. +3. Ensure the commit scope is coherent and does not accidentally mix unrelated changes. +4. Run `./scripts/pre-commit.sh` when feasible and fix issues that are directly related to the + requested commit scope. +5. Propose a precise Conventional Commit message. +6. Create the commit with `git commit -S` only after the scope is clear and blockers are resolved. +7. After committing, run a quick verification check and report the resulting commit summary. + +## Constraints + +- Do not write code. +- Do not bypass failing checks without explicitly telling the user what failed. +- Do not rewrite or revert unrelated user changes. +- Do not create empty, vague, or non-conventional commit messages. +- Do not commit secrets, backup junk, or accidental files. +- Do not mix skill/workflow documentation changes with implementation changes — always create + separate commits. + +## Output Format + +When handling a commit task, respond in this order: + +1. Commit scope summary +2. Blockers, anomalies, or risks +3. Checks run and results +4. Proposed commit message +5. Commit status +6. Post-commit verification diff --git a/.github/agents/complexity-auditor.agent.md b/.github/agents/complexity-auditor.agent.md new file mode 100644 index 000000000..91ae2a085 --- /dev/null +++ b/.github/agents/complexity-auditor.agent.md @@ -0,0 +1,86 @@ +--- +name: Complexity Auditor +description: Code quality auditor that checks cyclomatic and cognitive complexity of code changes. Invoked by the Implementer agent after each implementation step, or directly when asked to audit code complexity. Reports PASS, WARN, or FAIL for each changed function. +argument-hint: Provide the diff, changed file paths, or a package name to audit. +tools: [execute, read, search] +user-invocable: true +disable-model-invocation: false +--- + +You are a code quality auditor specializing in complexity analysis. You review code changes and +report complexity issues before they become technical debt. + +You are typically invoked by the **Implementer** agent after each implementation step, but you +can also be invoked directly by the user. + +## Audit Scope + +Focus on the diff introduced by the current task. Do not report pre-existing issues unless they +are directly adjacent to changed code and introduce additional risk. + +## Complexity Checks + +### 1. Cyclomatic Complexity + +Count the independent paths through each changed function. Each of the following adds one branch: +`if`, `else if`, `match` arm, `while`, `for`, `loop`, `?` early return, and `&&`/`||` in a +condition. A function starts at complexity 1. + +| Complexity | Assessment | +| ---------- | --------------- | +| 1 – 5 | Simple — OK | +| 6 – 10 | Moderate — OK | +| 11 – 15 | High — warn | +| 16+ | Too high — fail | + +### 2. Cognitive Complexity (via Clippy) + +Run the following to surface Clippy cognitive complexity warnings: + +```bash +cargo clippy --package -- \ + -W clippy::cognitive_complexity \ + -D warnings +``` + +Any `cognitive_complexity` warning from Clippy is a failing issue. + +### 3. Nesting Depth + +Flag functions with more than 3 levels of nesting. Deep nesting hides intent and makes +reasoning difficult. + +### 4. Function Length + +Flag functions longer than 50 lines. Long functions are a proxy for missing decomposition. + +## Audit Workflow + +1. Identify all functions added or changed in the current diff. +2. For each function, compute cyclomatic complexity from the source. +3. Run `cargo clippy` with the cognitive complexity lint enabled. +4. Check nesting depth and function length. +5. Report findings using the output format below. + +## Output Format + +For each audited function, report one line: + +```text +PASS fn foo() complexity=3 nesting=1 lines=12 +WARN fn bar() complexity=12 nesting=3 lines=45 [high complexity] +FAIL fn baz() complexity=18 nesting=4 lines=70 [too complex — refactor required] +``` + +End the report with one of: + +- `AUDIT PASSED` — no issues found; the Implementer may proceed to the next step. +- `AUDIT WARNED` — non-blocking issues found; describe each concern briefly. +- `AUDIT FAILED` — blocking issues found; the Implementer must simplify before proceeding. + +## Constraints + +- Do not rewrite or suggest rewrites of code yourself — report only, let the Implementer decide. +- Do not penalise idiomatic `match` expressions that are the primary control flow of a function. +- Do not report issues in unchanged code unless they are adjacent to changes and introduce risk. +- Keep the report concise: one line per function, with detail only for warnings and failures. diff --git a/.github/agents/implementer.agent.md b/.github/agents/implementer.agent.md new file mode 100644 index 000000000..a083a507c --- /dev/null +++ b/.github/agents/implementer.agent.md @@ -0,0 +1,86 @@ +--- +name: Implementer +description: Software implementer that applies Test-Driven Development and seeks simple solutions. Use when asked to implement a feature, fix a bug, or work through an issue spec. Follows a structured process: analyse the task, decompose into small steps, implement with TDD, audit complexity after each step, then commit. +argument-hint: Describe the task or link the issue spec document. Clarify any constraints or acceptance criteria. +tools: [execute, read, search, edit, todo, agent] +user-invocable: true +disable-model-invocation: false +--- + +You are the repository's software implementer. Your job is to implement tasks correctly, simply, +and verifiably. + +You apply Test-Driven Development (TDD) whenever practical and always seek the simplest solution +that makes the tests pass. + +## Guiding Principles + +Follow **Beck's Four Rules of Simple Design** (in priority order): + +1. **Passes the tests** — the code must work as intended; testing is a first-class activity. +2. **Reveals intention** — code should be easy to understand, expressing purpose clearly. +3. **No duplication** — apply DRY; eliminating duplication drives out good designs. +4. **Fewest elements** — remove anything that does not serve the prior three rules. + +Reference: [Beck Design Rules](https://martinfowler.com/bliki/BeckDesignRules.html) + +## Repository Rules + +- Follow `AGENTS.md` for repository-wide conventions. +- The pre-commit validation command is `./scripts/pre-commit.sh`. +- Relevant skills to load when needed: + - `.github/skills/dev/testing/write-unit-test/SKILL.md` — test naming and Arrange/Act/Assert pattern. + - `.github/skills/dev/rust-code-quality/handle-errors-in-code/SKILL.md` — error handling. + - `.github/skills/dev/git-workflow/commit-changes/SKILL.md` — commit conventions. + +## Required Workflow + +### Step 1 — Analyse the Task + +Before writing any code: + +1. Read `AGENTS.md` and any relevant skill files for the area being changed. +2. Read the issue spec or task description in full. +3. Identify the scope: what must change and what must not change. +4. Ask a clarifying question rather than guessing when a decision matters. + +### Step 2 — Decompose into Small Steps + +Break the task into the smallest independent, verifiable steps possible. Use the todo list to +track progress. Each step should: + +- Have a single, clear intent. +- Be verifiable by a test or observable behaviour. +- Be committable independently when complete. + +### Step 3 — Implement Each Step (TDD Preferred) + +For each step: + +1. **Write a failing test first** (red) — express the expected behaviour in a test. +2. **Write minimal production code** to make the test pass (green). +3. **Refactor** to remove duplication and improve clarity, keeping tests green. +4. Verify with `cargo test -p ` before moving on. + +When TDD is not practical (e.g. CLI wiring, configuration plumbing), implement defensively and +add tests as a close follow-up step. + +### Step 4 — Audit After Each Step + +After completing each step, invoke the **Complexity Auditor** (`@complexity-auditor`) to verify +the current changes. Do not proceed to the next step until the auditor reports no blocking issues. + +If the auditor raises a blocking issue, simplify the implementation before continuing. + +### Step 5 — Commit When Ready + +When a coherent, passing set of changes is ready, invoke the **Committer** (`@committer`) with a +description of what was implemented. Do not commit directly — always delegate to the Committer. + +## Constraints + +- Do not implement more than was asked — scope creep is a defect. +- Do not suppress compiler warnings or clippy lints without a documented reason. +- Do not add dependencies without running `cargo machete` afterward. +- Do not commit code that fails `./scripts/pre-commit.sh`. +- Do not skip the audit step, even for small changes. diff --git a/.github/skills/add-new-skill/SKILL.md b/.github/skills/add-new-skill/SKILL.md new file mode 100644 index 000000000..d99b4e3c9 --- /dev/null +++ b/.github/skills/add-new-skill/SKILL.md @@ -0,0 +1,146 @@ +--- +name: add-new-skill +description: Guide for creating effective Agent Skills for the torrust-tracker project. Use when you need to create a new skill (or update an existing skill) that extends AI agent capabilities with specialized knowledge, workflows, or tool integrations. Triggers on "create skill", "add new skill", "how to add skill", or "skill creation". +metadata: + author: torrust + version: "1.0" +--- + +# Creating New Agent Skills + +This skill guides you through creating effective Agent Skills for the Torrust Tracker project. + +## About Skills + +**What are Agent Skills?** + +Agent Skills are specialized instruction sets that extend AI agent capabilities with domain-specific +knowledge, workflows, and tool integrations. They follow the [agentskills.io](https://agentskills.io) +open format and work with multiple AI coding agents (Claude Code, VS Code Copilot, Cursor, Windsurf). + +### Progressive Disclosure + +Skills use a three-level loading strategy to minimize context window usage: + +1. **Metadata** (~100 tokens): `name` and `description` loaded at startup for all skills +2. **SKILL.md Body** (<5000 tokens): Loaded when a task matches the skill's description +3. **Bundled Resources**: Loaded on-demand only when referenced (scripts, references, assets) + +### When to Create a Skill vs Updating AGENTS.md + +| Use AGENTS.md for... | Use Skills for... | +| ------------------------------- | ------------------------------- | +| Always-on rules and constraints | On-demand workflows | +| "Always do X, never do Y" | Multi-step repeatable processes | +| Baseline conventions | Specialist domain knowledge | +| Rarely changes | Can be added/refined frequently | + +**Example**: "Use lowercase for skill filenames" → AGENTS.md rule. +"How to run pre-commit checks" → Skill. + +## Core Principles + +### 1. Concise is Key + +**Context window is shared** between system prompt, conversation history, other skills, +and your actual request. Only add context the agent doesn't already have. + +### 2. Set Appropriate Degrees of Freedom + +Match specificity to task fragility: + +- **High freedom** (text-based instructions): multiple approaches valid, context-dependent +- **Medium freedom** (pseudocode): preferred pattern exists, some variation acceptable +- **Low freedom** (specific scripts): operations are fragile, sequence must be followed + +### 3. Anatomy of a Skill + +A skill consists of: + +- **SKILL.md**: Frontmatter (metadata) + body (instructions) +- **Optional bundled resources**: `scripts/`, `references/`, `assets/` + +Keep SKILL.md concise (<500 lines). Move detailed content to reference files. + +### 4. Progressive Disclosure + +Split detailed content into reference files loaded on-demand: + +```markdown +## Advanced Features + +See [specification.md](references/specification.md) for Agent Skills spec. +See [patterns.md](references/patterns.md) for workflow patterns. +``` + +### 5. Content Strategy + +- **Include in SKILL.md**: essential commands and step-by-step workflows +- **Put in `references/`**: detailed descriptions, config options, troubleshooting +- **Link to official docs**: architecture docs, ADRs, contributing guides + +## Skill Creation Process + +### Step 1: Plan the Skill + +Answer: + +- What specific queries should trigger this skill? +- What tasks does it help accomplish? +- Does a similar skill already exist? + +### Step 2: Choose the Location + +Follow the directory layout: + +```text +.github/skills/ + add-new-skill/ + dev/ + git-workflow/ + maintenance/ + planning/ + rust-code-quality/ + testing/ +``` + +### Step 3: Write the SKILL.md + +Frontmatter rules: + +- `name`: lowercase letters, numbers, hyphens only; max 64 chars; no consecutive hyphens +- `description`: max 1024 chars; include trigger phrases; describe WHAT and WHEN +- `metadata.author`: `torrust` +- `metadata.version`: `"1.0"` + +### Step 4: Validate and Commit + +```bash +# Check spelling and markdown +linter cspell +linter markdown + +# Run all linters +linter all + +# Commit +git add .github/skills/ +git commit -S -m "docs(skills): add {skill-name} skill" +``` + +## Directory Layout + +```text +.github/skills/ + / + SKILL.md ← Required + references/ ← Optional: detailed docs + scripts/ ← Optional: executable scripts + assets/ ← Optional: templates, data +``` + +## References + +- Agent Skills specification: [references/specification.md](references/specification.md) +- Skill patterns: [references/patterns.md](references/patterns.md) +- Real examples: [references/examples.md](references/examples.md) diff --git a/.github/skills/add-new-skill/references/specification.md b/.github/skills/add-new-skill/references/specification.md new file mode 100644 index 000000000..90e73b8a6 --- /dev/null +++ b/.github/skills/add-new-skill/references/specification.md @@ -0,0 +1,65 @@ +# Agent Skills Specification Reference + +This document provides a reference to the Agent Skills specification from [agentskills.io](https://agentskills.io). + +## What is Agent Skills? + +Agent Skills is an open format for extending AI agent capabilities with specialized knowledge and +workflows. It's vendor-neutral and works with Claude Code, VS Code Copilot, Cursor, and Windsurf. + +## Core Concepts + +### Progressive Disclosure + +```text +Level 1: Metadata (name + description) - ~100 tokens - Loaded at startup for ALL skills +Level 2: SKILL.md body - <5000 tokens - Loaded when skill matches task +Level 3: Bundled resources - On-demand - Loaded only when referenced +``` + +### Directory Structure + +```text +.github/ +└── skills/ + └── skill-name/ + ├── SKILL.md # Required: frontmatter + instructions + ├── README.md # Optional: human-readable documentation + ├── scripts/ # Optional: executable code + ├── references/ # Optional: detailed docs loaded on-demand + └── assets/ # Optional: templates, images, data +``` + +## SKILL.md Format + +### Frontmatter (YAML) + +```yaml +--- +name: skill-name +description: | + What the skill does and when to use it. Include trigger phrases. +metadata: + author: torrust + version: "1.0" +--- +``` + +### Frontmatter Validation Rules + +**name**: + +- Required; max 64 characters +- Lowercase letters, numbers, hyphens only +- Cannot contain consecutive hyphens or XML tags + +**description**: + +- Required; max 1024 characters +- Should describe WHAT the skill does AND WHEN to use it +- Include trigger phrases/keywords + +## References + +- Official spec: +- GitHub Copilot skills docs: diff --git a/.github/skills/dev/git-workflow/commit-changes/SKILL.md b/.github/skills/dev/git-workflow/commit-changes/SKILL.md new file mode 100644 index 000000000..415ee2895 --- /dev/null +++ b/.github/skills/dev/git-workflow/commit-changes/SKILL.md @@ -0,0 +1,155 @@ +--- +name: commit-changes +description: Guide for committing changes in the torrust-tracker project. Covers conventional commit format, pre-commit verification checklist, GPG signing, and commit quality guidelines. Use when committing code, running pre-commit checks, or following project commit standards. Triggers on "commit", "commit changes", "how to commit", "pre-commit", "commit message", "commit format", or "conventional commits". +metadata: + author: torrust + version: "1.0" +--- + +# Committing Changes + +This skill guides you through the complete commit process for the Torrust Tracker project. + +## Quick Reference + +```bash +# One-time setup: install the pre-commit Git hook +./scripts/install-git-hooks.sh + +# Stage changes +git add + +# Commit with conventional format and GPG signature (MANDATORY) +# The pre-commit hook runs ./scripts/pre-commit.sh automatically +git commit -S -m "[()]: " +``` + +## Conventional Commit Format + +We follow [Conventional Commits](https://www.conventionalcommits.org/) specification. + +### Commit Message Structure + +```text +[optional scope]: + +[optional body] + +[optional footer(s)] +``` + +Scope should reflect the affected package or area (e.g., `tracker-core`, `udp-protocol`, `ci`, `docs`). + +### Commit Types + +| Type | Description | Example | +| ---------- | ------------------------------------- | ------------------------------------------------------------ | +| `feat` | New feature or enhancement | `feat(tracker-core): add peer expiry grace period` | +| `fix` | Bug fix | `fix(udp-protocol): resolve endianness in announce response` | +| `docs` | Documentation changes | `docs(agents): add root AGENTS.md` | +| `style` | Code style changes (formatting, etc.) | `style: apply rustfmt to all source files` | +| `refactor` | Code refactoring | `refactor(tracker-core): extract peer list to own module` | +| `test` | Adding or updating tests | `test(http-tracker-core): add announce response tests` | +| `chore` | Maintenance tasks | `chore: update dependencies` | +| `ci` | CI/CD related changes | `ci: add workflow for container publishing` | +| `perf` | Performance improvements | `perf(torrent-repository): switch to dashmap` | + +## GPG Commit Signing (MANDATORY) + +**All commits must be GPG signed.** Use the `-S` flag: + +```bash +git commit -S -m "your commit message" +``` + +## Pre-commit Verification (MANDATORY) + +### Git Hook + +The repository ships a `pre-commit` Git hook that runs `./scripts/pre-commit.sh` +automatically on every `git commit`. Install it once after cloning: + +```bash +./scripts/install-git-hooks.sh +``` + +Once installed, the hook fires on every commit and you do not need to run the script manually. + +### Automated Checks + +If the hook is not installed, run the script explicitly before committing. +**It must exit with code `0`.** + +> **⏱️ Expected runtime: ~3 minutes** on a modern developer machine. AI agents must set a +> command timeout of **at least 5 minutes** before invoking this script. + +```bash +./scripts/pre-commit.sh +``` + +The script runs: + +1. `cargo machete` — unused dependency check +2. `linter all` — all linters (markdown, YAML, TOML, clippy, rustfmt, shellcheck, cspell) +3. `cargo test --doc --workspace` — documentation tests +4. `cargo test --tests --benches --examples --workspace --all-targets --all-features` — all tests + +### Manual Checks (Cannot Be Automated) + +Verify these by hand before committing: + +- **Self-review the diff**: read through `git diff --staged` and check for obvious mistakes, + debug artifacts, or unintended changes +- **Documentation updated**: if public API or behaviour changed, doc comments and any relevant + `docs/` pages reflect the change +- **`AGENTS.md` updated**: if architecture, package structure, or key workflows changed, the + relevant `AGENTS.md` file is updated +- **New technical terms added to `project-words.txt`**: any new jargon or identifiers that + cspell does not know about are added alphabetically + +### Debugging a Failing Run + +```bash +linter markdown # Markdown +linter yaml # YAML +linter toml # TOML +linter clippy # Rust code analysis +linter rustfmt # Rust formatting +linter shellcheck # Shell scripts +linter cspell # Spell checking +``` + +Fix Rust formatting automatically: + +```bash +cargo fmt +``` + +## Hashtag Usage Warning + +**Only use `#` when intentionally referencing a GitHub issue.** + +GitHub auto-links `#NUMBER` to issues. Avoid accidental references: + +- ✅ `feat(tracker-core): add feature (see #42)` — intentional reference +- ❌ `fix: make feature #1 priority` — accidentally links to issue #1 + +Use ordered Markdown lists or plain numbers instead of `#N` step labels. + +## Commit Quality Guidelines + +### Good Commits (✅) + +- **Atomic**: Each commit represents one logical change +- **Descriptive**: Clear, concise description of what changed +- **Tested**: All tests pass +- **Linted**: All linters pass +- **Conventional**: Follows conventional commit format +- **Signed**: GPG signature present + +### Commits to Avoid (❌) + +- Too large: multiple unrelated changes in one commit +- Vague messages like "fix stuff" or "WIP" +- Missing scope when a package is clearly affected +- Unsigned commits diff --git a/.github/skills/dev/git-workflow/create-feature-branch/SKILL.md b/.github/skills/dev/git-workflow/create-feature-branch/SKILL.md new file mode 100644 index 000000000..bb2c82a55 --- /dev/null +++ b/.github/skills/dev/git-workflow/create-feature-branch/SKILL.md @@ -0,0 +1,113 @@ +--- +name: create-feature-branch +description: Guide for creating feature branches following the torrust-tracker branching conventions. Covers branch naming format, lifecycle, and common patterns. Use when creating branches for issues, starting work on tasks, or setting up development branches. Triggers on "create branch", "new branch", "checkout branch", "branch for issue", or "start working on issue". +metadata: + author: torrust + version: "1.0" +--- + +# Creating Feature Branches + +This skill guides you through creating feature branches following the Torrust Tracker branching +conventions. + +## Branch Naming Convention + +**Format**: `{issue-number}-{short-description}` (preferred) + +Alternative formats (no tracked issue): + +- `feat/{short-description}` +- `fix/{short-description}` +- `chore/{short-description}` + +**Rules**: + +- Always start with the GitHub issue number when one exists +- Use lowercase letters only +- Separate words with hyphens (not underscores) +- Keep description concise but descriptive + +## Creating a Branch + +### Standard Workflow + +```bash +# Ensure you're on latest develop +git checkout develop +git pull --ff-only + +# Create and checkout branch for issue #42 +git checkout -b 42-add-peer-expiry-grace-period +``` + +### With MCP GitHub Tools + +1. Get the issue number and title +2. Format the branch name: `{number}-{kebab-case-description}` +3. Create the branch from `develop` +4. Checkout locally: `git fetch && git checkout {branch-name}` + +## Branch Naming Examples + +✅ **Good branch names**: + +- `42-add-peer-expiry-grace-period` +- `156-refactor-udp-server-socket-binding` +- `203-add-e2e-mysql-tests` +- `1697-ai-agent-configuration` + +❌ **Avoid**: + +- `my-feature` — no issue number +- `FEATURE-123` — all caps +- `fix_bug` — underscores instead of hyphens +- `42_add_support` — underscores + +## Complete Branch Lifecycle + +### 1. Create Branch from `develop` + +```bash +git checkout develop +git pull --ff-only +git checkout -b 42-add-peer-expiry-grace-period +``` + +### 2. Develop + +Make commits following [commit conventions](../commit-changes/SKILL.md). + +### 3. Pre-commit Checks + +```bash +cargo machete +linter all +cargo test --doc --workspace +cargo test --tests --benches --examples --workspace --all-targets --all-features +``` + +### 4. Push to Your Fork + +```bash +git push {your-fork-remote} 42-add-peer-expiry-grace-period +``` + +### 5. Create Pull Request + +Target branch: `torrust/torrust-tracker:develop` + +### 6. Cleanup After Merge + +```bash +git checkout develop +git pull --ff-only +git branch -d 42-add-peer-expiry-grace-period +``` + +## Converting Issue Title to Branch Name + +1. Get issue number (e.g., #42) +2. Take issue title (e.g., "Add Peer Expiry Grace Period") +3. Convert to lowercase kebab-case: `add-peer-expiry-grace-period` +4. Prefix with issue number: `42-add-peer-expiry-grace-period` diff --git a/.github/skills/dev/git-workflow/open-pull-request/SKILL.md b/.github/skills/dev/git-workflow/open-pull-request/SKILL.md new file mode 100644 index 000000000..eca0fae3b --- /dev/null +++ b/.github/skills/dev/git-workflow/open-pull-request/SKILL.md @@ -0,0 +1,73 @@ +--- +name: open-pull-request +description: Open a pull request from a feature branch using GitHub CLI (preferred) or GitHub MCP tools. Covers pre-flight checks, correct base/head configuration for fork workflows, title/body conventions, and post-creation validation. Use when asked to "open PR", "create pull request", or "submit branch for review". +metadata: + author: torrust + version: "1.0" +--- + +# Open a Pull Request + +## CLI vs MCP Decision Rule + +- **Inner loop (fast local branch work):** prefer GitHub CLI (`gh pr create`). +- **Outer loop (cross-system coordination):** use MCP tools for structured/authenticated access. + +## Pre-flight Checks + +Before opening a PR: + +- [ ] Working tree is clean (`git status`) +- [ ] Branch is pushed to your fork remote +- [ ] Commits are GPG signed (`git log --show-signature -n 1`) +- [ ] All pre-commit checks passed (`linter all`, `cargo machete`, tests) + +## Title and Description Convention + +PR title: use Conventional Commit style, include issue reference. + +Examples: + +- `feat(tracker-core): [#42] add peer expiry grace period` +- `docs(agents): set up basic AI agent configuration (#1697)` + +PR body must include: + +- Summary of changes +- Files/packages touched +- Validation performed +- Issue link (`Closes #`) + +## Option A (Preferred): GitHub CLI + +```bash +gh pr create \ + --repo torrust/torrust-tracker \ + --base develop \ + --head : \ + --title "" \ + --body "<body>" +``` + +If successful, `gh` prints the PR URL. + +## Option B: GitHub MCP Tools + +When MCP pull request management tools are available, create the PR with: + +- `base`: `develop` +- `head`: `<fork-owner>:<branch-name>` +- Capture and share the resulting PR URL. + +## Post-creation Validation + +- [ ] PR targets `torrust/torrust-tracker:develop` +- [ ] Head branch is correct +- [ ] CI workflows started +- [ ] Issue linked in description + +## Troubleshooting + +- `fatal: ... does not appear to be a git repository`: push to correct remote (`git remote -v`) +- `A pull request already exists`: open existing PR URL instead of creating new +- Permission errors on upstream: use `owner:branch` fork syntax diff --git a/.github/skills/dev/git-workflow/release-new-version/SKILL.md b/.github/skills/dev/git-workflow/release-new-version/SKILL.md new file mode 100644 index 000000000..f30898511 --- /dev/null +++ b/.github/skills/dev/git-workflow/release-new-version/SKILL.md @@ -0,0 +1,147 @@ +--- +name: release-new-version +description: Guide for releasing a new version of the Torrust Tracker using the standard staging branch, tag, and crate publication workflow. Covers version bump, release commit, staging branch promotion, PR to main, release branch/tag creation, crate publication, and merge-back to develop. Use when asked to "release", "cut a version", "publish a new version", or "create release vX.Y.Z". +metadata: + author: torrust + version: "1.0" +--- + +# Release New Version + +Primary reference: [`docs/release_process.md`](../../../../../docs/release_process.md) + +## Release Steps (Mandatory Order) + +1. Stage `develop` → `staging/main` +2. Create release commit (bump version) +3. PR `staging/main` → `main` +4. Push `main` → `releases/vX.Y.Z` +5. Create signed tag `vX.Y.Z` on that branch +6. Verify deployment workflow + crate publication +7. Create GitHub release +8. Stage `main` → `staging/develop` (merge-back) +9. Bump next dev version, PR `staging/develop` → `develop` + +Do not reorder these steps. + +## Version Naming Rules + +- Version in code: `X.Y.Z` (release) or `X.Y.Z-develop` (development) +- Git tag: `vX.Y.Z` +- Release branch: `releases/vX.Y.Z` +- Staging branches: `staging/main`, `staging/develop` + +## Pre-Flight Checklist + +Before starting: + +- [ ] Clean working tree (`git status`) +- [ ] `develop` branch is up to date with `torrust/develop` +- [ ] All CI checks pass on `develop` +- [ ] Working version in manifests is `X.Y.Z-develop` + +## Commands + +### 1) Stage develop → staging/main + +```bash +git fetch --all +git push --force torrust develop:staging/main +``` + +### 2) Create Release Commit + +```bash +git stash +git switch staging/main +git reset --hard torrust/staging/main +# Edit version in all Cargo.toml files: +# change X.Y.Z-develop → X.Y.Z +git add -A +git commit -S -m "release: version X.Y.Z" +git push torrust +``` + +Edit `version` in: + +- `Cargo.toml` (workspace) +- All packages under `packages/` that publish crates +- `console/tracker-client/Cargo.toml` +- `contrib/bencode/Cargo.toml` + +Also update any internal path dependency `version` constraints. + +### 3) PR staging/main → main + +Create PR: "Release Version X.Y.Z" (title format) +Base: `torrust/torrust-tracker:main` +Head: `staging/main` +Merge after CI passes. + +### 4) Push releases/vX.Y.Z branch + +```bash +git fetch --all +git push torrust main:releases/vX.Y.Z +``` + +### 5) Create Signed Tag + +```bash +git switch releases/vX.Y.Z +git reset --hard torrust/releases/vX.Y.Z +git tag --sign vX.Y.Z +git push --tags torrust +``` + +### 6) Verify Deployment Workflow + +Check the +[deployment workflow](https://github.com/torrust/torrust-tracker/actions/workflows/deployment.yaml) +ran successfully and the following crates were published: + +- `torrust-tracker-contrib-bencode` +- `torrust-tracker-located-error` +- `torrust-tracker-primitives` +- `torrust-tracker-clock` +- `torrust-tracker-configuration` +- `torrust-tracker-torrent-repository` +- `torrust-tracker-test-helpers` +- `torrust-tracker` + +Crates must be published in dependency order. Each must be indexed on crates.io before the next +publishes. + +### 7) Create GitHub Release + +Create a release from tag `vX.Y.Z` after the deployment workflow passes. + +### 8) Merge-back: Stage main → staging/develop + +```bash +git fetch --all +git push --force torrust main:staging/develop +``` + +### 9) Bump Next Dev Version + +```bash +git stash +git switch staging/develop +git reset --hard torrust/staging/develop +# Edit version in all Cargo.toml files: +# change X.Y.Z → (next)X.Y.Z-develop (e.g. 3.0.0 → 3.0.1-develop) +git add -A +git commit -S -m "develop: bump to version (next)X.Y.Z-develop" +git push torrust +``` + +Create PR: "Version X.Y.Z was Released" +Base: `torrust/torrust-tracker:develop` +Head: `staging/develop` + +## Failure Handling + +- **Deployment workflow failed**: fix and rerun on same release branch +- **Crate already published**: do not republish; cut a patch release +- **Partial state (tag exists but branch doesn't)**: investigate before proceeding diff --git a/.github/skills/dev/git-workflow/review-pr/SKILL.md b/.github/skills/dev/git-workflow/review-pr/SKILL.md new file mode 100644 index 000000000..da4be9ca3 --- /dev/null +++ b/.github/skills/dev/git-workflow/review-pr/SKILL.md @@ -0,0 +1,66 @@ +--- +name: review-pr +description: Review a pull request for the torrust-tracker project. Covers checklist-based PR quality verification, code style standards, test requirements, documentation, and how to submit review feedback. Use when asked to review a PR, check a pull request, or provide feedback on code changes. Triggers on "review PR", "review pull request", "check PR quality", or "code review". +metadata: + author: torrust + version: "1.0" +--- + +# Reviewing a Pull Request + +## Quick Overview Approach + +1. Read the PR title and description for context +2. Check the diff for scope of change +3. Identify the affected packages and components +4. Apply the checklist below + +## PR Review Checklist + +### PR Metadata + +- [ ] Title follows Conventional Commits format +- [ ] Description clearly explains what changes were made and why +- [ ] Issue is linked (`Closes #<number>` or `Refs #<number>`) +- [ ] Target branch is `develop` (not `main`) + +### Code Quality + +- [ ] Code follows existing patterns in affected packages +- [ ] No unused imports, variables, or functions +- [ ] No `#[allow(...)]` suppressions unless clearly justified with a comment +- [ ] Errors handled properly (use `thiserror` for structured errors, avoid `.unwrap()`) +- [ ] No security vulnerabilities (OWASP Top 10 awareness) + +### Tests + +- [ ] New functionality has unit tests +- [ ] Integration tests added if applicable +- [ ] All existing tests still pass +- [ ] Test code is clean, readable, and maintainable + +### Documentation + +- [ ] Public API items have doc comments +- [ ] `AGENTS.md` updated if architecture changed +- [ ] Markdown docs updated if user-facing behavior changed +- [ ] Spell check: new technical terms added to `project-words.txt` + +### Rust-Specific + +- [ ] Imports grouped: std → external → internal +- [ ] Line length within `max_width = 130` +- [ ] GPG-signed commits + +## Providing Feedback + +Categorize comments to help the author prioritize: + +- **Blocker** — must fix before merge (correctness, security, breaking changes) +- **Suggestion** — improvement recommended but not blocking +- **Nit** — minor style/readability point + +## Standards Reference + +All code quality standards are defined in the root `AGENTS.md`. When pointing to a +standard, reference the relevant section of `AGENTS.md`. diff --git a/.github/skills/dev/git-workflow/run-linters/SKILL.md b/.github/skills/dev/git-workflow/run-linters/SKILL.md new file mode 100644 index 000000000..c779b413f --- /dev/null +++ b/.github/skills/dev/git-workflow/run-linters/SKILL.md @@ -0,0 +1,121 @@ +--- +name: run-linters +description: Run code quality checks and linters for the torrust-tracker project. Includes Rust clippy, rustfmt, markdown, YAML, TOML, spell checking, and shellcheck. Use when asked to lint code, check formatting, fix code quality issues, or prepare for commit. Triggers on "lint", "run linters", "check code quality", "fix formatting", "run clippy", "run rustfmt", or "pre-commit checks". +metadata: + author: torrust + version: "1.0" +--- + +# Run Linters + +## Quick Reference + +### Run All Linters + +```bash +linter all +``` + +**Always run `linter all` before every commit. It must exit with code `0`.** + +### Run a Single Linter + +```bash +linter markdown # Markdown (markdownlint) +linter yaml # YAML (yamllint) +linter toml # TOML (taplo) +linter cspell # Spell checker (cspell) +linter clippy # Rust code analysis (clippy) +linter rustfmt # Rust formatting (rustfmt) +linter shellcheck # Shell scripts (shellcheck) +``` + +## Common Workflows + +### Before Any Commit + +```bash +linter all # Must pass with exit code 0 +``` + +### Debug a Failing Full Run + +```bash +# Identify which linter is failing +linter markdown +linter yaml +linter toml +linter cspell +linter clippy +linter rustfmt +linter shellcheck +``` + +### During Development (Rust only) + +```bash +linter clippy # Check logic and code quality +linter rustfmt # Check formatting +``` + +## Fixing Common Issues + +### Rust Formatting Errors (rustfmt) + +```bash +cargo fmt # Auto-fix all Rust source files +``` + +Formatting rules from `rustfmt.toml`: + +- `max_width = 130` +- `group_imports = "StdExternalCrate"` +- `imports_granularity = "Module"` + +### Rust Clippy Errors + +Warnings are **errors** (configured as `-D warnings` in `.cargo/config.toml`). +Fix the underlying issue — do not `#[allow(...)]` unless truly unavoidable. + +Example: unused variable → use `_var` prefix or actually use the value. + +### Markdown Errors (markdownlint) + +Common issues: + +- Trailing whitespace +- Missing blank line before headings +- Incorrect heading levels +- Lines exceeding 120 characters + +Configuration in `.markdownlint.json`. + +### YAML Errors (yamllint) + +Common issues: + +- Trailing spaces +- Inconsistent indentation (2 spaces expected) +- Missing newline at end of file + +Configuration in `.yamllint-ci.yml`. + +### TOML Errors (taplo) + +```bash +taplo fmt **/*.toml # Auto-fix TOML formatting +``` + +### Spell Check Errors (cspell) + +For legitimate technical terms not in dictionaries, add them to `project-words.txt` +(alphabetical order, one per line). + +### Shell Script Errors (shellcheck) + +Fix the reported issue in the shell script. Common: use `[[ ]]` instead of `[ ]`, +quote variables, avoid `eval`. + +## Linter Details + +See [references/linters.md](references/linters.md) for detailed documentation on each linter. diff --git a/.github/skills/dev/git-workflow/run-linters/references/linters.md b/.github/skills/dev/git-workflow/run-linters/references/linters.md new file mode 100644 index 000000000..40b3ee5fb --- /dev/null +++ b/.github/skills/dev/git-workflow/run-linters/references/linters.md @@ -0,0 +1,85 @@ +# Linter Documentation + +This document provides detailed documentation for each linter used in the Torrust Tracker project. + +## Overview + +The project uses the `linter` binary from +[torrust/torrust-linting](https://github.com/torrust/torrust-linting) as a unified wrapper around +all linters. + +Install: `cargo install --locked --git https://github.com/torrust/torrust-linting --bin linter` + +## Rust Linters + +### clippy + +**Tool**: Rust's official linter. +**Config**: `.cargo/config.toml` (global `rustflags`) +**Run**: `linter clippy` + +Warnings are treated as errors via `-D warnings` in `.cargo/config.toml`. +Do not suppress warnings with `#[allow(...)]` unless absolutely necessary. + +**Critical flags** (from `.cargo/config.toml`): + +- `-D warnings` — all warnings are errors +- `-D unused` — unused items are errors +- `-D rust-2018-idioms` — enforces Rust 2018 idioms +- `-D future-incompatible` + +### rustfmt + +**Tool**: Rust code formatter. +**Config**: `rustfmt.toml` +**Run**: `linter rustfmt` +**Auto-fix**: `cargo fmt` + +Key formatting settings: + +- `max_width = 130` +- `group_imports = "StdExternalCrate"` +- `imports_granularity = "Module"` + +## Documentation Linters + +### markdownlint + +**Tool**: markdownlint +**Config**: `.markdownlint.json` +**Run**: `linter markdown` + +### cspell (Spell Checker) + +**Tool**: cspell +**Config**: `cspell.json` +**Dictionary**: `project-words.txt` +**Run**: `linter cspell` + +Add technical terms to `project-words.txt` (alphabetical order, one per line). + +## Configuration Linters + +### yamllint + +**Tool**: yamllint +**Config**: `.yamllint-ci.yml` +**Run**: `linter yaml` + +Expected: 2-space indentation, no trailing whitespace, newline at EOF. + +### taplo + +**Tool**: taplo +**Config**: `.taplo.toml` +**Run**: `linter toml` +**Auto-fix**: `taplo fmt **/*.toml` + +## Script Linters + +### shellcheck + +**Tool**: shellcheck +**Run**: `linter shellcheck` + +Checks all shell scripts. Use `[[ ]]` over `[ ]`, quote variables (`"$var"`), and avoid `eval`. diff --git a/.github/skills/dev/git-workflow/run-pre-commit-checks/SKILL.md b/.github/skills/dev/git-workflow/run-pre-commit-checks/SKILL.md new file mode 100644 index 000000000..b0eb24e4d --- /dev/null +++ b/.github/skills/dev/git-workflow/run-pre-commit-checks/SKILL.md @@ -0,0 +1,88 @@ +--- +name: run-pre-commit-checks +description: Run all mandatory pre-commit verification steps for the torrust-tracker project. Covers the pre-commit script (automated checks), manual review steps, and individual linter commands for debugging. Use before any commit or PR to ensure all quality gates pass. Triggers on "pre-commit checks", "run all checks", "verify before commit", or "check everything". +metadata: + author: torrust + version: "1.0" +--- + +# Run Pre-commit Checks + +## Git Hook (Recommended Setup) + +The repository ships a `pre-commit` Git hook that runs `./scripts/pre-commit.sh` +automatically on every `git commit`. Install it once after cloning: + +```bash +./scripts/install-git-hooks.sh +``` + +After installation the hook fires automatically; you do not need to invoke the script +manually before each commit. + +## Automated Checks + +> **⏱️ Expected runtime: ~3 minutes** on a modern developer machine. AI agents must set a +> command timeout of **at least 5 minutes** before invoking `./scripts/pre-commit.sh`. Agents +> with a default per-command timeout below 5 minutes will likely time out and report a false +> failure. + +Run the pre-commit script. **It must exit with code `0` before every commit.** + +```bash +./scripts/pre-commit.sh +``` + +The script runs these steps in order: + +1. `cargo machete` — unused dependency check +2. `linter all` — all linters (markdown, YAML, TOML, clippy, rustfmt, shellcheck, cspell) +3. `cargo test --doc --workspace` — documentation tests +4. `cargo test --tests --benches --examples --workspace --all-targets --all-features` — all tests + +> **MySQL tests**: MySQL-specific tests require a running instance and a feature flag: +> +> ```bash +> TORRUST_TRACKER_CORE_RUN_MYSQL_DRIVER_TEST=true cargo test --package bittorrent-tracker-core +> ``` +> +> These are not run by the pre-commit script. + +## Manual Checks (Cannot Be Automated) + +Verify these by hand before committing: + +- **Self-review the diff**: read through `git diff --staged` for debug artifacts or unintended changes +- **Documentation updated**: if public API or behaviour changed, doc comments and `docs/` pages reflect it +- **`AGENTS.md` updated**: if architecture or key workflows changed, the relevant `AGENTS.md` is updated +- **New technical terms in `project-words.txt`**: new jargon added alphabetically + +## Before Opening a PR (Recommended) + +```bash +cargo +nightly doc --no-deps --bins --examples --workspace --all-features +``` + +## Debugging Individual Linters + +Run individual linters to isolate a failure: + +```bash +linter markdown # Markdown +linter yaml # YAML +linter toml # TOML +linter clippy # Rust code analysis +linter rustfmt # Rust formatting +linter shellcheck # Shell scripts +linter cspell # Spell checking +``` + +| Failure | Fix | +| ------------------- | --------------------------------------- | +| Unused dependency | Remove from `Cargo.toml` | +| Clippy warning | Fix the underlying issue | +| rustfmt error | Run `cargo fmt` | +| Markdown lint error | Fix formatting per `.markdownlint.json` | +| Spell check error | Add term to `project-words.txt` | +| Test failure | Fix the failing test or code | +| Doc build error | Fix Rust doc comment | diff --git a/.github/skills/dev/maintenance/install-linter/SKILL.md b/.github/skills/dev/maintenance/install-linter/SKILL.md new file mode 100644 index 000000000..9112acd31 --- /dev/null +++ b/.github/skills/dev/maintenance/install-linter/SKILL.md @@ -0,0 +1,62 @@ +--- +name: install-linter +description: Install the torrust-linting `linter` binary and its external tool dependencies. Use when setting up a new development environment, after a fresh clone, or when the `linter` binary is missing. Triggers on "install linter", "setup linter", "linter not found", "install torrust-linting", "missing linter binary", or "set up development environment". +metadata: + author: torrust + version: "1.0" +--- + +# Install the Linter + +The project uses a unified `linter` binary from +[torrust/torrust-linting](https://github.com/torrust/torrust-linting) to run all quality checks. + +## Install the `linter` Binary + +```bash +cargo install --locked --git https://github.com/torrust/torrust-linting --bin linter +``` + +Verify the installation: + +```bash +linter --version +``` + +## Install External Tool Dependencies + +The `linter` binary delegates to external tools. Install them if they are not already present: + +| Linter | Tool | Install command | +| ----------- | ---------------- | ------------------------------------- | +| Markdown | markdownlint-cli | `npm install -g markdownlint-cli` | +| YAML | yamllint | `pip3 install yamllint` | +| TOML | taplo | `cargo install taplo-cli --locked` | +| Spell check | cspell | `npm install -g cspell` | +| Shell | shellcheck | `apt install shellcheck` | +| Rust | clippy / rustfmt | bundled with `rustup` (no extra step) | + +> The `linter` binary will attempt to install missing npm-based tools automatically on first run. +> System-packaged tools (`yamllint`, `shellcheck`) must be installed manually. + +## Configuration Files + +The linters read configuration from files in the project root. These are already present in the +repository — no manual setup is needed: + +| File | Used by | +| -------------------- | ------------ | +| `.markdownlint.json` | markdownlint | +| `.yamllint-ci.yml` | yamllint | +| `.taplo.toml` | taplo | +| `cspell.json` | cspell | + +## Verify Full Setup + +After installing the binary and its dependencies, run all linters to confirm everything works: + +```bash +linter all +``` + +It must exit with code `0`. See the `run-linters` skill for day-to-day usage. diff --git a/.github/skills/dev/maintenance/setup-dev-environment/SKILL.md b/.github/skills/dev/maintenance/setup-dev-environment/SKILL.md new file mode 100644 index 000000000..1228611b5 --- /dev/null +++ b/.github/skills/dev/maintenance/setup-dev-environment/SKILL.md @@ -0,0 +1,123 @@ +--- +name: setup-dev-environment +description: Set up a local development environment for torrust-tracker from scratch. Covers system dependencies, Rust toolchain, storage directories, linter binary, git hooks, and smoke tests. Use when onboarding to the project, setting up a new machine, or after a fresh clone. Triggers on "setup dev environment", "fresh clone", "onboarding", "install dependencies", "set up environment", or "getting started". +metadata: + author: torrust + version: "1.0" +--- + +# Set Up the Development Environment + +Full setup guide for a fresh clone of `torrust-tracker`. Follow the steps in order. + +Reference: [How to Set Up the Development Environment](https://torrust.com/blog/how-to-setup-the-development-environment) + +## Step 1: System Dependencies + +Install the required system packages (Debian/Ubuntu): + +```bash +sudo apt-get install libsqlite3-dev pkg-config libssl-dev make +``` + +> For other distributions, install the equivalent packages for SQLite3 development headers, OpenSSL +> development headers, `pkg-config`, and `make`. + +## Step 2: Rust Toolchain + +```bash +rustup show # Confirm toolchain is active +rustup update # Update to latest stable +rustup toolchain install nightly # Required for docs generation +``` + +The project MSRV is **1.72**. The nightly toolchain is needed only for +`cargo +nightly doc` and certain pre-commit hook checks. + +## Step 3: Build + +```bash +cargo build +``` + +This compiles all workspace crates and verifies that all dependencies resolve correctly. + +## Step 4: Create Storage Directories + +The tracker writes runtime data (databases, logs, TLS certs, config) to `storage/`, which is +git-ignored. Create the required folders once: + +```bash +mkdir -p ./storage/tracker/lib/database +mkdir -p ./storage/tracker/lib/tls +mkdir -p ./storage/tracker/etc +``` + +## Step 5: Install the Linter Binary + +```bash +cargo install --locked --git https://github.com/torrust/torrust-linting --bin linter +``` + +See the `install-linter` skill for external tool dependencies (markdownlint, yamllint, etc.). + +## Step 6: Install Additional Cargo Tools + +```bash +cargo install cargo-machete # Unused dependency checker +``` + +## Step 7: Install Git Hooks + +Install the project pre-commit hook (one-time, re-run after hook changes): + +```bash +./scripts/install-git-hooks.sh +``` + +The hook runs `./scripts/pre-commit.sh` automatically on every `git commit`. + +## Step 8: Smoke Test + +Run the tracker with the default development configuration to confirm the build works: + +```bash +cargo run +``` + +Expected output includes lines like: + +```text +Loading configuration from default configuration file: `./share/default/config/tracker.development.sqlite3.toml` +[UDP TRACKER] Starting on: udp://0.0.0.0:6969 +[HTTP TRACKER] Started on: http://0.0.0.0:7070 +[API] Started on http://127.0.0.1:1212 +[HEALTH CHECK API] Started on: http://127.0.0.1:1313 +``` + +Press `Ctrl-C` to stop. + +## Step 9: Verify Full Test Suite + +```bash +cargo test --doc --workspace +cargo test --tests --benches --examples --workspace --all-targets --all-features +``` + +Both commands must exit `0` before any commit. + +## Custom Configuration (Optional) + +To run with a custom config instead of the default template: + +```bash +cp share/default/config/tracker.development.sqlite3.toml storage/tracker/etc/tracker.toml +# Edit storage/tracker/etc/tracker.toml as needed +TORRUST_TRACKER_CONFIG_TOML_PATH="./storage/tracker/etc/tracker.toml" cargo run +``` + +## Useful Development Tools + +- **DB Browser for SQLite** — inspect and edit SQLite databases: <https://sqlitebrowser.org/> +- **qBittorrent** — BitTorrent client for manual testing: <https://www.qbittorrent.org/> +- **imdl** — torrent file editor (`cargo install imdl`): <https://github.com/casey/intermodal> diff --git a/.github/skills/dev/maintenance/update-dependencies/SKILL.md b/.github/skills/dev/maintenance/update-dependencies/SKILL.md new file mode 100644 index 000000000..c0aa1c867 --- /dev/null +++ b/.github/skills/dev/maintenance/update-dependencies/SKILL.md @@ -0,0 +1,120 @@ +--- +name: update-dependencies +description: Guide for updating project dependencies in the torrust-tracker project. Covers the manual cargo update workflow including branch creation, running checks, committing, and pushing. Distinguishes trivial updates (Cargo.lock only) from breaking-change updates (code rework needed). Use when updating dependencies, running cargo update, or bumping deps. Triggers on "update dependencies", "cargo update", "update deps", or "bump dependencies". +metadata: + author: torrust + version: "1.0" +--- + +# Updating Dependencies + +This skill guides you through updating project dependencies for the Torrust Tracker project. + +## Update Categories + +Before starting, decide which category the update falls into: + +| Category | Description | Branch / Issue | +| ------------ | -------------------------------------------- | -------------------------------------------------------------- | +| **Trivial** | `cargo update` only — no code changes needed | Timestamped branch, no issue required | +| **Breaking** | Dependency change requires code rework | If small: same branch. If large: open a separate issue per dep | + +Use `cargo update --dry-run` or read the dependency changelog to classify before starting. + +## Quick Reference + +```bash +# Get a timestamp (YYYYMMDD) +TIMESTAMP=$(date +%Y%m%d) + +# Create branch +git checkout develop && git pull --ff-only +git checkout -b "${TIMESTAMP}-update-dependencies" + +# Update dependencies +cargo update 2>&1 | tee /tmp/cargo-update.txt + +# If Cargo.lock has no changes, nothing to do — stop here. + +# Verify +./scripts/pre-commit.sh + +# Commit and push +git add Cargo.lock +git commit -S -m "chore: update dependencies" -m "$(cat /tmp/cargo-update.txt)" +git push {your-fork-remote} "${TIMESTAMP}-update-dependencies" +``` + +## Complete Workflow + +### Step 1: Create a Branch + +Generate a timestamp prefix to avoid branch name conflicts across repeated runs: + +```bash +TIMESTAMP=$(date +%Y%m%d) +git checkout develop +git pull --ff-only +git checkout -b "${TIMESTAMP}-update-dependencies" +``` + +For breaking-change updates that require a tracked issue: + +```bash +git checkout -b {issue-number}-update-dependencies +``` + +### Step 2: Run Cargo Update + +```bash +cargo update 2>&1 | tee /tmp/cargo-update.txt +``` + +If `Cargo.lock` has no changes, there is nothing to update — exit early. + +Review `/tmp/cargo-update.txt` to identify any major version bumps that may be breaking. + +### Step 3: Handle Breaking Changes + +If any updated dependency introduced a breaking API change: + +- **Small rework** (a few lines, no design decisions): fix it in this branch and continue. +- **Large rework** (architectural impact or significant effort): revert that specific dependency + in `Cargo.toml`, keep the other trivial updates, and open a new issue for the breaking + dependency separately. + +```bash +# Revert a single crate to its current locked version to defer it +cargo update --precise {old-version} {crate-name} +``` + +### Step 4: Verify + +```bash +cargo machete +./scripts/pre-commit.sh +``` + +Fix any failures before proceeding. + +### Step 5: Commit and Push + +```bash +git add Cargo.lock +git commit -S -m "chore: update dependencies" -m "$(cat /tmp/cargo-update.txt)" +git push {your-fork-remote} "${TIMESTAMP}-update-dependencies" +``` + +### Step 6: Open PR + +Target: `torrust/torrust-tracker:develop` +Title: `chore: update dependencies` + +## Decision Guide + +| Scenario | Action | +| ---------------------------------------------- | ---------------------------------------------------------- | +| `cargo update` with no code changes | Trivial — timestamped branch, no issue | +| Breaking change, small rework (< 1 hour) | Fix in the same branch, note in PR description | +| Breaking change, large rework (> 1 hour) | Defer: revert that dep, open a separate issue, separate PR | +| Multiple breaking deps, independent migrations | One issue + PR per dependency to keep diffs reviewable | diff --git a/.github/skills/dev/planning/cleanup-completed-issues/SKILL.md b/.github/skills/dev/planning/cleanup-completed-issues/SKILL.md new file mode 100644 index 000000000..a4c7b3966 --- /dev/null +++ b/.github/skills/dev/planning/cleanup-completed-issues/SKILL.md @@ -0,0 +1,88 @@ +--- +name: cleanup-completed-issues +description: Guide for cleaning up completed and closed issues in the torrust-tracker project. Covers removing issue documentation files from docs/issues/ and committing the cleanup. Supports single issue cleanup or batch cleanup. Use when cleaning up closed issues, removing issue docs, or maintaining the docs/issues/ folder. Triggers on "cleanup issue", "remove issue", "clean completed issues", "delete closed issue", or "maintain issue docs". +metadata: + author: torrust + version: "1.0" +--- + +# Cleaning Up Completed Issues + +## When to Clean Up + +- **After PR merge**: Remove the issue file when its PR is merged +- **Batch cleanup**: Periodically clean up multiple closed issues during maintenance +- **Before releases**: Tidy documentation before major releases + +## Cleanup Approaches + +### Option 1: Single Issue Cleanup (Recommended) + +1. Verify the issue is closed on GitHub +2. Remove the issue file from `docs/issues/` +3. Commit and push changes + +### Option 2: Batch Cleanup + +1. List all issue files in `docs/issues/` +2. Check status of each issue on GitHub +3. Remove all closed issue files +4. Commit and push with a descriptive message + +## Step-by-Step Process + +### Step 1: Verify Issue is Closed on GitHub + +**Single issue:** + +```bash +gh issue view {issue-number} --json state --jq .state +``` + +Expected: `CLOSED` + +**Batch:** + +```bash +for issue in 21 22 23 24; do + state=$(gh issue view "$issue" --json state --jq .state 2>/dev/null || echo "NOT_FOUND") + echo "$issue:$state" +done +``` + +### Step 2: Remove Issue Documentation File + +```bash +# Single issue +git rm docs/issues/42-add-peer-expiry-grace-period.md + +# Batch +git rm docs/issues/21-some-old-issue.md \ + docs/issues/22-another-old-issue.md +``` + +### Step 3: Commit and Push + +```bash +# Single issue +git commit -S -m "chore(issues): remove closed issue #42 documentation" + +# Batch +git commit -S -m "chore(issues): remove documentation for closed issues #21, #22, #23" + +git push {your-fork-remote} {branch} +``` + +## Determining If an Issue File Should Stay + +Keep issue files when: + +- The issue is still open +- The PR is open (still being worked on) +- The specification is referenced from other active docs + +Remove issue files when: + +- The issue is **closed** +- The implementing PR is **merged** +- The file is no longer referenced by active work diff --git a/.github/skills/dev/planning/create-adr/SKILL.md b/.github/skills/dev/planning/create-adr/SKILL.md new file mode 100644 index 000000000..930a4bfc9 --- /dev/null +++ b/.github/skills/dev/planning/create-adr/SKILL.md @@ -0,0 +1,112 @@ +--- +name: create-adr +description: Guide for creating Architectural Decision Records (ADRs) in the torrust-tracker project. Covers the timestamp-based file naming convention, free-form structure, index registration in the docs/adrs/README.md index table, and commit workflow. Use when documenting architectural decisions, recording design choices, or adding decision records. Triggers on "create ADR", "add ADR", "new decision record", "architectural decision", "document decision", or "add decision". +metadata: + author: torrust + version: "1.0" +--- + +# Creating Architectural Decision Records + +## Quick Reference + +```bash +# 1. Generate the filename prefix +date -u +"%Y%m%d%H%M%S" +# e.g. 20241115093012 + +# 2. Create the ADR file +# Format: YYYYMMDDHHMMSS_snake_case_title.md +touch docs/adrs/20241115093012_your_decision_title.md + +# 3. Update the index +# Add entry to docs/adrs/index.md + +# 4. Validate and commit +linter markdown +linter cspell +git commit -S -m "docs(adrs): add ADR for {short description}" +``` + +## When to Create an ADR + +Create an ADR when making a decision that: + +- Affects the project's architecture or design patterns +- Chooses one approach over alternatives that were considered +- Has consequences worth documenting for future contributors +- Answers "why was this done this way?" + +Do **not** create an ADR for trivial implementation choices or style preferences covered by linting. + +## File Naming Convention + +**Format**: `YYYYMMDDHHMMSS_snake_case_title.md` + +Generate the timestamp prefix: + +```bash +date -u +"%Y%m%d%H%M%S" +``` + +**Examples**: + +- `20240227164834_use_plural_for_modules_containing_collections.md` +- `20241115093012_adopt_axum_for_http_server.md` + +Location: `docs/adrs/` + +## ADR Structure + +There is no rigid template — derive structure from context. Use +[docs/templates/ADR.md](../../../docs/templates/ADR.md) as a starting point. + +Optional sections to add when relevant: + +- **Alternatives Considered**: other options explored and why they were rejected +- **Consequences**: positive and negative effects of the decision + +## Step-by-Step Process + +### Step 1: Generate Filename + +```bash +PREFIX=$(date -u +"%Y%m%d%H%M%S") +TITLE="your_decision_title" # snake_case +echo "docs/adrs/${PREFIX}_${TITLE}.md" +``` + +### Step 2: Write the ADR + +- **Description**: Explain the problem thoroughly — enough context for future contributors +- **Agreement**: State clearly what was decided and why +- **Date**: Today's date (`date -u +"%Y-%m-%d"`) +- **References**: Issues, PRs, external docs + +### Step 3: Update the Index + +Add a row to the index table in `docs/adrs/index.md`: + +```markdown +| [YYYYMMDDHHMMSS](YYYYMMDDHHMMSS_your_title.md) | YYYY-MM-DD | Short Title | One-sentence description. | +``` + +- The first column links to the ADR file using the timestamp as display text. +- The short description should allow a reader to understand the decision without opening the file. + +### Step 4: Validate and Commit + +```bash +linter markdown +linter cspell +linter all # full check + +git add docs/adrs/ +git commit -S -m "docs(adrs): add ADR for {short description}" +git push {your-fork-remote} {branch} +``` + +## Example ADR + +For a real example, see +[20240227164834_use_plural_for_modules_containing_collections.md](../../../docs/adrs/20240227164834_use_plural_for_modules_containing_collections.md). diff --git a/.github/skills/dev/planning/create-issue/SKILL.md b/.github/skills/dev/planning/create-issue/SKILL.md new file mode 100644 index 000000000..ed38c9933 --- /dev/null +++ b/.github/skills/dev/planning/create-issue/SKILL.md @@ -0,0 +1,101 @@ +--- +name: create-issue +description: Guide for creating GitHub issues in the torrust-tracker project. Covers the full workflow from specification drafting, user review, to GitHub issue creation with proper documentation and file naming. Supports task, bug, feature, and epic issue types. Use when creating issues, opening tickets, filing bugs, proposing tasks, or adding features. Triggers on "create issue", "open issue", "new issue", "file bug", "add task", "create epic", or "open ticket". +metadata: + author: torrust + version: "1.0" +--- + +# Creating Issues + +## Issue Types + +| Type | Label | When to Use | +| ----------- | --------- | -------------------------------------------- | +| **Task** | `task` | Single implementable unit of work | +| **Bug** | `bug` | Something broken that needs fixing | +| **Feature** | `feature` | New capability or enhancement | +| **Epic** | `epic` | Major feature area containing multiple tasks | + +## Workflow Overview + +The process is **spec-first**: write and review a specification before creating the GitHub issue. + +1. **Draft specification** document in `docs/issues/` (no template — write from scratch) +2. **User reviews** the draft specification +3. **Create GitHub issue** +4. **Rename spec file** to include the issue number +5. **Pre-commit checks** and commit the spec + +**Never create the GitHub issue before the user reviews and approves the specification.** + +## Step-by-Step Process + +### Step 1: Draft Issue Specification + +Create a specification file with a **temporary name** (no issue number yet): + +```bash +touch docs/issues/{short-description}.md +``` + +Use [docs/templates/ISSUE.md](../../../docs/templates/ISSUE.md) as the starting structure. +Use **placeholders** for the issue number until after creation (e.g., `[To be assigned]`). + +After drafting, run linters: + +```bash +linter markdown +linter cspell +``` + +### Step 2: User Reviews the Draft + +**STOP HERE** — present the draft to the user. Iterate until approved. + +### Step 3: Create the GitHub Issue + +After user approval, create the GitHub issue. Options: + +**GitHub CLI:** + +```bash +gh issue create \ + --repo torrust/torrust-tracker \ + --title "{title}" \ + --body "{body}" \ + --label "{label}" +``` + +**MCP GitHub tools** (if available): use `mcp_github_github_issue_write` with `title`, `body`, and `labels`. + +### Step 4: Rename the Spec File + +Rename using the assigned issue number: + +```bash +git mv docs/issues/{short-description}.md \ + docs/issues/{number}-{short-description}.md +``` + +Update any issue number placeholders inside the file. + +### Step 5: Commit and Push + +```bash +linter all # Must pass + +git add docs/issues/ +git commit -S -m "docs(issues): add issue specification for #{number}" +git push {your-fork-remote} {branch} +``` + +## Naming Convention + +File name format: `{number}-{short-description}.md` + +Examples: + +- `1697-ai-agent-configuration.md` +- `42-add-peer-expiry-grace-period.md` +- `523-internal-linting-tool.md` diff --git a/.github/skills/dev/planning/write-markdown-docs/SKILL.md b/.github/skills/dev/planning/write-markdown-docs/SKILL.md new file mode 100644 index 000000000..a2c166efa --- /dev/null +++ b/.github/skills/dev/planning/write-markdown-docs/SKILL.md @@ -0,0 +1,70 @@ +--- +name: write-markdown-docs +description: Guide for writing Markdown documentation in this project. Covers GitHub Flavored Markdown pitfalls, especially the critical #NUMBER pattern that auto-links to GitHub issues and PRs (NEVER use #1, #2, #3 as step/list numbers). Use ordered lists or plain numbers instead. Covers intentional vs accidental autolinks for issues, @mentions, and commit SHAs. Use when writing .md files, documentation, issue descriptions, PR descriptions, or README updates. Triggers on "markdown", "write docs", "documentation", "#number", "github markdown", "autolink", "markdown pitfall", or "GFM". +metadata: + author: torrust + version: "1.0" +--- + +# Writing Markdown Documentation + +## Critical: #NUMBER Auto-links to GitHub Issues + +**GitHub automatically converts `#NUMBER` → link to issue/PR/discussion.** + +```markdown +❌ Bad: accidentally links to issues + +- Task #1: Set up infrastructure ← links to GitHub issue #1 +- Task #2: Configure database ← links to GitHub issue #2 + +Step #1: Install dependencies ← links to GitHub issue #1 +``` + +The links pollute the referenced issues with unrelated backlinks and confuse readers. + +### Fix: Use Ordered Lists or Plain Numbers + +```markdown +✅ Solution 1: Ordered list (automatic numbering) + +1. Set up infrastructure +2. Configure database +3. Deploy application + +✅ Solution 2: Plain numbers (no hash) + +- Task 1: Set up infrastructure +- Task 2: Configure database + +✅ Solution 3: Alternative formats + +- Task (1): Set up infrastructure +- Task [1]: Set up infrastructure +``` + +## When #NUMBER IS Intentional + +Use `#NUMBER` only when you explicitly want to link to that GitHub issue/PR: + +```markdown +✅ Intentional: referencing issue +This implements the behavior described in #42. +Closes #1697. +``` + +## Other GFM Auto-links to Know + +```markdown +@username → links to GitHub user profile (use intentionally for mentions) +abc1234 (SHA) → links to commit (useful for references) +owner/repo#42 → cross-repo issue link +``` + +## Checklist Before Committing Docs + +- [ ] No `#NUMBER` patterns used for enumeration or step numbering +- [ ] Ordered lists use Markdown syntax (`1.` `2.` `3.`) +- [ ] Any `#NUMBER` present is an intentional issue/PR reference +- [ ] Tables are consistently formatted +- [ ] `linter markdown` and `linter cspell` pass diff --git a/.github/skills/dev/rust-code-quality/handle-errors-in-code/SKILL.md b/.github/skills/dev/rust-code-quality/handle-errors-in-code/SKILL.md new file mode 100644 index 000000000..7b326ce60 --- /dev/null +++ b/.github/skills/dev/rust-code-quality/handle-errors-in-code/SKILL.md @@ -0,0 +1,114 @@ +--- +name: handle-errors-in-code +description: Guide for error handling in this Rust project. Covers the four principles (clarity, context, actionability, explicit enums over anyhow), the thiserror pattern for structured errors, including what/where/when/why context, writing actionable help text, and avoiding vague errors. Also covers the located-error package for errors with source location. Use when writing error types, handling Results, adding error variants, or reviewing error messages. Triggers on "error handling", "error type", "Result", "thiserror", "anyhow", "error enum", "error message", "handle error", "add error variant", or "located-error". +metadata: + author: torrust + version: "1.0" +--- + +# Handling Errors in Code + +## Core Principles + +1. **Clarity** — Users immediately understand what went wrong +2. **Context** — Include what/where/when/why +3. **Actionability** — Tell users how to fix it +4. **Explicit enums over `anyhow`** — Prefer structured errors for pattern matching + +## Prefer Explicit Enum Errors + +```rust +// ✅ Correct: explicit, matchable, clear +#[derive(Debug, thiserror::Error)] +pub enum TrackerError { + #[error("Torrent '{info_hash}' not found in whitelist")] + TorrentNotWhitelisted { info_hash: InfoHash }, + + #[error("Peer limit exceeded for torrent '{info_hash}': max {limit}")] + PeerLimitExceeded { info_hash: InfoHash, limit: usize }, +} + +// ❌ Wrong: opaque, hard to match +return Err(anyhow::anyhow!("Something went wrong")); +return Err("Invalid input".into()); +``` + +## Include Actionable Fix Instructions in Display + +When the error is user-facing, add instructions: + +```rust +#[error( + "Configuration file not found at '{path}'.\n\ + Copy the default: cp share/default/config/tracker.toml {path}" +)] +ConfigNotFound { path: PathBuf }, +``` + +## Context Requirements + +Each error should answer: + +- **What**: What operation was being performed? +- **Where**: Which component, file, or resource? +- **When**: Under what conditions? +- **Why**: What caused the failure? + +```rust +// ✅ Good: full context +#[error("UDP socket bind failed for '{addr}': {source}. Is port {port} already in use?")] +SocketBindFailed { addr: SocketAddr, port: u16, source: std::io::Error }, + +// ❌ Bad: no context +return Err("bind failed".into()); +``` + +## The `located-error` Package + +For errors that benefit from source location tracking, use the `located-error` package: + +```toml +[dependencies] +torrust-tracker-located-error = { workspace = true } +``` + +```rust +use torrust_tracker_located_error::Located; + +// Wraps any error with file and line information +let err = Located(my_error).into(); +``` + +## Unwrap and Expect Policy + +| Context | `.unwrap()` | `.expect("msg")` | `?` / `Result` | +| ---------------------- | ----------- | ----------------------------------------- | -------------- | +| Production code | Never | Only when failure is logically impossible | Default | +| Tests and doc examples | Acceptable | Preferred when message adds clarity | — | + +```rust +// ✅ Production: propagate errors with ? +fn load_config(path: &Path) -> Result<Config, ConfigError> { + let content = std::fs::read_to_string(path) + .map_err(|e| ConfigError::FileAccess { path: path.to_path_buf(), source: e })?; + toml::from_str(&content) + .map_err(|e| ConfigError::InvalidToml { path: path.to_path_buf(), source: e }) +} + +// ✅ Tests: unwrap() is fine +#[test] +fn it_should_parse_valid_config() { + let config = Config::parse(VALID_TOML).unwrap(); + assert_eq!(config.http_api.bind_address, "127.0.0.1:1212"); +} +``` + +## Quick Checklist + +- [ ] Error type uses `thiserror::Error` derive +- [ ] Error message includes specific context (names, paths, addresses, values) +- [ ] Error message includes fix instructions where possible +- [ ] Prefer `enum` over `Box<dyn Error>` or `anyhow` in library code +- [ ] No vague messages like "invalid input" or "error occurred" +- [ ] No `.unwrap()` in production code (tests and doc examples are fine) +- [ ] Consider `located-error` for diagnostics-rich errors diff --git a/.github/skills/dev/rust-code-quality/handle-secrets/SKILL.md b/.github/skills/dev/rust-code-quality/handle-secrets/SKILL.md new file mode 100644 index 000000000..b3e6e5d43 --- /dev/null +++ b/.github/skills/dev/rust-code-quality/handle-secrets/SKILL.md @@ -0,0 +1,87 @@ +--- +name: handle-secrets +description: Guide for handling sensitive data (secrets) in this Rust project. NEVER use plain String for API tokens, passwords, or other credentials. Use the secrecy crate's Secret<T> wrapper to prevent accidental exposure through Debug output, logs, and error messages. Call .expose_secret() only when the actual value is needed. Use when working with credentials, API keys, tokens, passwords, or any sensitive configuration. Triggers on "secret", "API token", "password", "credential", "sensitive data", "secrecy", or "expose secret". +metadata: + author: torrust + version: "1.0" +--- + +# Handling Sensitive Data (Secrets) + +## Core Rule + +**NEVER use plain `String` for sensitive data.** Wrap secrets in `secrecy::Secret<String>` +(or similar) to prevent accidental exposure. + +```rust +// ❌ WRONG: secret leaked in Debug output +pub struct ApiConfig { + pub token: String, +} +println!("{config:?}"); // → ApiConfig { token: "secret_abc123" } — LEAKED! +``` + +```rust +// ✅ CORRECT: secret redacted in Debug +use secrecy::Secret; +pub struct ApiConfig { + pub token: Secret<String>, +} +println!("{config:?}"); // → ApiConfig { token: Secret([REDACTED]) } +``` + +## Using the `secrecy` Crate + +Add the dependency: + +```toml +[dependencies] +secrecy = { workspace = true } +``` + +Basic usage: + +```rust +use secrecy::{Secret, ExposeSecret}; + +// Wrap the secret +let token = Secret::new(String::from("my-api-token")); + +// Access the value only when truly needed (e.g., making the actual API call) +let token_str: &str = token.expose_secret(); +``` + +## What to Protect + +Wrap with `Secret<T>` when the value is: + +- API tokens (REST API admin token, external service tokens) +- Passwords (database credentials, service accounts) +- Private keys or certificates + +## Rules for `.expose_secret()` + +- Call **as late as possible** — only at the point where the value is required +- **Never** call in `log!`, `debug!`, `info!`, `warn!`, `error!` macros +- **Never** call in `Display` or `Debug` implementations +- **Never** include in error messages that may be logged or shown to users + +```rust +// ✅ Correct: called at last moment for HTTP header +let response = client + .get(url) + .header("Authorization", format!("Bearer {}", token.expose_secret())) + .send() + .await?; + +// ❌ Wrong: exposed in log +tracing::debug!("Using token: {}", token.expose_secret()); +``` + +## Checklist + +- [ ] No plain `String` fields for tokens, passwords, or private keys +- [ ] `Secret<String>` (or equivalent) used for all sensitive values +- [ ] `.expose_secret()` called only at the last moment +- [ ] No `.expose_secret()` in log statements or error messages +- [ ] No sensitive values in `Display` or `Debug` output diff --git a/.github/skills/dev/testing/write-unit-test/SKILL.md b/.github/skills/dev/testing/write-unit-test/SKILL.md new file mode 100644 index 000000000..5ba1a8381 --- /dev/null +++ b/.github/skills/dev/testing/write-unit-test/SKILL.md @@ -0,0 +1,221 @@ +--- +name: write-unit-test +description: Guide for writing unit tests following project conventions including behavior-driven naming (it*should*\*), AAA pattern, MockClock for deterministic time testing, and parameterized tests with rstest. Use when adding tests for domain entities, value objects, utilities, or tracker logic. Triggers on "write unit test", "add test", "test coverage", "unit testing", or "add unit tests". +metadata: + author: torrust + version: "1.0" +--- + +# Writing Unit Tests + +## Core Principles + +Unit tests in this project are written against the **Test Desiderata** — the 12 properties that +make tests valuable, defined by Kent Beck. Not every property applies equally to every test, but +treat them as the standard to reason about and optimize for. + +| Property | What it means | +| ------------------------- | ----------------------------------------------------------------------------------- | +| **Isolated** | Tests return the same result regardless of run order. No shared mutable state. | +| **Composable** | Different dimensions of variability can be tested separately and results combined. | +| **Deterministic** | Same inputs always produce the same result. No randomness, no wall-clock time. | +| **Fast** | Tests run in milliseconds. Unit tests must never block on I/O or sleep. | +| **Writable** | Writing the test should cost much less than writing the code it covers. | +| **Readable** | A reader can understand what behaviour is being tested and why, without context. | +| **Behavioral** | Tests are sensitive to changes in observable behaviour, not internal structure. | +| **Structure-insensitive** | Refactoring the implementation should not break tests that test the same behaviour. | +| **Automated** | Tests run without human intervention (`cargo test`). | +| **Specific** | When a test fails, the cause is immediately obvious from the failure message. | +| **Predictive** | Passing tests give genuine confidence the code is ready for production. | +| **Inspiring** | Passing the full suite inspires confidence to ship. | + +Some properties support each other (automation makes tests faster). Some trade off against each +other (more predictive tests tend to be slower). Use composability to resolve apparent conflicts. + +Reference: <https://testdesiderata.com/> and Kent Beck's original papers on +[Test Desiderata](https://medium.com/@kentbeck_7670/test-desiderata-94150638a4b3) and +[Programmer Test Principles](https://medium.com/@kentbeck_7670/programmer-test-principles-d01c064d7934). + +### Project-specific conventions + +- **Behavior-driven naming** — test names document what the code does +- **AAA Pattern** — Arrange → Act → Assert (clear structure) +- **Deterministic** — use `MockClock` instead of real time (see Phase 2) +- **Isolated** — no shared mutable state between tests +- **Fast** — unit tests run in milliseconds + +## Phase 1: Basic Unit Test + +### Naming Convention + +**Format**: `it_should_{expected_behavior}_when_{condition}` + +- Always use the `it_should_` prefix +- Never use the `test_` prefix +- Use `when_` or `given_` for conditions +- Be specific and descriptive + +```rust +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn it_should_return_error_when_info_hash_is_invalid() { + // Arrange + let invalid_hash = "not-a-valid-hash"; + + // Act + let result = InfoHash::from_str(invalid_hash); + + // Assert + assert!(result.is_err()); + } + + #[test] + fn it_should_parse_valid_info_hash() { + // Arrange + let valid_hex = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + + // Act + let result = InfoHash::from_str(valid_hex); + + // Assert + assert!(result.is_ok()); + } +} +``` + +### Running Tests + +```bash +# Run all tests in a package +cargo test -p bittorrent-tracker-core + +# Run specific test by name +cargo test it_should_return_error_when_info_hash_is_invalid + +# Run tests in a module +cargo test info_hash::tests + +# Run with output +cargo test -- --nocapture +``` + +## Phase 2: Deterministic Time with `clock::Stopped` + +The `clock` workspace package provides `clock::Stopped` for deterministic time testing. +Never call `std::time::SystemTime::now()` or `chrono::Utc::now()` directly in production code +that needs testing. Instead, use the type-level clock abstraction. + +### Use the Type-Level Clock Alias + +Copy the following boilerplate into each crate that needs a clock. The `CurrentClock` alias +automatically selects `Working` in production and `Stopped` in tests: + +```rust +/// Working version, for production. +#[cfg(not(test))] +pub(crate) type CurrentClock = torrust_tracker_clock::clock::Working; + +/// Stopped version, for testing. +#[cfg(test)] +pub(crate) type CurrentClock = torrust_tracker_clock::clock::Stopped; +``` + +In production code, obtain the current time via the `Time` trait: + +```rust +use torrust_tracker_clock::clock::Time as _; + +pub fn is_peer_expired(last_seen: std::time::Duration, ttl: u32) -> bool { + let now = CurrentClock::now(); // returns DurationSinceUnixEpoch (= std::time::Duration) + now.saturating_sub(last_seen) > std::time::Duration::from_secs(u64::from(ttl)) +} +``` + +### Control Time in Tests + +Use `clock::Stopped::local_set` to pin the clock to a specific instant. The stopped clock is +thread-local, so tests are isolated from each other by default. + +```rust +#[cfg(test)] +mod tests { + use std::time::Duration; + + use torrust_tracker_clock::clock::{stopped::Stopped as _, Time as _}; + use torrust_tracker_clock::clock::Stopped; + + use super::*; + + #[test] + fn it_should_mark_peer_as_expired_when_ttl_has_elapsed() { + // Arrange — pin the clock to a known instant + let fixed_time = Duration::from_secs(1_700_000_100); + Stopped::local_set(&fixed_time); + + let last_seen = Duration::from_secs(1_700_000_000); + let ttl = 60u32; + + // Act + let expired = is_peer_expired(last_seen, ttl); + + // Assert + assert!(expired); + + // Clean up — reset to zero so other tests start from a clean state + Stopped::local_reset(); + } +} +``` + +> **Key points** +> +> - `Stopped::now()` defaults to `Duration::ZERO` at the start of each test thread. +> - `Stopped::local_set(&duration)` sets the current time for the calling thread only. +> - `Stopped::local_reset()` resets back to `Duration::ZERO`. +> - `Stopped::local_add(&duration)` advances the clock by the given amount. +> - Import the `Stopped` trait (`use …::stopped::Stopped as _`) to bring its methods into scope. + +## Phase 3: Parameterized Tests with rstest + +Use `rstest` for multiple input/output combinations to avoid repetition. + +```toml +[dev-dependencies] +rstest = { workspace = true } +``` + +```rust +use rstest::rstest; + +#[rstest] +#[case("aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", true)] +#[case("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", true)] +#[case("not-a-hash", false)] +#[case("", false)] +fn it_should_validate_info_hash(#[case] input: &str, #[case] is_valid: bool) { + let result = InfoHash::from_str(input); + assert_eq!(result.is_ok(), is_valid, "input: {input}"); +} +``` + +## Phase 4: Test Helpers + +The `test-helpers` workspace package provides shared test utilities. + +```toml +[dev-dependencies] +torrust-tracker-test-helpers = { workspace = true } +``` + +Check the package for available mock servers, fixture generators, and utility types. + +## Quick Checklist + +- [ ] Test name uses `it_should_` prefix +- [ ] Test follows AAA pattern with comments (`// Arrange`, `// Act`, `// Assert`) +- [ ] No `std::time::SystemTime::now()` in production code — use the `CurrentClock` type alias instead +- [ ] No shared mutable state between tests +- [ ] `cargo test -p <package>` passes diff --git a/.github/workflows/copilot-setup-steps.yml b/.github/workflows/copilot-setup-steps.yml new file mode 100644 index 000000000..2017038b9 --- /dev/null +++ b/.github/workflows/copilot-setup-steps.yml @@ -0,0 +1,53 @@ +name: "Copilot Setup Steps" + +# Automatically run the setup steps when they are changed to allow for easy +# validation, and allow manual testing through the repository's "Actions" tab. +on: + workflow_dispatch: + push: + paths: + - .github/workflows/copilot-setup-steps.yml + - scripts/install-git-hooks.sh + - scripts/pre-commit.sh + pull_request: + paths: + - .github/workflows/copilot-setup-steps.yml + - scripts/install-git-hooks.sh + - scripts/pre-commit.sh + +jobs: + # The job MUST be called `copilot-setup-steps` or it will not be picked up + # by Copilot. + copilot-setup-steps: + runs-on: ubuntu-latest + timeout-minutes: 30 + + # Set the permissions to the lowest permissions possible needed for your + # steps. Copilot will be given its own token for its operations. + permissions: + contents: read + + steps: + - name: Checkout code + uses: actions/checkout@v6 + + - name: Set up Rust toolchain + uses: dtolnay/rust-toolchain@stable + + - name: Enable Rust cache + uses: Swatinem/rust-cache@v2 + + - name: Build workspace + run: cargo build --workspace + + - name: Install linter + run: cargo install --locked --git https://github.com/torrust/torrust-linting --bin linter + + - name: Install cargo-machete + run: cargo install cargo-machete + + - name: Install Git pre-commit hooks + run: ./scripts/install-git-hooks.sh + + - name: Smoke-check — run all linters + run: linter all diff --git a/.gitignore b/.gitignore index fd83ee918..4b811d59f 100644 --- a/.gitignore +++ b/.gitignore @@ -17,4 +17,5 @@ codecov.json integration_tests_sqlite3.db lcov.info perf.data* +repomix-output.xml rustc-ice-*.txt diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..801bf8eef --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,389 @@ +# Torrust Tracker — AI Assistant Instructions + +**Repository**: [torrust/torrust-tracker](https://github.com/torrust/torrust-tracker) + +## 📋 Project Overview + +**Torrust Tracker** is a high-quality, production-grade BitTorrent tracker written in Rust. It +matchmakes peers and collects statistics, supporting the UDP, HTTP, and TLS socket types with +native IPv4/IPv6 support, private/whitelisted mode, and a management REST API. + +- **Language**: Rust (edition 2021, MSRV 1.72) +- **License**: AGPL-3.0-only +- **Version**: 3.0.0-develop +- **Web framework**: [Axum](https://github.com/tokio-rs/axum) +- **Async runtime**: Tokio +- **Protocols**: BitTorrent UDP (BEP 15), HTTP (BEP 3/23), REST management API +- **Databases**: SQLite3, MySQL +- **Workspace type**: Cargo workspace (multi-crate monorepo) + +## 🏗️ Tech Stack + +- **Languages**: Rust, YAML, TOML, Markdown, Shell scripts +- **Web framework**: Axum (HTTP server + REST API) +- **Async runtime**: Tokio (multi-thread) +- **Testing**: testcontainers (E2E) +- **Databases**: SQLite3, MySQL +- **Containerization**: Docker / Podman (`Containerfile`) +- **CI**: GitHub Actions +- **Linting tools**: markdownlint, yamllint, taplo, cspell, shellcheck, clippy, rustfmt (unified + under the `linter` binary from [torrust/torrust-linting](https://github.com/torrust/torrust-linting)) + +## 📁 Key Directories + +- `src/` — Main binary and library entry points (`main.rs`, `lib.rs`, `app.rs`, `container.rs`) +- `src/bin/` — Additional binary targets (`e2e_tests_runner`, `http_health_check`, `profiling`) +- `src/bootstrap/` — Application bootstrap logic +- `src/console/` — Console entry points +- `packages/` — Cargo workspace packages (all domain logic lives here; see package catalog below) +- `console/` — Console tools (e.g., `tracker-client`) +- `contrib/` — Community-contributed utilities (`bencode`) and developer tooling +- `contrib/dev-tools/` — Developer tools: git hooks (`pre-commit.sh`, `pre-push.sh`), + container scripts, and init scripts +- `tests/` — Integration tests (`integration.rs`, `servers/`) +- `docs/` — Project documentation, ADRs, issue specs, and benchmarking guides +- `docs/adrs/` — Architectural Decision Records +- `docs/issues/` — Issue specs / implementation plans +- `share/default/` — Default configuration files and fixtures +- `storage/` — Runtime data (git-ignored); databases, logs, config +- `.github/workflows/` — CI/CD workflows (testing, coverage, container, deployment) +- `.github/skills/` — Agent Skills for specialized workflows and task-specific guidance +- `.github/agents/` — Custom Copilot agents and their repository-specific definitions + +## 📦 Package Catalog + +All packages live under `packages/`. The workspace version is `3.0.0-develop`. + +| Package | Prefix / Layer | Description | +| --------------------------------- | -------------- | ------------------------------------------------ | +| `axum-server` | `axum-*` | Base Axum HTTP server infrastructure | +| `axum-http-tracker-server` | `axum-*` | BitTorrent HTTP tracker server (BEP 3/23) | +| `axum-rest-tracker-api-server` | `axum-*` | Management REST API server | +| `axum-health-check-api-server` | `axum-*` | Health monitoring endpoint | +| `http-tracker-core` | `*-core` | HTTP-specific tracker domain logic | +| `udp-tracker-core` | `*-core` | UDP-specific tracker domain logic | +| `tracker-core` | `*-core` | Central tracker peer-management logic | +| `http-protocol` | `*-protocol` | HTTP tracker protocol (BEP 3/23) parsing | +| `udp-protocol` | `*-protocol` | UDP tracker protocol (BEP 15) framing/parsing | +| `swarm-coordination-registry` | domain | Torrent/peer coordination registry | +| `configuration` | domain | Config file parsing, environment variables | +| `primitives` | domain | Core domain types (InfoHash, PeerId, …) | +| `clock` | utilities | Mockable time source for deterministic testing | +| `located-error` | utilities | Diagnostic errors with source locations | +| `test-helpers` | utilities | Mock servers, test data generation | +| `server-lib` | shared | Shared server library utilities | +| `tracker-client` | client tools | CLI tracker interaction/testing client | +| `rest-tracker-api-client` | client tools | REST API client library | +| `rest-tracker-api-core` | client tools | REST API core logic | +| `udp-tracker-server` | server | UDP tracker server implementation | +| `torrent-repository` | domain | Torrent metadata storage and InfoHash management | +| `events` | domain | Domain event definitions | +| `metrics` | domain | Prometheus metrics integration | +| `torrent-repository-benchmarking` | benchmarking | Torrent storage benchmarks | + +**Console tools** (under `console/`): + +| Tool | Description | +| ---------------- | ------------------------------------ | +| `tracker-client` | Client for interacting with trackers | + +**Community contributions** (under `contrib/`): + +| Crate | Description | +| --------- | ------------------------------- | +| `bencode` | Bencode encode/decode utilities | + +## 🏷️ Package Naming Conventions + +| Prefix | Responsibility | Dependencies | +| ------------ | -------------------------------------- | ------------------------ | +| `axum-*` | HTTP server components using Axum | Axum framework | +| `*-server` | Server implementations | Corresponding `*-core` | +| `*-core` | Domain logic and business rules | Protocol implementations | +| `*-protocol` | BitTorrent protocol implementations | BEP specifications | +| `udp-*` | UDP protocol-specific implementations | Tracker core | +| `http-*` | HTTP protocol-specific implementations | Tracker core | + +## 📄 Key Configuration Files + +| File | Used by | +| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| `.markdownlint.json` | markdownlint | +| `.yamllint-ci.yml` | yamllint | +| `.taplo.toml` | taplo (TOML formatting) | +| `cspell.json` | cspell (spell checker) configuration | +| `project-words.txt` | cspell project-specific dictionary | +| `rustfmt.toml` | rustfmt (`group_imports = "StdExternalCrate"`, `max_width = 130`) | +| `.cargo/config.toml` | Cargo aliases (`cov`, `cov-lcov`, `cov-html`, `time`) and global `rustflags` (`-D warnings`, `-D unused`, `-D rust-2018-idioms`, …) | +| `Cargo.toml` | Cargo workspace root | +| `compose.yaml` | Docker Compose for local dev and demo | +| `Containerfile` | Container image definition | +| `codecov.yaml` | Code coverage configuration | + +## 🧪 Build & Test + +### Setup + +```sh +rustup show # Check active toolchain +rustup update # Update toolchain +rustup toolchain install nightly # Optional: only needed for manual cargo +nightly doc; the repo hook runs ./scripts/pre-commit.sh +``` + +### Build + +```sh +cargo build # Build all workspace crates +cargo build --release # Release build +cargo build --package <pkg> # Build a specific package +``` + +### Test + +```sh +cargo test --doc --workspace # Documentation tests +cargo test --tests --benches --examples --workspace \ + --all-targets --all-features # All tests +cargo test -p <package-name> # Single package + +# MySQL-specific tests (requires a running MySQL instance) +TORRUST_TRACKER_CORE_RUN_MYSQL_DRIVER_TEST=true \ + cargo test --package bittorrent-tracker-core + +# Integration tests (root) +cargo test --test integration # tests/integration.rs +``` + +### E2E Tests + +```sh +cargo run --bin e2e_tests_runner -- \ + --config-toml-path "./share/default/config/tracker.e2e.container.sqlite3.toml" +``` + +### Documentation + +```sh +cargo +nightly doc --no-deps --bins --examples --workspace --all-features +``` + +### Benchmarks + +```sh +cargo bench --package torrent-repository-benchmarking +``` + +See [docs/benchmarking.md](docs/benchmarking.md) and [docs/profiling.md](docs/profiling.md). + +## 🔍 Lint Commands + +The project uses the `linter` binary from +[torrust/torrust-linting](https://github.com/torrust/torrust-linting). + +```sh +# Install the linter binary +cargo install --locked --git https://github.com/torrust/torrust-linting --bin linter + +# Run all linters (MANDATORY before every commit and PR) +linter all + +# Run individual linters +linter markdown # markdownlint +linter yaml # yamllint +linter toml # taplo +linter cspell # spell checker +linter clippy # Rust linter +linter rustfmt # Rust formatter check +linter shellcheck # shell scripts +``` + +**`linter all` must exit with code `0` before every commit. PRs that fail CI linting are +rejected without review.** + +## 🔗 Dependencies Check + +```sh +cargo machete # Check for unused dependencies (mandatory before commits) +``` + +Install via: `cargo install cargo-machete` + +## 🎨 Code Style + +- **rustfmt**: Format with `cargo fmt` before committing. Config: `rustfmt.toml` + (`group_imports = "StdExternalCrate"`, `imports_granularity = "Module"`, `max_width = 130`). +- **Compile flags**: `.cargo/config.toml` enables strict global `rustflags` (`-D warnings`, + `-D unused`, `-D rust-2018-idioms`, `-D future-incompatible`, and others). All code must + compile cleanly with these flags — no suppressions unless absolutely necessary. +- **clippy**: No warnings allowed (`cargo clippy -- -D warnings`). +- **Imports**: All imports at the top of the file, grouped (std → external crates → internal + crate). Prefer short imported names over fully-qualified paths + (e.g., `Arc<MyType>` not `std::sync::Arc<crate::my::MyType>`). Use full paths only to + disambiguate naming conflicts. +- **TOML**: Must pass `taplo fmt --check **/*.toml`. Auto-fix with `taplo fmt **/*.toml`. +- **Markdown**: Must pass markdownlint. +- **YAML**: Must pass `yamllint -c .yamllint-ci.yml`. +- **Spell checking**: Add new technical terms to `project-words.txt` (one word per line, + alphabetical order). + +## 🤝 Collaboration Principles + +These rules apply repository-wide to every assistant, including custom agents. + +When acting as an assistant in this repository: + +- Do not flatter the user or agree with weak ideas by default. +- Push back when a request, diff, or proposed commit looks wrong. +- Flag unclear but important points before they become problems. +- Ask a clarifying question instead of making a random choice when the decision matters. +- Call out likely misses: naming inconsistencies, accidental generated files, + staged-versus-unstaged mismatches, missing docs updates, or suspicious commit scope. + +When raising a likely mistake or blocker, say so clearly and early instead of burying it after +routine status updates. + +## 🔧 Essential Rules + +1. **Linting gate**: `linter all` must exit `0` before every commit. No exceptions. +2. **GPG commit signing**: All commits **must** be signed with GPG (`git commit -S`). +3. **Never commit `storage/` or `target/`**: These directories contain runtime data and build + artifacts. They are git-ignored; never force-add them. +4. **Unused dependencies**: Run `cargo machete` before committing. Remove any unused + dependencies immediately. +5. **Rust imports**: All imports at the top of the file, grouped (std → external crates → + internal crate). Prefer short imported names over fully-qualified paths. +6. **Continuous self-review**: Review your own work against project quality standards. Apply + self-review at three levels: + - **Mandatory** — before opening a pull request + - **Strongly recommended** — before each commit + - **Recommended** — after completing each small, independent, deployable change +7. **Security**: Do not report security vulnerabilities through public GitHub issues. Send an + email to `info@nautilus-cyberneering.de` instead. See [SECURITY.md](SECURITY.md). + +## 🌿 Git Workflow + +**Branch naming**: + +```text +<issue-number>-<short-description> # e.g. 1697-ai-agent-configuration (preferred) +feat/<short-description> # for features without a tracked issue +fix/<short-description> # for bug fixes +chore/<short-description> # for maintenance tasks +``` + +**Commit messages** follow [Conventional Commits](https://www.conventionalcommits.org/): + +```text +feat(<scope>): add X +fix(<scope>): resolve Y +chore(<scope>): update Z +docs(<scope>): document W +refactor(<scope>): restructure V +ci(<scope>): adjust pipeline U +test(<scope>): add tests for T +``` + +Scope should reflect the affected package or area (e.g., `tracker-core`, `udp-protocol`, `ci`, `docs`). + +**Branch strategy**: + +- Feature branches are cut from `develop` +- PRs target `develop` +- `develop` → `staging/main` → `main` (release pipeline) +- PRs must pass all CI status checks before merge + +See [docs/release_process.md](docs/release_process.md) for the full release workflow. + +## 🧭 Development Principles + +For detailed information see [`docs/`](docs/). + +**Core Principles:** + +- **Observability**: If it happens, we can see it — even after it happens (deep traceability) +- **Testability**: Every component must be testable in isolation and as part of the whole +- **Modularity**: Clear package boundaries; servers contain only network I/O logic +- **Extensibility**: Core logic is framework-agnostic for easy protocol additions + +**Code Quality Standards** — both production and test code must be: + +- **Clean**: Well-structured with clear naming and minimal complexity +- **Maintainable**: Easy to modify and extend without breaking existing functionality +- **Readable**: Clear intent that can be understood by other developers +- **Testable**: Designed to support comprehensive testing at all levels + +**Beck's Four Rules of Simple Design** (in priority order): + +1. **Passes the tests**: The code must work as intended — testing is a first-class activity +2. **Reveals intention**: Code should be easy to understand, expressing purpose clearly +3. **No duplication**: Apply DRY — eliminating duplication drives out good designs +4. **Fewest elements**: Remove anything that doesn't serve the prior three rules + +Reference: [Beck Design Rules](https://martinfowler.com/bliki/BeckDesignRules.html) + +## 🐳 Container / Docker + +```sh +# Run the latest image +docker run -it torrust/tracker:latest +# or with Podman +podman run -it docker.io/torrust/tracker:latest + +# Build and run via Docker Compose +docker compose up -d # Start all services (detached) +docker compose logs -f tracker # Follow tracker logs +docker compose down # Stop and remove containers +``` + +**Volume mappings** (local `storage/` → container paths): + +```text +./storage/tracker/lib → /var/lib/torrust/tracker +./storage/tracker/log → /var/log/torrust/tracker +./storage/tracker/etc → /etc/torrust/tracker +``` + +**Ports**: UDP tracker: `6969`, HTTP tracker: `7070`, REST API: `1212` + +See [docs/containers.md](docs/containers.md) for detailed container documentation. + +## 🎯 Auto-Invoke Skills + +Agent Skills live under [`.github/skills/`](.github/skills/). Each skill is a `SKILL.md` file +with YAML frontmatter and Markdown instructions covering a repeatable workflow. + +> Skills supplement (not replace) the rules in this file. Rules apply always; skills activate +> when their workflows are needed. + +**For VS Code**: Enable `chat.useAgentSkills` in settings to activate skill discovery. + +**Learn more**: See [Agent Skills Specification (agentskills.io)](https://agentskills.io/specification). + +## 📚 Documentation + +- [Documentation Index](docs/index.md) +- [Package Architecture](docs/packages.md) +- [Benchmarking](docs/benchmarking.md) +- [Profiling](docs/profiling.md) +- [Containers](docs/containers.md) +- [Release Process](docs/release_process.md) +- [ADRs](docs/adrs/README.md) +- [Issues / Implementation Plans](docs/issues/) +- [API docs (docs.rs)](https://docs.rs/torrust-tracker/) +- [Report a security vulnerability](SECURITY.md) + +### Quick Navigation + +| Task | Start Here | +| ------------------------------------ | ---------------------------------------------------- | +| Understand the architecture | [`docs/packages.md`](docs/packages.md) | +| Run the tracker in a container | [`docs/containers.md`](docs/containers.md) | +| Read all docs | [`docs/index.md`](docs/index.md) | +| Understand an architectural decision | [`docs/adrs/README.md`](docs/adrs/README.md) | +| Read or write an issue spec | [`docs/issues/`](docs/issues/) | +| Run benchmarks | [`docs/benchmarking.md`](docs/benchmarking.md) | +| Run profiling | [`docs/profiling.md`](docs/profiling.md) | +| Understand the release process | [`docs/release_process.md`](docs/release_process.md) | +| Report a security vulnerability | [`SECURITY.md`](SECURITY.md) | +| Agent skills reference | [`.github/skills/`](.github/skills/) | +| Custom agents reference | [`.github/agents/`](.github/agents/) | diff --git a/cSpell.json b/cSpell.json deleted file mode 100644 index 43eb391d3..000000000 --- a/cSpell.json +++ /dev/null @@ -1,23 +0,0 @@ -{ - "$schema": "https://raw.githubusercontent.com/streetsidesoftware/cspell/main/cspell.schema.json", - "version": "0.2", - "dictionaryDefinitions": [ - { - "name": "project-words", - "path": "./project-words.txt", - "addWords": true - } - ], - "dictionaries": [ - "project-words" - ], - "enableFiletypes": [ - "dockerfile", - "shellscript", - "toml" - ], - "ignorePaths": [ - "target", - "/project-words.txt" - ] -} \ No newline at end of file diff --git a/cspell.json b/cspell.json index 02f29f7f9..39ddf510e 100644 --- a/cspell.json +++ b/cspell.json @@ -22,6 +22,7 @@ "contrib/bencode/benches/*.bencode", "contrib/dev-tools/su-exec/**", ".github/labels.json", - "/project-words.txt" + "/project-words.txt", + "repomix-output.xml" ] -} \ No newline at end of file +} diff --git a/docs/adrs/20260420200013_adopt_custom_github_copilot_aligned_agent_framework.md b/docs/adrs/20260420200013_adopt_custom_github_copilot_aligned_agent_framework.md new file mode 100644 index 000000000..556e131fb --- /dev/null +++ b/docs/adrs/20260420200013_adopt_custom_github_copilot_aligned_agent_framework.md @@ -0,0 +1,86 @@ +# Adopt a Custom, GitHub-Copilot-Aligned Agent Framework + +## Description + +As AI coding agents become a more common part of the development workflow, the project needs a +clear strategy for how agents should interact with the codebase. Several third-party "agent +frameworks" exist that promise to give agents structure and purpose, but they each come with +trade-offs that may not fit the tracker's needs. + +This ADR records the decision to build a lightweight, first-party agent framework using the +open standards that GitHub Copilot already supports natively: `AGENTS.md`, Agent Skills, and +Custom Agent profiles. + +## Agreement + +We adopt a custom, GitHub-Copilot-aligned agent framework consisting of: + +- **`AGENTS.md`** at the repository root (and in key subdirectories) — following the + [agents.md](https://agents.md/) open standard stewarded by the Agentic AI Foundation under the + Linux Foundation. Provides AI coding agents with project context, build steps, test commands, + conventions, and essential rules. +- **Agent Skills** under `.github/skills/` — following the + [Agent Skills specification](https://agentskills.io/specification). Each skill is a directory + containing a `SKILL.md` file with YAML frontmatter and Markdown instructions, covering + repeatable tasks such as committing changes, running linters, creating ADRs, or setting up the + development environment. +- **Custom Agent profiles** under `.github/agents/` — Markdown files with YAML frontmatter + defining specialised Copilot agents (e.g. `committer`, `implementer`, `complexity-auditor`) + that can be invoked directly or as subagents. +- **`copilot-setup-steps.yml`** workflow — prepares the GitHub Copilot cloud agent environment + before it starts working on any task. + +### Alternatives Considered + +**[obra/superpowers](https://github.com/obra/superpowers)** + +A framework that adds "superpowers" to coding agents through a set of conventions and tools. +Not adopted for the following reasons: + +1. **Complexity mismatch** — introduces abstractions heavier than what tracker development needs. +1. **Precision requirements** — the tracker involves low-level Rust programming where agent work + must be reviewed carefully; generic productivity frameworks are not designed for that + constraint. +1. **Tooling churn risk** — depending on a third-party framework risks forced refactoring if + that framework is deprecated or pivots. + +**[gsd-build/get-shit-done](https://github.com/gsd-build/get-shit-done)** + +A productivity-oriented agent framework with opinionated workflows. +Not adopted for the same reasons as above, plus: + +1. **GitHub-first ecosystem** — the tracker is hosted on GitHub and makes intensive use of + GitHub resources (Actions, Copilot, MCP tools). Staying aligned with GitHub Copilot avoids + unnecessary integration friction. + +### Why the Custom Approach + +1. **Tailored fit** — shaped precisely to Torrust conventions, commit style, linting gates, and + package structure from day one. +1. **Proven in practice** — the same approach has already been validated during the development + of `torrust-tracker-deployer`. +1. **Agent-agnostic by design** — expressed as plain Markdown files (`AGENTS.md`, `SKILL.md`, + agent profiles), decoupled from any single agent product. Migration or multi-agent use is + straightforward. +1. **Incremental adoption** — individual skills, custom agents, or patterns from evaluated + frameworks can still be cherry-picked and integrated progressively if specific value is + identified. +1. **Stability** — a first-party approach is more stable than depending on a third-party + framework whose roadmap we do not control. + +## Date + +2026-04-20 + +## References + +- Issue: https://github.com/torrust/torrust-tracker/issues/1697 +- PR: https://github.com/torrust/torrust-tracker/pull/1699 +- AGENTS.md specification: https://agents.md/ +- Agent Skills specification: https://agentskills.io/specification +- GitHub Copilot — About agent skills: https://docs.github.com/en/copilot/concepts/agents/about-agent-skills +- GitHub Copilot — About custom agents: https://docs.github.com/en/copilot/concepts/agents/copilot-cli/about-custom-agents +- Customize the Copilot cloud agent environment: https://docs.github.com/en/copilot/how-tos/use-copilot-agents/cloud-agent/customize-the-agent-environment +- obra/superpowers: https://github.com/obra/superpowers +- gsd-build/get-shit-done: https://github.com/gsd-build/get-shit-done +- torrust-tracker-deployer (validated reference implementation): https://github.com/torrust/torrust-tracker-deployer diff --git a/docs/adrs/README.md b/docs/adrs/README.md index 85986fc36..5fd40aa24 100644 --- a/docs/adrs/README.md +++ b/docs/adrs/README.md @@ -1,23 +1,32 @@ # Architectural Decision Records (ADRs) -This directory contains the architectural decision records (ADRs) for the -project. ADRs are a way to document the architectural decisions made in the -project. +This directory contains the architectural decision records (ADRs) for the project. +ADRs document architectural decisions — what was decided, why, and what alternatives +were considered. More info: <https://adr.github.io/>. -## How to add a new record +See [index.md](index.md) for the full list of ADRs. -For the prefix: +## How to Add a New ADR -```s +Generate the timestamp prefix (UTC): + +```shell date -u +"%Y%m%d%H%M%S" ``` -Then you can create a new markdown file with the following format: +Create a new Markdown file using the format `YYYYMMDDHHMMSS_snake_case_title.md`: -```s +```shell 20230510152112_title.md ``` -For the time being, we are not following any specific template. +Then add a row to the [Index](index.md) table. + +There is no rigid template. A typical ADR includes: + +- **Description** — the problem or context motivating the decision +- **Agreement** — what was decided and why +- **Date** — decision date (`YYYY-MM-DD`) +- **References** — related issues, PRs, external docs diff --git a/docs/adrs/index.md b/docs/adrs/index.md new file mode 100644 index 000000000..b6063e3ff --- /dev/null +++ b/docs/adrs/index.md @@ -0,0 +1,6 @@ +# ADR Index + +| ADR | Date | Title | Short Description | +| --------------------------------------------------------------------------------------- | ---------- | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------ | +| [20240227164834](20240227164834_use_plural_for_modules_containing_collections.md) | 2024-02-27 | Use plural for modules containing collections | Module names should use plural when they contain multiple types with the same responsibility (e.g. `requests/`, `responses/`). | +| [20260420200013](20260420200013_adopt_custom_github_copilot_aligned_agent_framework.md) | 2026-04-20 | Adopt a custom, GitHub-Copilot-aligned agent framework | Use AGENTS.md, Agent Skills, and Custom Agent profiles instead of third-party agent frameworks. | diff --git a/docs/issues/1697-ai-agent-configuration.md b/docs/issues/1697-ai-agent-configuration.md index 8e0e7b932..925f04ea5 100644 --- a/docs/issues/1697-ai-agent-configuration.md +++ b/docs/issues/1697-ai-agent-configuration.md @@ -64,24 +64,24 @@ effectively without requiring repeated manual instructions. Create `AGENTS.md` in the repository root, adapting the above files to the tracker. At minimum the file must cover: -- [ ] Repository link and project overview (language, license, MSRV, web framework, protocols, databases) -- [ ] Tech stack (languages, frameworks, databases, containerization, linting tools) -- [ ] Key directories (`src/`, `src/bin/`, `packages/`, `console/`, `contrib/`, `tests/`, `docs/`, `share/`, `storage/`, `.github/workflows/`) -- [ ] Package catalog (all workspace packages with their layer and description) -- [ ] Package naming conventions (`axum-*`, `*-server`, `*-core`, `*-protocol`) -- [ ] Key configuration files (`.markdownlint.json`, `.yamllint-ci.yml`, `.taplo.toml`, `cspell.json`, `rustfmt.toml`, etc.) -- [ ] Build & test commands (`cargo build`, `cargo test --doc`, `cargo test --all-targets`, E2E runner, benchmarks) -- [ ] Lint commands (`linter all` and individual linters; how to install the `linter` binary) -- [ ] Dependencies check (`cargo machete`) -- [ ] Code style (rustfmt rules, clippy policy, import grouping, per-format rules) -- [ ] Collaboration principles (no flattery, push back on weak ideas, flag blockers early) -- [ ] Essential rules (linting gate, GPG commit signing, no `storage/`/`target/` commits, `cargo machete`) -- [ ] Git workflow (branch naming, Conventional Commits, branch strategy: `develop` → `staging/main` → `main`) -- [ ] Development principles (observability, testability, modularity, extensibility; Beck's four rules) -- [ ] Container / Docker (key commands, ports, volume mount paths) -- [ ] Auto-invoke skills placeholder (to be filled in when `.github/skills/` is populated) -- [ ] Documentation quick-navigation table -- [ ] Add a brief entry to `docs/index.md` pointing contributors to `AGENTS.md`, `.github/skills/`, and `.github/agents/` +- [x] Repository link and project overview (language, license, MSRV, web framework, protocols, databases) +- [x] Tech stack (languages, frameworks, databases, containerization, linting tools) +- [x] Key directories (`src/`, `src/bin/`, `packages/`, `console/`, `contrib/`, `tests/`, `docs/`, `share/`, `storage/`, `.github/workflows/`) +- [x] Package catalog (all workspace packages with their layer and description) +- [x] Package naming conventions (`axum-*`, `*-server`, `*-core`, `*-protocol`) +- [x] Key configuration files (`.markdownlint.json`, `.yamllint-ci.yml`, `.taplo.toml`, `cspell.json`, `rustfmt.toml`, etc.) +- [x] Build & test commands (`cargo build`, `cargo test --doc`, `cargo test --all-targets`, E2E runner, benchmarks) +- [x] Lint commands (`linter all` and individual linters; how to install the `linter` binary) +- [x] Dependencies check (`cargo machete`) +- [x] Code style (rustfmt rules, clippy policy, import grouping, per-format rules) +- [x] Collaboration principles (no flattery, push back on weak ideas, flag blockers early) +- [x] Essential rules (linting gate, GPG commit signing, no `storage/`/`target/` commits, `cargo machete`) +- [x] Git workflow (branch naming, Conventional Commits, branch strategy: `develop` → `staging/main` → `main`) +- [x] Development principles (observability, testability, modularity, extensibility; Beck's four rules) +- [x] Container / Docker (key commands, ports, volume mount paths) +- [x] Auto-invoke skills placeholder (to be filled in when `.github/skills/` is populated) +- [x] Documentation quick-navigation table +- [x] Add a brief entry to `docs/index.md` pointing contributors to `AGENTS.md`, `.github/skills/`, and `.github/agents/` Commit message: `docs(agents): add root AGENTS.md` @@ -103,13 +103,13 @@ Checkpoint: Define reusable, project-specific skills that agents can load to perform specialized tasks on this repository consistently. -- [ ] Create `.github/skills/` directory -- [ ] Review and confirm the candidate skills listed below (add, remove, or adjust before starting implementation) -- [ ] For each skill, create a directory with: +- [x] Create `.github/skills/` directory +- [x] Review and confirm the candidate skills listed below (add, remove, or adjust before starting implementation) +- [x] For each skill, create a directory with: - `SKILL.md` — YAML frontmatter (`name`, `description`, optional `license`, `compatibility`) + step-by-step instructions - `scripts/` (optional) — executable scripts the agent can run - `references/` (optional) — additional reference documentation -- [ ] Validate skill files against the Agent Skills spec (name rules: lowercase, hyphens, no consecutive hyphens, max 64 chars; description: max 1024 chars) +- [x] Validate skill files against the Agent Skills spec (name rules: lowercase, hyphens, no consecutive hyphens, max 64 chars; description: max 1024 chars) **Candidate initial skills** (ported / adapted from `torrust-tracker-deployer`): @@ -131,37 +131,39 @@ Directory layout to mirror the deployer structure: testing/ ``` -**`add-new-skill`** — meta-skill: guide for creating new Agent Skills for this repository. +**`add-new-skill`** ✅ — meta-skill: guide for creating new Agent Skills for this repository. **`dev/git-workflow/`**: -- `commit-changes` — commit following Conventional Commits; pre-commit verification checklist. -- `create-feature-branch` — branch naming convention and lifecycle. -- `open-pull-request` — open a PR via GitHub CLI or GitHub MCP tool; pre-flight checks. -- `release-new-version` — version bump, signed release commit, signed tag, CI verification. -- `review-pr` — review a PR against Torrust quality standards and checklist. -- `run-linters` — run the full linting suite (`linter all`); fix individual linter failures. -- `run-pre-commit-checks` — mandatory quality gates before every commit. +- `commit-changes` ✅ — commit following Conventional Commits; pre-commit verification checklist. +- `create-feature-branch` ✅ — branch naming convention and lifecycle. +- `open-pull-request` ✅ — open a PR via GitHub CLI or GitHub MCP tool; pre-flight checks. +- `release-new-version` ✅ — version bump, signed release commit, signed tag, CI verification. +- `review-pr` ✅ — review a PR against Torrust quality standards and checklist. +- `run-linters` ✅ — run the full linting suite (`linter all`); fix individual linter failures. +- `run-pre-commit-checks` ✅ — mandatory quality gates before every commit. **`dev/maintenance/`**: -- `update-dependencies` — run `cargo update`, create branch, commit, push, open PR. +- `install-linter` ✅ — install the `linter` binary and its external tool dependencies. +- `setup-dev-environment` ✅ — full onboarding guide: system deps, Rust toolchain, storage dirs, linter, git hooks, smoke test. +- `update-dependencies` ✅ — run `cargo update`, create branch, commit, push, open PR. **`dev/planning/`**: -- `create-adr` — create an Architectural Decision Record in `docs/adrs/`. -- `create-issue` — draft and open a GitHub issue following project conventions. -- `write-markdown-docs` — GFM pitfalls (auto-links, ordered list numbering, etc.). -- `cleanup-completed-issues` — remove issue doc files and update roadmap after PR merge. +- `create-adr` ✅ — create an Architectural Decision Record in `docs/adrs/`. +- `create-issue` ✅ — draft and open a GitHub issue following project conventions. +- `write-markdown-docs` ✅ — GFM pitfalls (auto-links, ordered list numbering, etc.). +- `cleanup-completed-issues` ✅ — remove issue doc files and update roadmap after PR merge. **`dev/rust-code-quality/`**: -- `handle-errors-in-code` — `thiserror`-based structured errors; what/where/when/why context. -- `handle-secrets` — wrapper types for tokens/passwords; never use plain `String` for secrets. +- `handle-errors-in-code` ✅ — `thiserror`-based structured errors; what/where/when/why context. +- `handle-secrets` ✅ — wrapper types for tokens/passwords; never use plain `String` for secrets. **`dev/testing/`**: -- `write-unit-test` — `it_should_*` naming, AAA pattern, `MockClock`, `TempDir`, `rstest`. +- `write-unit-test` ✅ — `it_should_*` naming, AAA pattern, `MockClock`, `TempDir`, `rstest`. Commit message: `docs(agents): add initial agent skills under .github/skills/` @@ -185,22 +187,36 @@ Checkpoint: Define custom GitHub Copilot agents tailored to Torrust project workflows so that specialized tasks can be delegated to focused agents with the right prompt context. -- [ ] Create `.github/agents/` directory -- [ ] Identify workflows that benefit from a dedicated agent (e.g. issue implementation planner, code reviewer, documentation writer, release drafter) -- [ ] For each agent, create `.github/agents/<agent-name>.md` with: +- [x] Create `.github/agents/` directory +- [x] Identify workflows that benefit from a dedicated agent +- [x] For each agent, create `.github/agents/<agent-name>.md` with: - YAML frontmatter: `name` (optional), `description`, optional `tools` - Prompt body: role definition, scope, constraints, and step-by-step instructions -- [ ] Test each custom agent by assigning it to a task or issue in GitHub Copilot CLI +- [x] Test each custom agent by assigning it to a task or issue in GitHub Copilot CLI **Candidate initial agents**: -- `committer` — commit specialist: reads branch/diff, runs pre-commit checks (`linter all`), - proposes a GPG-signed Conventional Commit message, and creates the commit only after scope and - checks are clear. Reference: +- `committer` ✅ — commit specialist: reads branch/diff, runs pre-commit checks + (`./scripts/pre-commit.sh`), proposes a GPG-signed Conventional Commit message, and creates + the commit only after scope and checks are clear. Reference: [`torrust-tracker-demo/.github/agents/commiter.agent.md`](https://raw.githubusercontent.com/torrust/torrust-tracker-demo/refs/heads/main/.github/agents/commiter.agent.md) -- `issue-planner` — given a GitHub issue, produces a detailed implementation plan document (like the ones in `docs/issues/`) including branch name, task breakdown, checkpoints, and commit message suggestions -- `code-reviewer` — reviews PRs against Torrust coding conventions, clippy rules, and security considerations -- `docs-writer` — creates or updates documentation files following the existing docs structure +- `implementer` ✅ — software implementer that applies Test-Driven Development and seeks the + simplest solution. Follows a structured process: analyse → decompose into small steps → + implement with TDD → call the Complexity Auditor after each step → call the Committer when + ready. Guided by Beck's Four Rules of Simple Design. +- `complexity-auditor` ✅ — code quality auditor that checks cyclomatic and cognitive complexity + of changes after each implementation step. Reports PASS/WARN/FAIL per function using thresholds + and Clippy's `cognitive_complexity` lint. Called by the Implementer; can also be invoked + directly. + +**Future agents** (not yet implemented): + +- `issue-planner` — given a GitHub issue, produces a detailed implementation plan document + (like those in `docs/issues/`) including branch name, task breakdown, checkpoints, and commit + message suggestions. +- `code-reviewer` — reviews PRs against Torrust coding conventions, clippy rules, and security + considerations. +- `docs-writer` — creates or updates documentation files following the existing docs structure. Commit message: `docs(agents): add initial custom agents under .github/agents/` @@ -223,6 +239,13 @@ Once the root file is stable, evaluate whether any workspace packages have suffi conventions or setup to warrant their own `AGENTS.md`. This can be tracked as a separate follow-up issue. +- [x] Evaluate workspace packages for package-specific conventions +- [x] Add `packages/AGENTS.md` — guidance scoped to all workspace packages +- [x] Add `src/AGENTS.md` — guidance scoped to the main binary/library source + +> **Note**: Completed as part of Task 1. `packages/AGENTS.md` and `src/AGENTS.md` were added +> alongside the root `AGENTS.md`. + --- ### Task 5: Add `copilot-setup-steps.yml` workflow @@ -241,19 +264,20 @@ https://raw.githubusercontent.com/torrust/torrust-tracker-deployer/refs/heads/ma Minimum steps to include: -- [ ] Trigger on `workflow_dispatch`, `push` and `pull_request` (scoped to the workflow file path) -- [ ] `copilot-setup-steps` job on `ubuntu-latest`, `timeout-minutes: 30`, `permissions: contents: read` -- [ ] `actions/checkout@v5` — check out the repository (verify this is still the latest stable +- [x] Trigger on `workflow_dispatch`, `push` and `pull_request` (scoped to the workflow file path) +- [x] `copilot-setup-steps` job on `ubuntu-latest`, `timeout-minutes: 30`, `permissions: contents: read` +- [x] `actions/checkout@v6` — check out the repository (verify this is still the latest stable version on the GitHub Marketplace before merging) -- [ ] `dtolnay/rust-toolchain@stable` — install the stable Rust toolchain (pin MSRV if needed) -- [ ] `Swatinem/rust-cache@v2` — cache `target/` and `~/.cargo` between runs -- [ ] `cargo build` warm-up — build the workspace (or key packages) so incremental compilation is +- [x] `dtolnay/rust-toolchain@stable` — install the stable Rust toolchain (pin MSRV if needed) +- [x] `Swatinem/rust-cache@v2` — cache `target/` and `~/.cargo` between runs +- [x] `cargo build` warm-up — build the workspace (or key packages) so incremental compilation is ready when Copilot starts editing -- [ ] Install the `linter` binary — +- [x] Install the `linter` binary — `cargo install --locked --git https://github.com/torrust/torrust-linting --bin linter` -- [ ] Install `cargo-machete` — `cargo install cargo-machete`; ensures Copilot can run unused +- [x] Install `cargo-machete` — `cargo install cargo-machete`; ensures Copilot can run unused dependency checks (`cargo machete`) as required by the essential rules -- [ ] Smoke-check: run `linter all` to confirm the environment is healthy before Copilot begins +- [x] Smoke-check: run `linter all` to confirm the environment is healthy before Copilot begins +- [x] Install Git pre-commit hooks — `./scripts/install-git-hooks.sh` Commit message: `ci(copilot): add copilot-setup-steps workflow` @@ -307,8 +331,8 @@ frameworks that were evaluated. 8. Incremental adoption — individual skills, custom agents, or patterns from those frameworks can still be cherry-picked and integrated progressively if specific value is identified. -- [ ] Create `docs/adrs/<YYYYMMDDHHMMSS>_ai-agent-framework-approach.md` using the `create-adr` skill -- [ ] Record the decision, the alternatives considered, and the reasoning above +- [x] Create `docs/adrs/<YYYYMMDDHHMMSS>_ai-agent-framework-approach.md` using the `create-adr` skill +- [x] Record the decision, the alternatives considered, and the reasoning above Commit message: `docs(adrs): add ADR for AI agent framework approach` @@ -325,10 +349,10 @@ Checkpoint: ## Acceptance Criteria -- [ ] `AGENTS.md` exists at the repo root and contains accurate, up-to-date project guidance. -- [ ] At least one skill is available under `.github/skills/` and can be successfully activated by GitHub Copilot. -- [ ] At least one custom agent is available under `.github/agents/` and can be assigned to a task. -- [ ] `copilot-setup-steps.yml` exists, the workflow runs successfully in the **Actions** tab, and `linter all` exits with code `0` inside it. -- [ ] An ADR exists in `docs/adrs/` documenting the decision to use a custom GitHub-Copilot-aligned agent framework. -- [ ] All files pass spelling checks (`cspell`) and markdown linting. -- [ ] A brief entry in `docs/index.md` points contributors to `AGENTS.md`, `.github/skills/`, and `.github/agents/`. +- [x] `AGENTS.md` exists at the repo root and contains accurate, up-to-date project guidance. +- [x] At least one skill is available under `.github/skills/` and can be successfully activated by GitHub Copilot. +- [x] At least one custom agent is available under `.github/agents/` and can be assigned to a task. +- [x] `copilot-setup-steps.yml` exists, the workflow runs successfully in the **Actions** tab, and `linter all` exits with code `0` inside it. +- [x] An ADR exists in `docs/adrs/` documenting the decision to use a custom GitHub-Copilot-aligned agent framework. +- [x] All files pass spelling checks (`cspell`) and markdown linting. +- [x] A brief entry in `docs/index.md` points contributors to `AGENTS.md`, `.github/skills/`, and `.github/agents/`. diff --git a/docs/templates/ADR.md b/docs/templates/ADR.md new file mode 100644 index 000000000..fa8aebe27 --- /dev/null +++ b/docs/templates/ADR.md @@ -0,0 +1,24 @@ +# [Title] + +## Description + +What is the issue motivating this decision? Provide enough context for future +readers who have no prior background. + +## Agreement + +What was decided and why? Be concrete. Include code examples if the decision +involves specific patterns. + +Optional sub-sections: + +- **Alternatives Considered** — other options explored and why they were rejected +- **Consequences** — positive and negative effects of the decision + +## Date + +YYYY-MM-DD + +## References + +Links to related issues, PRs, ADRs, and external documentation. diff --git a/docs/templates/ISSUE.md b/docs/templates/ISSUE.md new file mode 100644 index 000000000..7c899bacd --- /dev/null +++ b/docs/templates/ISSUE.md @@ -0,0 +1,33 @@ +# Issue: {Title} + +## Overview + +Clear description of what needs to be done and why. + +## Goals + +- [ ] Goal 1 +- [ ] Goal 2 + +## Implementation Plan + +### Task 1: {Task Title} + +- [ ] Sub-task a +- [ ] Sub-task b + +### Task 2: {Task Title} + +- [ ] Sub-task a +- [ ] Sub-task b + +## Acceptance Criteria + +- [ ] All tests pass +- [ ] `linter all` exits with code `0` +- [ ] Documentation updated + +## References + +- Related issues: #{number} +- Related ADRs: `docs/adrs/...` diff --git a/packages/AGENTS.md b/packages/AGENTS.md new file mode 100644 index 000000000..231bfe3a9 --- /dev/null +++ b/packages/AGENTS.md @@ -0,0 +1,152 @@ +# Torrust Tracker — Packages + +This directory contains all Cargo workspace packages. All domain logic, protocol +implementations, server infrastructure, and utility libraries live here. + +For full project context see the [root AGENTS.md](../AGENTS.md). + +## Architecture + +Packages are organized in strict layers. Dependencies only flow downward — a package may only +depend on packages in the same layer or a lower one. + +```text +┌────────────────────────────────────────────────────────────────┐ +│ Servers (delivery layer) │ +│ axum-http-tracker-server axum-rest-tracker-api-server │ +│ axum-health-check-api-server udp-tracker-server │ +├────────────────────────────────────────────────────────────────┤ +│ Core (domain layer) │ +│ http-tracker-core udp-tracker-core tracker-core │ +│ rest-tracker-api-core swarm-coordination-registry │ +├────────────────────────────────────────────────────────────────┤ +│ Protocols │ +│ http-protocol udp-protocol │ +├────────────────────────────────────────────────────────────────┤ +│ Domain / Shared │ +│ torrent-repository configuration primitives │ +│ events metrics clock located-error server-lib │ +├────────────────────────────────────────────────────────────────┤ +│ Utilities / Test support │ +│ test-helpers │ +└────────────────────────────────────────────────────────────────┘ +``` + +**Key architectural rule**: Servers contain only network I/O logic. All business rules live in +`*-core` packages. Protocol parsing is isolated in `*-protocol` packages. + +See [docs/packages.md](../docs/packages.md) for a full diagram. + +## Package Catalog + +### Servers (`axum-*`, `udp-tracker-server`) + +Delivery layer — accept network connections, dispatch to core handlers, return responses. +These packages must not contain business logic. + +| Package | Entry point | Protocol | +| ------------------------------ | ------------ | ----------- | +| `axum-http-tracker-server` | `src/lib.rs` | HTTP BEP 3 | +| `axum-rest-tracker-api-server` | `src/lib.rs` | REST (JSON) | +| `axum-health-check-api-server` | `src/lib.rs` | HTTP | +| `axum-server` | `src/lib.rs` | Axum base | +| `udp-tracker-server` | `src/lib.rs` | UDP BEP 15 | + +### Core (`*-core`) + +Domain layer — business rules, request validation, response building. No Axum or networking +imports. Each core package exposes a `container` module that wires up its dependencies via +dependency injection. + +| Package | Purpose | +| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| `tracker-core` | Central peer management: announce/scrape handlers, auth, whitelist, database abstraction (SQLite/MySQL drivers in `src/databases/driver/`) | +| `http-tracker-core` | HTTP-specific validation and response formatting | +| `udp-tracker-core` | UDP connection cookies, crypto, banning logic | +| `rest-tracker-api-core` | REST API statistics and container wiring | +| `swarm-coordination-registry` | Registry of torrents and their peer swarms | + +### Protocols (`*-protocol`) + +Strict BEP implementations — parse and serialize wire formats only. No tracker logic. + +| Package | BEP | Handles | +| --------------- | ------ | -------------------------------------------------------------- | +| `http-protocol` | BEP 3 | URL parameter parsing, bencoded responses, compact peer format | +| `udp-protocol` | BEP 15 | Message framing, connection IDs, transaction IDs | + +### Domain / Shared + +| Package | Purpose | +| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `torrent-repository` | Torrent metadata storage; InfoHash management; peer coordination | +| `configuration` | Config file parsing (`share/default/config/`) and env var loading (`TORRUST_TRACKER_CONFIG_TOML`, `TORRUST_TRACKER_CONFIG_TOML_PATH`); versioned under `src/v2_0_0/` | +| `primitives` | Core domain types: `InfoHash`, `PeerId`, `Peer`, `SwarmMetadata`, `ServiceBinding` | +| `events` | Async event bus (broadcaster / receiver / shutdown) used across packages | +| `metrics` | Prometheus-compatible metrics: counters, gauges, labels, samples | +| `server-lib` | Shared HTTP server utilities: logging, service registrar, signal handling | +| `clock` | Mockable time source — use `clock::Working` in production, `clock::Stopped` in tests | +| `located-error` | Error decorator that captures the source file/line of the original error | + +### Client Tools + +| Package | Purpose | +| ------------------------- | -------------------------------------------------------- | +| `tracker-client` | Generic HTTP and UDP tracker clients (used by E2E tests) | +| `rest-tracker-api-client` | Typed REST API client library | + +### Utilities / Test support + +| Package | Purpose | +| --------------------------------- | ---------------------------------------------------------- | +| `test-helpers` | Mock servers, test data generators, shared test fixtures | +| `torrent-repository-benchmarking` | Criterion benchmarks for alternative torrent storage impls | + +## Naming Conventions + +| Prefix / Suffix | Responsibility | May depend on | +| --------------- | ----------------------------------------- | ----------------------------- | +| `axum-*` | HTTP server components using Axum | `*-core`, Axum framework | +| `*-server` | Server implementations | Corresponding `*-core` | +| `*-core` | Domain logic and business rules | `*-protocol`, domain packages | +| `*-protocol` | BitTorrent protocol parsing/serialization | `primitives` | +| `udp-*` | UDP-specific implementations | `tracker-core` | +| `http-*` | HTTP-specific implementations | `tracker-core` | + +## Adding or Modifying a Package + +1. Create the directory under `packages/<new-package>/` with a `Cargo.toml` and `src/lib.rs`. +2. Add the package to the workspace `[members]` in the root `Cargo.toml`. +3. Follow the naming conventions above. +4. Each package must have: + - A crate-level doc comment in `src/lib.rs` explaining its purpose and layer. + - At minimum one unit test (doc-test acceptable for simple utility crates). +5. Run `cargo machete` after adding dependencies — unused deps must not be committed. +6. Run `linter all` before committing. + +## Testing Packages + +```sh +# All tests for a specific package +cargo test -p <package-name> + +# Doc tests only +cargo test --doc -p <package-name> + +# MySQL-specific tests in tracker-core (requires a running MySQL instance) +TORRUST_TRACKER_CORE_RUN_MYSQL_DRIVER_TEST=true cargo test -p bittorrent-tracker-core +``` + +Use `clock::Stopped` (from the `clock` package) in unit tests that need deterministic time. +Use `test-helpers` for mock tracker servers in integration tests. + +## Key Dependency Notes + +- `swarm-coordination-registry` is the authoritative store for peer swarms; `tracker-core` + delegates peer lookups to it. +- `configuration` is the only package that reads from the filesystem or environment at startup; + other packages receive config structs as arguments. +- `located-error` wraps any `std::error::Error` — use it at module boundaries to preserve + error origin context without losing the original error type. +- `events` provides the only sanctioned inter-package async communication channel; avoid direct + `tokio::sync` coupling between packages. diff --git a/project-words.txt b/project-words.txt index 6a8a264ad..9458ebbf3 100644 --- a/project-words.txt +++ b/project-words.txt @@ -1,19 +1,23 @@ Addrs adduser alekitto +analyse appuser Arvid ASMS asyn autoclean AUTOINCREMENT +autolinks automock Avicora Azureus +backlinks bdecode bencode bencoded bencoding +behaviour beps binascii binstall @@ -44,6 +48,7 @@ Containerfile conv curr cvar +cyclomatic Cyberneering dashmap datagram @@ -124,6 +129,7 @@ ostr Pando peekable peerlist +penalise programatik proot proto @@ -134,6 +140,7 @@ Rasterbar realpath reannounce Registar +repomix repr reqs reqwest @@ -159,6 +166,7 @@ SHLVL skiplist slowloris socketaddr +specialised sqllite subsec Swatinem @@ -171,6 +179,7 @@ testcontainers Tera thiserror tlsv +toplevel Torrentstorm torrust torrustracker @@ -183,6 +192,7 @@ underflows Unsendable untuple uroot +usize Vagaa valgrind Vitaly diff --git a/scripts/install-git-hooks.sh b/scripts/install-git-hooks.sh new file mode 100755 index 000000000..478377791 --- /dev/null +++ b/scripts/install-git-hooks.sh @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +# Install project Git hooks from .githooks/ into .git/hooks/. +# +# Usage: +# ./scripts/install-git-hooks.sh +# +# Run once after cloning the repository. Re-run to update hooks after +# they change. + +set -euo pipefail + +REPO_ROOT="$(git rev-parse --show-toplevel)" +HOOKS_SRC="${REPO_ROOT}/.githooks" +HOOKS_DST="$(git rev-parse --git-path hooks)" +mkdir -p "${HOOKS_DST}" + +if [ ! -d "${HOOKS_SRC}" ]; then + echo "ERROR: .githooks/ directory not found at ${HOOKS_SRC}" + exit 1 +fi + +installed=0 + +for hook in "${HOOKS_SRC}"/*; do + hook_name="$(basename "${hook}")" + dest="${HOOKS_DST}/${hook_name}" + + cp "${hook}" "${dest}" + chmod +x "${dest}" + + echo "Installed: ${hook_name} → .git/hooks/${hook_name}" + installed=$((installed + 1)) +done + +echo "" +echo "==========================================" +echo "SUCCESS: ${installed} hook(s) installed." +echo "==========================================" diff --git a/scripts/pre-commit.sh b/scripts/pre-commit.sh new file mode 100755 index 000000000..c360ad6b6 --- /dev/null +++ b/scripts/pre-commit.sh @@ -0,0 +1,83 @@ +#!/bin/bash +# Pre-commit verification script +# Run all mandatory checks before committing changes. +# +# Usage: +# ./scripts/pre-commit.sh +# +# Expected runtime: ~3 minutes on a modern developer machine. +# AI agents: set a per-command timeout of at least 5 minutes before invoking this script. +# +# All steps must pass (exit 0) before committing. + +set -euo pipefail + +# ============================================================================ +# STEPS +# ============================================================================ +# Each step: "description|success_message|command" + +declare -a STEPS=( + "Checking for unused dependencies (cargo machete)|No unused dependencies found|cargo machete" + "Running all linters|All linters passed|linter all" + "Running documentation tests|Documentation tests passed|cargo test --doc --workspace" + "Running all tests|All tests passed|cargo test --tests --benches --examples --workspace --all-targets --all-features" +) + +# ============================================================================ +# HELPER FUNCTIONS +# ============================================================================ + +format_time() { + local total_seconds=$1 + local minutes=$((total_seconds / 60)) + local seconds=$((total_seconds % 60)) + if [ "$minutes" -gt 0 ]; then + echo "${minutes}m ${seconds}s" + else + echo "${seconds}s" + fi +} + +run_step() { + local step_number=$1 + local total_steps=$2 + local description=$3 + local success_message=$4 + local command=$5 + + echo "[Step ${step_number}/${total_steps}] ${description}..." + + local step_start=$SECONDS + local -a cmd_array + read -ra cmd_array <<< "${command}" + "${cmd_array[@]}" + local step_elapsed=$((SECONDS - step_start)) + + echo "PASSED: ${success_message} ($(format_time "${step_elapsed}"))" + echo +} + +trap 'echo ""; echo "=========================================="; echo "FAILED: Pre-commit checks failed!"; echo "Fix the errors above before committing."; echo "=========================================="; exit 1' ERR + +# ============================================================================ +# MAIN +# ============================================================================ + +TOTAL_START=$SECONDS +TOTAL_STEPS=${#STEPS[@]} + +echo "Running pre-commit checks..." +echo + +for i in "${!STEPS[@]}"; do + IFS='|' read -r description success_message command <<< "${STEPS[$i]}" + run_step $((i + 1)) "${TOTAL_STEPS}" "${description}" "${success_message}" "${command}" +done + +TOTAL_ELAPSED=$((SECONDS - TOTAL_START)) +echo "==========================================" +echo "SUCCESS: All pre-commit checks passed! ($(format_time "${TOTAL_ELAPSED}"))" +echo "==========================================" +echo +echo "You can now safely stage and commit your changes." diff --git a/src/AGENTS.md b/src/AGENTS.md new file mode 100644 index 000000000..88296f152 --- /dev/null +++ b/src/AGENTS.md @@ -0,0 +1,109 @@ +# `src/` — Binary and Library Entry Points + +This directory contains only the top-level wiring of the application: the binary entry points, +the bootstrap sequence, and the dependency-injection container. All domain logic lives in +`packages/`; this directory merely assembles and launches it. + +## File Map + +| Path | Purpose | +| --------------------------- | ------------------------------------------------------------------------------------------------------------------------- | +| `main.rs` | Binary entry point. Calls `app::run()`, waits for Ctrl-C, then cancels jobs and waits for graceful shutdown. | +| `lib.rs` | Library crate root and crate-level documentation. Re-exports the public API used by integration tests and other binaries. | +| `app.rs` | `run()` and `start()` — orchestrates the full startup sequence (setup → load data from DB → start jobs). | +| `container.rs` | `AppContainer` — dependency-injection struct that holds `Arc`-wrapped instances of every per-layer container. | +| `bootstrap/app.rs` | `setup()` — loads config, validates it, initializes logging and global services, builds `AppContainer`. | +| `bootstrap/config.rs` | `initialize_configuration()` — reads config from the environment / file. | +| `bootstrap/jobs/` | One module per service: each module exposes a starter function called from `app::start_jobs`. | +| `bootstrap/jobs/manager.rs` | `JobManager` — collects `JoinHandle`s, owns the `CancellationToken`, and drives graceful shutdown. | +| `bin/e2e_tests_runner.rs` | Binary that runs E2E tests by delegating to `src/console/ci/`. | +| `bin/http_health_check.rs` | Minimal HTTP health-check binary used inside containers (avoids curl/wget dependency). | +| `bin/profiling.rs` | Binary for Valgrind / kcachegrind profiling sessions. | +| `console/` | Internal console apps (`ci/e2e`, `profiling`) used by the extra binaries above. | + +## Bootstrap Flow + +```text +main() + └─ app::run() + ├─ bootstrap::app::setup() + │ ├─ bootstrap::config::initialize_configuration() ← reads TOML / env vars + │ ├─ configuration.validate() ← panics on invalid config + │ ├─ initialize_global_services() ← logging, crypto seed + │ └─ AppContainer::initialize(&configuration) ← builds all containers + │ + └─ app::start(&config, &app_container) + ├─ load_data_from_database() ← peer keys, whitelist, metrics + └─ start_jobs() + ├─ start_swarm_coordination_registry_event_listener + ├─ start_tracker_core_event_listener + ├─ start_http_core_event_listener + ├─ start_udp_core_event_listener + ├─ start_udp_server_stats_event_listener + ├─ start_udp_server_banning_event_listener + ├─ start_the_udp_instances ← one job per configured UDP bind address + ├─ start_the_http_instances ← one job per configured HTTP bind address + ├─ start_torrent_cleanup + ├─ start_peers_inactivity_update + ├─ start_the_http_api + └─ start_health_check_api ← always started +``` + +Shutdown (`main`): receives `Ctrl-C` → calls `jobs.cancel()` (fires the `CancellationToken`) → +waits up to 10 seconds for all `JoinHandle`s to complete. + +## `AppContainer` + +`AppContainer` (`container.rs`) is a plain struct — not a framework, not a trait object tree. +It holds one `Arc<…Container>` per architectural layer: + +| Field | Layer / Package | +| ------------------------------------------------------------------------------------------------ | -------------------------------------------------------- | +| `registar` | `server-lib` — tracks active server socket registrations | +| `swarm_coordination_registry_container` | `swarm-coordination-registry` | +| `tracker_core_container` | `tracker-core` | +| `http_tracker_core_services` / `http_tracker_instance_containers` | `http-tracker-core` | +| `udp_tracker_core_services` / `udp_tracker_server_container` / `udp_tracker_instance_containers` | `udp-tracker-core` / `udp-tracker-server` | + +`AppContainer::initialize` is the only place where domain containers are constructed. +Every `bootstrap/jobs/` starter receives an `&Arc<AppContainer>` and pulls out exactly what it +needs — no globals, no lazy statics for domain objects. + +## `JobManager` + +`JobManager` (`bootstrap/jobs/manager.rs`) is a thin wrapper around a `Vec<Job>` (each `Job` +holds a name + `JoinHandle<()>`) and a shared `CancellationToken`: + +- `push(name, handle)` — registers a job. +- `push_opt(name, handle)` — convenience for jobs that may be disabled. +- `cancel()` — fires the token; all jobs that own a clone of it will observe cancellation. +- `wait_for_all(timeout)` — joins all handles with a timeout, logging warnings for any that + exceed it. + +## Adding a New Service + +When wiring a new server or background task, follow this checklist in order: + +1. **Package** — add the new crate under `packages/` with the appropriate layer prefix. +2. **Container field** — add an `Arc<NewServiceContainer>` field to `AppContainer` and + initialize it inside `AppContainer::initialize`. +3. **Job launcher** — create `src/bootstrap/jobs/new_service.rs` and register it in + `src/bootstrap/jobs/mod.rs`. +4. **Wire into `app::start_jobs`** — call the new starter function and push its handle to + `job_manager`. +5. **Graceful shutdown** — ensure the new service listens for the `CancellationToken` passed + from `JobManager`. +6. **Config guard** — if the service is optional, gate the starter behind the appropriate + config field and use `push_opt`. + +## Key Rules for This Directory + +- **No domain logic here.** This directory is pure wiring. Business rules belong in `packages/`. +- **No globals for domain objects.** All state flows through `AppContainer`. +- **Startup errors panic.** `bootstrap::app::setup()` panics on invalid config or a bad crypto + seed — this is intentional (fail fast before binding ports). +- **Health check always starts.** The health-check API job is unconditional — do not gate it + behind a config flag. +- **`lib.rs` is the integration-test surface.** Integration tests import + `torrust_tracker_lib::…`. Keep the public API in `lib.rs` stable; avoid leaking internal + bootstrap details.