Merge torrust#853: refactor!: secure JWT authentication with RS256 and token revocation (ADR-T-007)

6126eec fix(auth): harden token parsing and remove panics in ban logic (Peer Cat)
593276a refactor(jwt): consolidate session validation into single code path (ADR-T-007 Phase 7) (Peer Cat)
e23cd3f feat(jwt): atomic token revocation and hardened session validation (ADR-T-007) (Peer Cat)
49504f6 feat: add generate-auth-keypair CLI and container auto-generation (ADR-T-007 Phase 6) (Peer Cat)
4183c12 feat(jwt): auto-generate ephemeral RSA keys when none configured (ADR-T-007 Phase 5) (Peer Cat)
7c8ca72 fix(e2e): provision JWT PEM keys and fix health-check script (Peer Cat)
4adbe3c feat(jwt): add token revocation via per-user generation counter (ADR-T-007 Phase 4) (Peer Cat)
40fe5e5 refactor(jwt)!: switch from HMAC-HS256 to RS256 asymmetric signing (ADR-T-007 Phase 3) (Peer Cat)
8411e7d refactor(jwt): implement ADR-T-007 Phase 2 — per-purpose signing keys and RFC 7519 claims (Peer Cat)
b6f51c3 refactor!: centralise JWT handling into dedicated module (ADR-T-007 Phase 1) (Peer Cat)

Pull request description:

  ### Summary

  Replace the JWT authentication system end-to-end: HMAC-HS256 shared secret → RS256 asymmetric signing, RFC 7519 claims, per-user token revocation with atomic database operations, ephemeral key auto-generation, a CLI for persistent key provisioning, and consolidated session validation via a single code path. Implemented across 7 phases in 9 incremental commits (ADR-T-007).

  ### Motivation

  The previous setup used a single low-entropy HMAC-HS256 secret for all token types, had no standard registered claims, baked stale role data into payloads with no way to invalidate it, hard-coded expiration durations, panicked on encode/decode failures, and offered no mechanism to revoke tokens on password change or ban.

  ### Changes by phase

  | Commit | Phase | Description |
  |--------|-------|-------------|
  | `b6f51c3` | 1 — Structural cleanup | New `jwt.rs` module centralises all `jsonwebtoken` usage; `sign()`/`parse_token()` return `Result`; configurable token lifetimes replace hard-coded durations; config key renamed to `jwt_signing_secret` |
  | `8411e7d` | 2 — Claim redesign | Per-purpose signing keys (`session` / `email_verification`); RFC 7519 registered claims (`sub`, `iss`, `aud`, `iat`, `exp`); 32-byte minimum key length enforced; role/username fields marked advisory-only |
  | `40fe5e5` | 3 — RS256 asymmetric signing | RSA-2048 PEM key pair replaces HMAC secrets; `kid` header (SHA-256 of public key) for future key rotation; verification is synchronous since keys are pre-loaded at startup |
  | `4adbe3c` | 4 — Token revocation | `token_generation` column added to `torrust_users` (MySQL + SQLite migrations); `gen` claim in session JWTs; password changes, role grants, and bans increment the counter, invalidating all outstanding tokens |
  | `7c8ca72` | — (fix) | Provision PEM keys in E2E container scripts; make health-check script directly invocable |
  | `4183c12` | 5 — Ephemeral keys | Remove shipped development key pair; auto-generate RSA-2048 in-memory at startup when no keys are configured; sessions work immediately but don't survive restarts |
  | `49504f6` | 6 — CLI & container keys | `torrust-generate-auth-keypair` binary writes PEM key pair to stdout; container entry script auto-generates persistent keys on first boot into `/etc/torrust/index/auth/` with restrictive permissions |
  | `e23cd3f` | — (hardening) | Atomic database methods for password change, ban, and admin grant (`transaction` / single `UPDATE`); generation check tightened from `<` to `!=`; defence-in-depth `is_user_banned` fallback; `BearerToken::value()` → `as_str()`; `parse_token` requires space after `"Bearer"` prefix |
  | `593276a` | 7 — Consolidate validation | `JsonWebToken::validate_session` becomes the sole entry point for session-token validation — JWT signature/expiry, generation counter, and ban check all happen in one place; removes ~45 lines of duplicated logic across three call sites |

  ### Breaking changes

  - **Config (Phase 1):** `auth.user_claim_token_pepper` → `auth.jwt_signing_secret` (legacy alias accepted).
  - **Config (Phase 3):** `auth.session_signing_key` and `auth.email_verification_signing_key` removed. Replaced by `auth.private_key_path` / `auth.public_key_path` (or inline `_pem` variants). Existing HS256 tokens are rejected after upgrade.
  - **Config (Phase 5):** Key paths are now optional — omitting them triggers ephemeral key generation instead of an error.
  - **Database (Phase 4):** New `token_generation BIGINT/INTEGER NOT NULL DEFAULT 0` column on `torrust_users` (migration included for both MySQL and SQLite).
  - **API:** `BearerToken` extractor rejects missing or malformed `Authorization` headers at the extraction boundary. `get_optional_logged_in_user` removed; logic moved into extractors.

  ### New files

  | File | Purpose |
  |------|---------|
  | jwt.rs | Centralised JWT module (370 lines) — signing, verification, claims, ephemeral key generation, `validate_session` |
  | generate_auth_keypair.rs | CLI tool for RSA key pair generation (110 lines) |
  | jwt.rs | Crate tests: sign/verify round-trips, audience cross-contamination, tampered tokens |
  | auth.rs | Crate tests: `parse_token` whitespace trimming, empty bearer, non-ASCII rejection |
  | 007-jwt-system-refactor.md | ADR documenting the full 7-phase design |
  | entry_script_sh | Container auto-generation of persistent auth keys |
  | `migrations/*/20260414000000_torrust_user_token_generation.sql` | Token generation column (MySQL + SQLite) |

  ### Testing

  - New crate tests in jwt.rs and auth.rs covering sign/verify round-trips, audience enforcement, tampered tokens, bearer parsing edge cases
  - All existing E2E and integration tests updated for the new config shape and passing
  - Ephemeral key path exercised by default in tests

  ### References

  - ADR-T-007: JWT System Refactor (007-jwt-system-refactor.md)

ACKs for top commit:
  peer-cat:
    ACK 6126eec
  da2ce7:
    ACK 6126eec

Tree-SHA512: 7a385c184f8ff0c3cde110f97decd62cd241d92a0671eb3479c988e8de11898e50b8631db666179775c942023ed09619ed03e012d4ae08449226f1cd51e5fd80

Author: da2ce7

.env.local

@@ -1,6 +1,5 @@
 DATABASE_URL=sqlite://storage/database/data.db?mode=rwc
 TORRUST_INDEX_CONFIG_TOML=
-TORRUST_INDEX_CONFIG_OVERRIDE_AUTH__SECRET_KEY=MaxVerstappenWC2021
 USER_ID=1000
 TORRUST_TRACKER_CONFIG_TOML=
 TORRUST_TRACKER_CONFIG_OVERRIDE_CORE__DATABASE__DRIVER=Sqlite3

CHANGELOG.md

@@ -13,12 +13,67 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - 188 crate-level tests for the domain error system (`src/tests/errors/`):
   status-code mapping, display messages, `From` impl coverage, and
   `ApiError` delegation (ADR-T-006 §1–§4).
+- ADR-T-007: Document rationale for JWT system refactor.
+- Centralised JWT module (`src/jwt.rs`) consolidating all `jsonwebtoken` usage:
+  key loading, signing, verification, and algorithm configuration.
+- `SessionClaims` with RFC 7519 registered claims (`sub`, `iss`, `aud`, `iat`,
+  `exp`) plus advisory `role`, `username`, and revocation `gen` fields.
+- `VerifyClaims` with `aud: "email-verification"` for purpose separation.
+- RSA key pair configuration: `auth.private_key_path` / `auth.public_key_path`
+  (or inline PEM via `auth.private_key_pem` / `auth.public_key_pem`).
+- Ephemeral auto-generated RSA-2048 key pair when no keys are configured.
+  Sessions do not survive server restarts with ephemeral keys. Deployers who
+  want persistent sessions supply their own key pair via config.
+- `torrust-generate-auth-keypair` CLI binary for generating RSA-2048 key pairs.
+  Outputs both PEM blocks to stdout; refuses to run if stdout is a terminal.
+- Container auto-generation of persistent auth keys on first boot. The entry
+  script runs `torrust-generate-auth-keypair` and writes the PEM files to
+  `/etc/torrust/index/auth/` on the volume. Sessions survive restarts with no
+  manual setup.
+- `kid` (Key ID) header in every JWT for future key rotation support.
+- Configurable token lifetimes: `auth.session_token_lifetime_secs` (default:
+  2 weeks) and `auth.email_verification_token_lifetime_secs` (default: ~10 years).
+- `token_generation` column on `torrust_users` (migration for SQLite and MySQL).
+- Token revocation: password changes, role changes (admin grant), and bans
+  increment `token_generation`; tokens with an older `gen` claim are rejected.
+- Consolidated session validation: `JsonWebToken::validate_session` is the
+  sole entry point for verifying a session JWT, checking the token-generation
+  counter, and rejecting banned users. All callers delegate here.
+- `BearerToken` extractor rejects missing/malformed `Authorization` headers at
+  the extraction boundary (`AuthError::TokenNotFound` / `AuthError::TokenInvalid`).
+- `ExtractOptionalLoggedInUser` catches extraction rejection and returns `None`
+  for anonymous requests.
+- `AuthError::TokenRevoked` variant for revoked-token responses.
+- Crate tests for the JWT module (session + email-verification round-trips,
+  audience cross-contamination, tampered/garbage tokens).
+- Crate tests for `parse_token` (valid extraction, whitespace trimming,
+  empty bearer, missing prefix, non-ASCII rejection).
 
 ### Changed
 
+- **BREAKING:** JWT signing algorithm changed from HMAC-HS256 to RS256
+  (RSA + SHA-256). Existing HS256 tokens are invalidated; users must re-login.
+- **BREAKING:** JWT claims redesigned from `UserClaims { user, exp }` to
+  `SessionClaims { sub, iss, aud, iat, exp, role, username, gen }`. Existing
+  tokens without the new claims fail deserialization.
+- **BREAKING:** Configuration keys changed — `auth.user_claim_token_pepper` /
+  `auth.session_signing_key` / `auth.email_verification_signing_key` replaced
+  by `auth.private_key_path` and `auth.public_key_path` (or inline PEM).
+  Deployers must generate an RSA key pair.
 - **BREAKING:** Replace `ServiceError` (41 variants) and `ServiceResult` with
   domain-scoped error enums: `AuthError`, `UserError`, `TorrentError`,
   `CategoryTagError`, and a thin `ApiError` wrapper (ADR-T-006).
+- `Authentication::get_user_id_from_bearer_token` now takes `BearerToken`
+  directly instead of `Option<BearerToken>`.
+- `ExtractLoggedInUser` and `ExtractOptionalLoggedInUser` use `BearerToken`
+  directly instead of the old `Extract` wrapper.
+- `parse_token` returns `Result` instead of panicking on malformed headers.
+- JWT `exp` validation relies solely on the `jsonwebtoken` library; redundant
+  manual expiration check removed.
+- Token signing uses `Result` propagation instead of `.unwrap()` / `.expect()`.
+- `UserClaims` is now a type alias for `SessionClaims` (backward-compatible).
+- `VerifyClaims` moved from `mailer` into the `jwt` module (re-exported for
+  backward compatibility).
 - Service functions now return domain-specific `Result<T, DomainError>` instead
   of `Result<T, ServiceError>`.
 - Each domain error co-locates its HTTP status-code mapping via a
@@ -28,6 +83,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Removed
 
+- `bearer_token::Extract` wrapper struct (replaced by `BearerToken` directly).
+- `get_optional_logged_in_user` free function (logic moved into extractors).
+- `get_claims_from_bearer_token` private method on `Authentication` (inlined).
+- `ClaimTokenPepper` / `JwtSigningSecret` / `user_claim_token_pepper` config
+  keys (replaced by RSA key pair configuration).
 - `ServiceError` enum and `ServiceResult` type alias from `src/errors.rs`.
 - `http_status_code_for_service_error` and `map_database_error_to_service_error`
   helper functions.

Cargo.lock

@@ -80,6 +80,7 @@ pbkdf2 = { version = "0", features = ["simple"] }
 pin-project-lite = "0"
 rand = "0.10"
 regex = "1"
+rsa = { version = "0.9", default-features = false, features = ["std", "pem"] }
 reqwest = { version = "0", features = ["json", "multipart", "query"] }
 serde = { version = "1", features = ["derive", "rc"] }
 serde_bencode = "0"
@@ -88,6 +89,7 @@ serde_derive = "1"
 serde_json = "1"
 serde_with = "3"
 sha-1 = "0"
+sha2 = "0"
 sqlx = { version = "0", features = ["migrate", "mysql", "runtime-tokio-native-tls", "sqlite", "time"] }
 tera = { version = "1", default-features = false }
 text-colorizer = "1"
@@ -102,6 +104,10 @@ url = { version = "2", features = ["serde"] }
 urlencoding = "2"
 uuid = { version = "1", features = ["v4"] }
 
+[[bin]]
+name = "torrust-generate-auth-keypair"
+path = "src/bin/generate_auth_keypair.rs"
+
 [dev-dependencies]
 tempfile = "3"
 which = "8"

Cargo.toml

@@ -71,7 +71,9 @@ COPY --from=build_debug \
 RUN cargo nextest run --workspace-remap /test/src/ --extract-to /test/src/ --no-run --archive-file /test/torrust-index-debug.tar.zst
 RUN cargo nextest run --workspace-remap /test/src/ --target-dir-remap /test/src/target/ --cargo-metadata /test/src/target/nextest/cargo-metadata.json --binaries-metadata /test/src/target/nextest/binaries-metadata.json
 
-RUN mkdir -p /app/bin/; cp -l /test/src/target/debug/torrust-index /app/bin/torrust-index
+RUN mkdir -p /app/bin/; \
+  cp -l /test/src/target/debug/torrust-index /app/bin/torrust-index; \
+  cp -l /test/src/target/debug/torrust-generate-auth-keypair /app/bin/torrust-generate-auth-keypair
 # RUN mkdir /app/lib/; cp -l $(realpath $(ldd /app/bin/torrust-index | grep "libz\.so\.1" | awk '{print $3}')) /app/lib/libz.so.1
 RUN chown -R root:root /app; chmod -R u=rw,go=r,a+X /app; chmod -R a+x /app/bin
 
@@ -87,7 +89,8 @@ RUN cargo nextest run --workspace-remap /test/src/ --target-dir-remap /test/src/
 
 RUN mkdir -p /app/bin/; \
   cp -l /test/src/target/release/torrust-index /app/bin/torrust-index; \
-  cp -l /test/src/target/release/health_check /app/bin/health_check;
+  cp -l /test/src/target/release/health_check /app/bin/health_check; \
+  cp -l /test/src/target/release/torrust-generate-auth-keypair /app/bin/torrust-generate-auth-keypair
 # RUN mkdir -p /app/lib/; cp -l $(realpath $(ldd /app/bin/torrust-index | grep "libz\.so\.1" | awk '{print $3}')) /app/lib/libz.so.1
 RUN chown -R root:root /app; chmod -R u=rw,go=r,a+X /app; chmod -R a+x /app/bin
 

Containerfile

@@ -93,19 +93,34 @@ _Optionally, you may choose to supply the entire configuration as an environment
 TORRUST_INDEX_CONFIG_TOML=$(cat "./storage/index/etc/index.toml") cargo run
 ```
 
-_For deployment, you __should__ override:
-
-- The `tracker_api_token` and the `index_auth_secret_key` by using environmental variables:_
+_For deployment, you __should__ override the `tracker_api_token`:_
 
 ```sh
-# Please use the secret that you generated for the torrust-tracker configuration.
-# Override secret in configuration using an environmental variable
+# Override secrets in configuration using environmental variables
 TORRUST_INDEX_CONFIG_TOML=$(cat "./storage/index/etc/index.toml") \
   TORRUST_INDEX_CONFIG_OVERRIDE_TRACKER__TOKEN=$(cat "./storage/tracker/lib/tracker_api_admin_token.secret") \
-  TORRUST_INDEX_CONFIG_OVERRIDE_AUTH__SECRET_KEY="MaxVerstappenWC2021" \
   cargo run
 ```
 
+_By default, an ephemeral RSA key pair is auto-generated in memory for JWT
+signing. Sessions will not survive server restarts. For **persistent sessions**,
+generate your own RSA key pair and configure the paths:_
+
+```sh
+# Generate an RSA key pair for JWT signing:
+openssl genrsa -out private.pem 2048
+openssl rsa -in private.pem -pubout -out public.pem
+
+# Supply key paths via environment variables:
+TORRUST_INDEX_CONFIG_TOML=$(cat "./storage/index/etc/index.toml") \
+  TORRUST_INDEX_CONFIG_OVERRIDE_AUTH__PRIVATE_KEY_PATH="/path/to/private.pem" \
+  TORRUST_INDEX_CONFIG_OVERRIDE_AUTH__PUBLIC_KEY_PATH="/path/to/public.pem" \
+  cargo run
+```
+
+> **Container deployments** auto-generate persistent keys on first boot — no
+> manual setup is required. See the [container guide][containers.md] for details.
+
 > Please view our [crate documentation][docs] for more detailed instructions.
 
 ### Services
@@ -127,6 +142,7 @@ The following services are provided by the default configuration:
 - [ADR-T-004: Remove `located-error` Package](adr/004-remove-located-error.md) — Replace the `torrust-index-located-error` wrapper with `tracing` for error context.
 - [ADR-T-005: Migrate to Rust Edition 2024](adr/005-edition-2024.md) — Migrate the entire workspace to `edition = "2024"` and raise the MSRV to 1.85.
 - [ADR-T-006: Refactor the Error System](adr/006-error-system-refactor.md) — Replace the 41-variant `ServiceError` god enum with domain-scoped error enums (`AuthError`, `UserError`, `TorrentError`, `CategoryTagError`) and a thin `ApiError` wrapper.
+- [ADR-T-007: Refactor the JWT System](adr/007-jwt-system-refactor.md) — Centralise JWT handling into `src/jwt.rs`, redesign claims to RFC 7519, move to RS256 asymmetric signing, and consolidate session validation into a single code path.
 
 ## Contributing