Merge torrust#869: feat!: Global command-line output contract (ADR-T-010)

5a6dab0 feat(cli): surface panic payloads in debug diagnostics (Peer Cat)
503e90e fix(container): decouple debug runtime binary from nextest archive (Peer Cat)
c68a626 docs(cli): consolidate ADR-T-010 implementation record (Peer Cat)
c430995 test(cli): add ADR-T-010 regression guards (Peer Cat)
d07fee7 docs(cli): refresh ADR-T-010 command examples (Peer Cat)
4089926 feat(cli)!: migrate container entry script to ADR-T-010 (Peer Cat)
0dd220b feat(cli)!: complete ADR-T-010 shared-library cleanup (Peer Cat)
685ad5a feat(cli)!: migrate maintenance commands to ADR-T-010 (Peer Cat)
b5332e1 feat(cli)!: migrate torrent helpers to ADR-T-010 (Peer Cat)
e8adbff feat(cli)!: implement ADR-T-010 root JSON logging boundary (Peer Cat)
c9faf16 feat(cli)!: implement ADR-T-010 helper control plane (Cameron Garnham)
4bbb32c feat(cli)!: implement ADR-T-010 stage-one output contract (Cameron Garnham)
ceba40e docs(adr-010): establish global CLI output contract (Cameron Garnham)

Pull request description:

  ## Summary

  Lands ADR-T-010 end to end: every shipped, documented, or operator-facing first-party Torrust Index command-line entrypoint now follows one JSON-only stream contract.

  Stdout is reserved for machine-readable result data. Stderr carries machine-readable diagnostics and control records as JSON/NDJSON. Commands that emit stdout result data refuse direct terminal stdout, so automation cannot accidentally mix data streams with human-facing diagnostics.

  > ⚠ **Breaking for operators and scripts.** Any automation that scraped previous plain-text output from helper binaries, root maintenance commands, `parse_torrent`, server logs, or container startup must switch to exit codes plus JSON/NDJSON parsing.

  ## Why

  ADR-T-009 introduced strict stream rules for container helper binaries, but the underlying problem was repository-wide: every CLI entrypoint had its own stdout/stderr behavior.

  That made shell integration brittle:

  - helper JSON could be corrupted by plain diagnostics;
  - `clap` help/errors and Rust panic output could still leak raw text;
  - server and maintenance logs were not consistently machine-readable;
  - `parse_torrent` and maintenance tools mixed operator text with command results;
  - container startup used plain text and `DEBUG=1` shell tracing instead of structured records;
  - command-reachable libraries could still print or exit outside the command boundary.

  This PR promotes the helper contract into a global CLI contract and migrates the existing command surface to match it.

  ## Highlights

  - **ADR-T-010 becomes canonical.** [adr/010-global-command-line-output-contract.md](adr/010-global-command-line-output-contract.md) now records the completed repository-wide contract, command classifications, shared schema, implementation summary, redaction policy, and guards.
  - **Shared CLI infrastructure.** `torrust-index-cli-common` now owns JSON `clap` help/version/usage handling, stderr control-plane records, baseline exit classes, JSON panic diagnostics, JSON tracing setup, TTY refusal, stdout JSON emission, command runners, locked stderr writes, and redaction helpers.
  - **Helper binaries migrated.** `torrust-index-auth-keypair`, `torrust-index-config-probe`, and `torrust-index-health-check` now share the same JSON control plane. Successful stdout payloads are single JSON objects with a top-level `schema` field; help, version, argv errors, TTY refusal, panic diagnostics, and tracing go to stderr as JSON.
  - **Server logging boundary migrated.** `torrust-index` now emits JSON tracing records on stderr. The configured logging threshold remains the default filter, while non-empty `RUST_LOG` takes precedence. Binary entrypoints return explicit `ExitCode` values instead of relying on Rust default termination.
  - **Torrent helper commands migrated.** `parse_torrent` is now a stdout-result command emitting `schema`, `torrent`, `original_v1_info_hash`, and `input_byte_length`. `create_test_torrent` is a no-stdout side-effect command with JSON stderr diagnostics.
  - **Maintenance commands migrated.** `import_tracker_statistics`, `seeder`, and `upgrade` now keep stdout empty and emit help, usage errors, status, diagnostics, tracing, and panic records as JSON/NDJSON on stderr.
  - **Upgrade path hardened.** The v1-to-v2 upgrader now propagates typed errors for setup, migrations, transfer validation, torrent decoding, missing fields, and timestamp conversion instead of relying on plain output, color formatting, unwraps, or panic-driven failures.
  - **Command-reachable libraries cleaned up.** Shutdown notices use structured tracing, mail template failures are returned to callers, terminal color formatting is removed from command paths, and parsing helpers leave reporting to their command callers.
  - **Container entry script migrated.** Startup validation failures, status notices, expected utility failures, `jq` failures, unexpected shell exits, and `DEBUG=1` phase records now emit JSON/NDJSON on stderr. Helper stdout stays internal to command substitutions.
  - **Regression guards added.** Workspace Clippy now denies accidental raw `print_stdout`, `print_stderr`, and direct `exit` usage outside approved boundaries, and [tests/cli_contract.rs](tests/cli_contract.rs) keeps in-scope binaries returning `ExitCode`.

  ## Breaking Changes

  | Surface | Before | After |
  |---|---|---|
  | CLI stream contract | Per-command stdout/stderr behavior | ADR-T-010 JSON-only contract across first-party entrypoints |
  | Server logs | Human-formatted tracing output | JSON records on stderr |
  | Helper stdout schemas | Some payloads lacked explicit schema versioning | Successful helper payloads include top-level `schema` |
  | Helper help/errors/panics | Plain `clap` or Rust panic text could reach stderr | JSON control-plane records on stderr |
  | `parse_torrent` | Legacy plain-text CLI behavior | Single JSON stdout result; refuses direct terminal stdout |
  | `create_test_torrent` | Plain status/diagnostic output | Empty stdout; JSON/NDJSON stderr |
  | `import_tracker_statistics`, `seeder`, `upgrade` | Plain status/errors and panic/unwrap paths | Empty stdout; JSON/NDJSON stderr; explicit exit codes |
  | Container entry script | Plain startup text and `DEBUG=1` / `set -x` tracing | JSON/NDJSON stderr records and explicit debug phase records |
  | Command libraries | Some direct printing/exiting/color formatting | Structured diagnostics returned or logged through the command boundary |

  ## Operator Migration

  Stdout-producing commands should be piped or redirected before inspection:

  ```sh
  torrust-index-auth-keypair | jq .
  torrust-index-config-probe | jq .
  torrust-index-health-check http://127.0.0.1:3001/health_check | jq .
  cargo run --quiet --bin parse_torrent -- "$fixture" | jq .'
  ```

  ## Credits
  Thank-you for @da2ce7 for writing the ADR 10, and starting the work.

ACKs for top commit:
  peer-cat:
    ACK 5a6dab0
  da2ce7:
    ACK 5a6dab0

Tree-SHA512: 3253c4d6cc995fc468e730925bc7a7f59087356f30d2a2b109f69d0cc6c216c0cf8e18941549b2a78814e426963d7600a917d905e4ae3c72ed2d4445b068dd64

Author: da2ce7

CHANGELOG.md

@@ -7,14 +7,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
-**Highlights:** container infrastructure refactor (ADR-T-009),
+**Highlights:** global command-line output contract (ADR-T-010),
+container infrastructure refactor (ADR-T-009),
 native role-based authorization replacing Casbin (ADR-T-008),
 RSA-signed JWTs with revocation support (ADR-T-007), domain-scoped
 error system (ADR-T-006), MSRV raised to 1.88.
 
 ### Breaking changes
 
 - MSRV raised from 1.85 to 1.88.
+- First-party command-line entrypoints are now governed by ADR-T-010's
+  JSON-only output contract. Stdout is reserved for machine-readable result
+  data, stderr is reserved for machine-readable diagnostics/control records, and
+  stdout-producing commands refuse direct terminal stdout. `parse_torrent` now
+  emits JSON result data on stdout. `create_test_torrent`,
+  `import_tracker_statistics`, `seeder`, and `upgrade` now keep stdout empty
+  while reporting status and diagnostics as JSON on stderr. The container entry
+  script also reports validation failures, status records, utility failures, and
+  debug phase records as JSON/NDJSON on stderr instead of plain text or shell
+  trace output. Scripts that scraped previous plain-text command or startup
+  output must switch to exit codes and JSON/NDJSON stderr parsing.
+- The `torrust-index` server's application logs now use JSON records on stderr
+  instead of the previous human-formatted tracing output. Log consumers should
+  parse stderr as NDJSON or pipe it through a JSON viewer.
+- `torrust-index-auth-keypair` and `torrust-index-health-check` stdout JSON now
+  includes a top-level `schema` field. Scripts that expected an exact object
+  shape must tolerate or consume the schema field.
 - `database.connect_url` and `tracker.token` are now mandatory
   schema fields with no defaults. Supply them via env-var override
   (`TORRUST_INDEX_CONFIG_OVERRIDE_DATABASE__CONNECT_URL`,
@@ -57,6 +75,56 @@ error system (ADR-T-006), MSRV raised to 1.88.
   domain-scoped enums: `AuthError`, `UserError`, `TorrentError`,
   `CategoryTagError`, with a thin `ApiError` wrapper (ADR-T-006).
 
+### ADR-T-010 — Global command-line output contract
+
+#### Added
+
+- ADR-T-010 establishes a repository-wide JSON-only output contract for
+  first-party command-line entrypoints. Stdout is reserved for result data;
+  stderr carries diagnostics and control records; commands that emit stdout
+  result data refuse direct terminal stdout.
+- `torrust-index-cli-common` provides the shared implementation for that
+  contract: JSON `clap` help/version/usage handling, JSON panic diagnostics,
+  JSON stderr tracing, TTY refusal, stdout JSON emission, command runners,
+  baseline exit-code classes, control-plane record types, and redaction
+  helpers.
+- Regression coverage now protects the contract with CLI behavior tests,
+  binary-boundary checks, and workspace lint rules denying accidental raw
+  stream output or direct process exits outside the shared CLI boundary.
+
+#### Changed
+
+- Helper binaries (`torrust-index-auth-keypair`,
+  `torrust-index-config-probe`, and `torrust-index-health-check`) share the
+  JSON CLI boundary. Their successful stdout payloads remain single JSON
+  objects, now explicitly versioned with a top-level `schema` field, while
+  help, version, argv errors, TTY refusal, panic diagnostics, and tracing are
+  emitted as JSON records on stderr.
+- The `torrust-index` server and root Rust binaries return explicit
+  `ExitCode` values at their `main` boundaries and install the shared JSON
+  panic hook. Central application logging uses JSON tracing on stderr, with a
+  non-empty `RUST_LOG` taking precedence over the configured default filter.
+- `parse_torrent` is a stdout-result command. It emits one JSON object with
+  `schema`, `torrent`, `original_v1_info_hash`, and `input_byte_length`, leaves
+  stdout empty on failure, and refuses direct terminal stdout with a JSON
+  diagnostic record.
+- `create_test_torrent`, `import_tracker_statistics`, `seeder`, and `upgrade`
+  are no-stdout side-effect commands. They keep stdout empty, report status and
+  diagnostics as JSON/NDJSON on stderr, and propagate command failures instead
+  of printing plain text or relying on panic output.
+- Command-reachable shared libraries use the command diagnostic path instead of
+  raw stream output. Shutdown notices are structured tracing records, mail
+  template failures are returned to callers, terminal color formatting is
+  removed from command paths, and parsing helpers leave reporting decisions to
+  their command callers.
+- The container entry script follows the JSON stderr contract during startup:
+  it captures helper stdout internally, keeps its own stdout empty before
+  `su-exec`, checks for `jq` before JSON-dependent helpers run, emits explicit
+  `DEBUG=1` phase records instead of `set -x`, and wraps controlled utility
+  failures with captured stderr fields.
+- Operator documentation and command examples describe the completed contract
+  across the README, container guide, upgrade notes, and command module docs.
+
 ### ADR-T-009 — Container infrastructure refactor
 
 #### Added
@@ -67,7 +135,7 @@ error system (ADR-T-006), MSRV raised to 1.88.
   configuration system. Leaf crate — no `tokio`, `reqwest`, `sqlx`,
   `hyper`, `rustls`, `native-tls`, or `openssl` in its dep closure.
 - Helper-binary crates split into leaves with no HTTP/TLS in their
-  dep closure: `torrust-index-cli-common` (shared P9 scaffolding —
+  dep closure: `torrust-index-cli-common` (shared ADR-T-010 scaffolding —
   `refuse_if_stdout_is_tty`, `init_json_tracing`, `emit`, `BaseArgs`),
   `torrust-index-health-check` (stdlib-only, Happy Eyeballs IPv6/IPv4
   fallback), `torrust-index-auth-keypair` (RSA-2048 key generator),
@@ -126,7 +194,7 @@ error system (ADR-T-006), MSRV raised to 1.88.
 - `EXPOSE ${IMPORTER_API_PORT}/tcp` in Containerfile; port 3002
   mapped in compose.
 - `restart: unless-stopped` on index and tracker compose services.
-- `DEBUG=1` env-var gate for entry-script shell tracing (`set -x`).
+- `DEBUG=1` env-var gate for entry-script JSON phase diagnostics.
 - `#[doc(hidden)] pub mod test_helpers` in `torrust-index-config`
   exposing `PLACEHOLDER_TOML` and `placeholder_settings()` — single
   source of truth for the ~40 tests across both crates that
@@ -391,8 +459,8 @@ error system (ADR-T-006), MSRV raised to 1.88.
 
 - Dev-only ports (MySQL 3306, tracker 6969/7070/1212, mailcatcher
   1025/1080) no longer bind to `0.0.0.0`; bound to `127.0.0.1`.
-- Entry script `set -x` gated behind `DEBUG=1` to avoid leaking
-  env vars into logs.
+- Entry script debug mode now emits structured JSON phase records instead of
+  enabling `set -x`, avoiding shell-trace leakage of env vars into logs.
 - Compose credentials annotated as DEV-ONLY with TODO for Docker
   secrets migration (ADR-T-009 §S1).
 

Cargo.lock

@@ -52,6 +52,7 @@ opt-level = 3
 opt-level = 3
 
 [dependencies]
+torrust-index-cli-common = { version = "4.0.0-develop", path = "packages/index-cli-common" }
 torrust-index-config = { version = "4.0.0-develop", path = "packages/index-config" }
 torrust-index-render-text-as-image = { version = "0.1.0", path = "packages/render-text-as-image" }
 
@@ -100,14 +101,12 @@ sha-1 = "0"
 sha2 = "0"
 sqlx = { version = "0", features = ["migrate", "mysql", "runtime-tokio-native-tls", "sqlite", "time"] }
 tera = { version = "1", default-features = false }
-text-colorizer = "1"
 thiserror = "2"
 tokio = { version = "1", features = ["fs", "io-util", "macros", "net", "rt-multi-thread", "signal", "sync", "time"] }
 toml = "1"
 tower = { version = "0", features = ["timeout"] }
 tower-http = { version = "0", features = ["compression-full", "cors", "propagate-header", "request-id", "trace"] }
 tracing = "0"
-tracing-subscriber = { version = "0", features = ["json"] }
 url = { version = "2", features = ["serde"] }
 urlencoding = "2"
 uuid = { version = "1", features = ["v4"] }
@@ -139,9 +138,12 @@ warnings = { level = "deny", priority = -1 }
 all = { level = "deny", priority = -1 }
 complexity = { level = "deny", priority = -1 }
 correctness = { level = "deny", priority = -1 }
+exit = { level = "deny", priority = 0 }
 nursery = { level = "warn", priority = -1 }
 pedantic = { level = "deny", priority = -1 }
 perf = { level = "deny", priority = -1 }
+print_stderr = { level = "deny", priority = 0 }
+print_stdout = { level = "deny", priority = 0 }
 style = { level = "deny", priority = -1 }
 suspicious = { level = "deny", priority = -1 }
 

Cargo.toml

@@ -12,7 +12,7 @@ RUN cargo binstall --no-confirm --locked cargo-chef cargo-nextest
 FROM rust:slim-trixie AS tester
 WORKDIR /tmp
 
-RUN apt-get update; apt-get install -y curl sqlite3; apt-get autoclean
+RUN apt-get update; apt-get install -y curl jq sqlite3; apt-get autoclean
 RUN curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/v1.18.1/install-from-binstall-release.sh | bash
 RUN cargo binstall --no-confirm --locked cargo-nextest imdl
 
@@ -80,6 +80,12 @@ RUN cargo chef prepare --recipe-path /build/recipe.json
 ## Cook (debug)
 FROM chef AS dependencies_debug
 WORKDIR /build/src
+# The debug archive stage holds both the Cargo target directory and the
+# nextest archive in one layer. Full debuginfo makes that layer exceed
+# common builder storage limits while not changing the test surface.
+ENV CARGO_INCREMENTAL=0 \
+  CARGO_PROFILE_DEV_DEBUG=0 \
+  CARGO_PROFILE_TEST_DEBUG=0
 COPY --from=recipe /build/recipe.json /build/recipe.json
 RUN cargo chef cook --workspace --all-targets --all-features --recipe-path /build/recipe.json
 RUN cargo nextest archive --workspace --all-targets --all-features --archive-file /build/temp.tar.zst ; rm -f /build/temp.tar.zst
@@ -98,6 +104,13 @@ WORKDIR /build/src
 COPY . /build/src
 RUN cargo nextest archive --workspace --all-targets --all-features --archive-file /build/torrust-index-debug.tar.zst
 
+## Build Runtime Binary (debug)
+FROM dependencies_debug AS build_debug_runtime
+WORKDIR /build/src
+COPY . /build/src
+RUN unset CARGO_PROFILE_DEV_DEBUG CARGO_PROFILE_TEST_DEBUG; \
+  cargo build --package torrust-index --all-features --bin torrust-index
+
 ## Build Archive (release)
 FROM dependencies AS build
 WORKDIR /build/src
@@ -314,6 +327,8 @@ ENV TORRUST_INDEX_CONFIG_TOML_PATH=/etc/torrust/index/index.toml \
 EXPOSE 3001/tcp 3002/tcp
 VOLUME ["/var/lib/torrust/index","/var/log/torrust/index","/etc/torrust/index"]
 COPY --from=test_debug /app/ /usr/
+COPY --from=build_debug_runtime --chmod=0755 --chown=0:0 \
+  /build/src/target/debug/torrust-index /usr/bin/torrust-index
 # jq binary for entry-script JSON consumption (§2.2 step 4).
 # Root-only (0500) — same posture as busybox and su-exec.
 # The two shared libraries (libjq, libonig) are required at

Containerfile

@@ -162,6 +162,90 @@ The following services are provided by the default configuration:
 - API
   - `http://127.0.0.1:3001/`.
 
+### Command-Line Output
+
+First-party Torrust Index command-line entrypoints are governed by
+[ADR-T-010](./adr/010-global-command-line-output-contract.md): stdout is
+reserved for machine-readable result data, stderr is reserved for
+machine-readable diagnostics/control records, and commands that emit stdout
+result data refuse to write it directly to a terminal.
+
+The `torrust-index` server binary is a no-stdout command. Application tracing is
+emitted as JSON records on stderr; the configured `[logging].threshold` selects
+the default filter, and a non-empty `RUST_LOG` environment variable overrides
+that default. Panics that cross the binary boundary are reported as ADR-T-010
+JSON control-plane records on stderr.
+
+Command-reachable server libraries use the same diagnostic path. Shutdown
+grace-period notices are structured tracing records, and mail-template
+initialization or rendering failures are propagated to callers for JSON
+diagnostic reporting instead of being printed or exiting from the mailer
+library.
+
+The shared helper infrastructure now wraps `clap` help, version, and usage
+errors as JSON control-plane records on stderr, installs a JSON-only panic hook,
+and uses JSON tracing on stderr. The container helper binaries emit exactly one
+JSON object on stdout when successful, include a top-level `schema` field, and
+should be inspected through a pipe or redirect:
+
+```sh
+torrust-index-auth-keypair | jq .
+torrust-index-config-probe | jq .
+torrust-index-health-check http://127.0.0.1:3001/health_check | jq .
+```
+
+For helper diagnostics, a non-empty `RUST_LOG` environment variable takes
+precedence over `--debug`; otherwise `--debug` raises the default diagnostic
+filter to debug.
+
+The container entry script is also a no-stdout orchestration command. It captures
+helper stdout internally, keeps its own stdout empty before `su-exec`, and emits
+startup validation failures, status records, utility failures, and `DEBUG=1`
+phase diagnostics as JSON records on stderr. Use `docker logs ... 2>&1` or your
+runtime's stderr capture and parse those lines as NDJSON when automation needs
+startup diagnostics.
+
+Two root diagnostic commands have also been migrated. `parse_torrent` is a
+stdout-result command: it emits one JSON object containing `schema`, `torrent`,
+`original_v1_info_hash`, and `input_byte_length`, and it refuses direct terminal
+stdout. Pipe or redirect it before inspection:
+
+```sh
+fixture=./tests/fixtures/torrents/6c690018c5786dbbb00161f62b0712d69296df97_with_custom_info_dict_key.torrent
+cargo run --quiet --bin parse_torrent -- "$fixture" | jq .
+```
+
+`create_test_torrent` is a no-stdout side-effect command. It writes the torrent
+file into an existing destination directory, keeps stdout empty, and emits JSON
+status or diagnostic records on stderr:
+
+```sh
+mkdir -p ./output/test/torrents
+cargo run --quiet --bin create_test_torrent -- ./output/test/torrents 2>create-test-torrent.ndjson
+jq . create-test-torrent.ndjson
+```
+
+The root maintenance binaries `import_tracker_statistics`, `seeder`, and
+`upgrade` are no-stdout side-effect commands. They keep stdout empty, use the
+shared JSON `clap` wrapper for help, version, and argv errors, and emit status
+or diagnostic records as JSON/NDJSON on stderr. Automation should branch on the
+process exit code and parse stderr as JSON when it needs diagnostics:
+
+```sh
+cargo run --quiet --bin import_tracker_statistics -- 2>import-tracker-statistics.ndjson
+
+cargo run --quiet --bin seeder -- \
+  --api-base-url "http://localhost:3001" \
+  --number-of-torrents 10 \
+  --user admin \
+  --password "$TORRUST_INDEX_ADMIN_PASSWORD" \
+  --interval 0 \
+  2>seeder.ndjson
+
+cargo run --quiet --bin upgrade -- ./data.db ./data_v2.db ./uploads 2>upgrade.ndjson
+jq . upgrade.ndjson
+```
+
 ## Documentation
 
 - [API (Version 1)][api]
@@ -177,6 +261,7 @@ The following services are provided by the default configuration:
 - [ADR-T-007: Refactor the JWT System](adr/007-jwt-system-refactor.md) — Centralise JWT handling into `src/jwt.rs`, redesign claims to RFC 7519, move to RS256 asymmetric signing, and consolidate session validation into a single code path.
 - [ADR-T-008: Refactor the Roles and Permissions System](adr/008-roles-and-permissions-refactor.md) — Replace Casbin with a native Rust permission system (`PermissionMatrix` + `RequirePermission<A>` Axum extractors), migrate from `administrator: bool` to a `role` column, and add a `/me/permissions` discovery endpoint.
 - [ADR-T-009: Container Infrastructure Refactor](adr/009-container-infrastructure-refactor.md) — Split the runtime image into `release` (distroless, root-only toolset) and `debug` bases; extract three helper binaries (`torrust-index-health-check`, `torrust-index-auth-keypair`, `torrust-index-config-probe`) into their own workspace crates with no HTTP/TLS/async-runtime deps; strip credentials from shipped TOMLs and make `database.connect_url` / `tracker.token` mandatory schema fields; split Compose into a production-shaped `compose.yaml` baseline plus an auto-loaded `compose.override.yaml` dev sandbox; and add an internal audit record for vendored `su-exec`.
+- [ADR-T-010: Global Command-Line Output Contract](adr/010-global-command-line-output-contract.md) — Apply the JSON-only stdout/stderr contract across first-party command-line entrypoints: stdout is result JSON, stderr is diagnostic JSON/NDJSON, and commands with stdout result data refuse direct TTY output.
 
 ## Contributing