diff --git a/docs/deployments/hetzner-demo-tracker/README.md b/docs/deployments/hetzner-demo-tracker/README.md index 83424dd1..7485121e 100644 --- a/docs/deployments/hetzner-demo-tracker/README.md +++ b/docs/deployments/hetzner-demo-tracker/README.md @@ -42,7 +42,9 @@ Deploy a public Torrust Tracker demo instance to Hetzner Cloud and document ever 8. [Observations](observations.md) — cross-cutting insights and learnings about the deployer 9. [Maintenance](maintenance/README.md) — post-deployment operational tasks: - [Secrets rotation](maintenance/secrets-rotation.md) — rotate all secrets after AI-assisted deployment -10. [Tracker Registry](tracker-registry.md) — submit the tracker to public registries (newTrackon) +10. [Tracker Registry](tracker-registry.md) — submit the tracker to public registries (newTrackon): + - [newTrackon Prerequisites](post-provision/newtrackon-prerequisites.md) — BEP 34 DNS TXT records and + unique-IP policy required by newTrackon (needed to list the UDP1 tracker, see [issue #407](https://github.com/torrust/torrust-tracker-deployer/issues/407)) 11. [Bugs](bugs.md) — all deployer bugs discovered during this deployment (11 bugs, 1 fixed) 12. [Improvements](improvements.md) — all improvement recommendations collected in one place (13 items) diff --git a/docs/deployments/hetzner-demo-tracker/media/hetzner-console-all-four-floating-ips.png b/docs/deployments/hetzner-demo-tracker/media/hetzner-console-all-four-floating-ips.png new file mode 100644 index 00000000..54e210ed Binary files /dev/null and b/docs/deployments/hetzner-demo-tracker/media/hetzner-console-all-four-floating-ips.png differ diff --git a/docs/deployments/hetzner-demo-tracker/media/hetzner-console-dns-config-after-udp1-changes.png b/docs/deployments/hetzner-demo-tracker/media/hetzner-console-dns-config-after-udp1-changes.png new file mode 100644 index 00000000..66a1cae4 Binary files /dev/null and b/docs/deployments/hetzner-demo-tracker/media/hetzner-console-dns-config-after-udp1-changes.png differ diff --git a/docs/deployments/hetzner-demo-tracker/media/hetzner-console-dns-config-before-udp1-changes.png b/docs/deployments/hetzner-demo-tracker/media/hetzner-console-dns-config-before-udp1-changes.png new file mode 100644 index 00000000..b9ab7cf4 Binary files /dev/null and b/docs/deployments/hetzner-demo-tracker/media/hetzner-console-dns-config-before-udp1-changes.png differ diff --git a/docs/deployments/hetzner-demo-tracker/media/hetzner-console-floating-ips-renamed-for-http1.png b/docs/deployments/hetzner-demo-tracker/media/hetzner-console-floating-ips-renamed-for-http1.png new file mode 100644 index 00000000..5ae4138f Binary files /dev/null and b/docs/deployments/hetzner-demo-tracker/media/hetzner-console-floating-ips-renamed-for-http1.png differ diff --git a/docs/deployments/hetzner-demo-tracker/media/newtrackon-home-three-trackers-listed.png b/docs/deployments/hetzner-demo-tracker/media/newtrackon-home-three-trackers-listed.png new file mode 100644 index 00000000..90f6c684 Binary files /dev/null and b/docs/deployments/hetzner-demo-tracker/media/newtrackon-home-three-trackers-listed.png differ diff --git a/docs/deployments/hetzner-demo-tracker/media/newtrackon-submitted-udp1-accepted.png b/docs/deployments/hetzner-demo-tracker/media/newtrackon-submitted-udp1-accepted.png new file mode 100644 index 00000000..bebf676e Binary files /dev/null and b/docs/deployments/hetzner-demo-tracker/media/newtrackon-submitted-udp1-accepted.png differ diff --git a/docs/deployments/hetzner-demo-tracker/media/newtrackon-submitted-udp1-rejected-udp-timeout.png b/docs/deployments/hetzner-demo-tracker/media/newtrackon-submitted-udp1-rejected-udp-timeout.png new file mode 100644 index 00000000..2e0adafd Binary files /dev/null and b/docs/deployments/hetzner-demo-tracker/media/newtrackon-submitted-udp1-rejected-udp-timeout.png differ diff --git a/docs/deployments/hetzner-demo-tracker/post-provision/README.md b/docs/deployments/hetzner-demo-tracker/post-provision/README.md index 0a1cedfc..9eb61f9b 100644 --- a/docs/deployments/hetzner-demo-tracker/post-provision/README.md +++ b/docs/deployments/hetzner-demo-tracker/post-provision/README.md @@ -13,6 +13,15 @@ on the server via SSH. | 2. Volume Setup | [volume-setup.md](volume-setup.md) | ✅ Done | | 3. Hetzner Backups | [hetzner-backups.md](hetzner-backups.md) | ✅ Done | +## Post-Deployment Steps + +Steps performed after the tracker is running and during ongoing operations: + +| Step | Guide | Status | +| --------------------------- | ---------------------------------------------------------- | ------- | +| 4. newTrackon Prerequisites | [newtrackon-prerequisites.md](newtrackon-prerequisites.md) | ✅ Done | +| 5. IPv6 UDP Tracker Issue | [ipv6-udp-tracker-issue.md](ipv6-udp-tracker-issue.md) | ✅ Done | + ## Why Before `configure`? - **DNS**: The `configure` command installs Caddy as a TLS reverse proxy. Caddy uses diff --git a/docs/deployments/hetzner-demo-tracker/post-provision/dns-setup.md b/docs/deployments/hetzner-demo-tracker/post-provision/dns-setup.md index c0aa97ce..d0391013 100644 --- a/docs/deployments/hetzner-demo-tracker/post-provision/dns-setup.md +++ b/docs/deployments/hetzner-demo-tracker/post-provision/dns-setup.md @@ -184,20 +184,20 @@ in the Cloud Console are only accessible via the **Hetzner Cloud API** — the o ### Records to Create -| Subdomain | Type | Value | -| --------- | ---- | ----------------------- | -| `http1` | A | `116.202.176.169` | -| `http1` | AAAA | `2a01:4f8:1c0c:9aae::1` | -| `http2` | A | `116.202.176.169` | -| `http2` | AAAA | `2a01:4f8:1c0c:9aae::1` | -| `api` | A | `116.202.176.169` | -| `api` | AAAA | `2a01:4f8:1c0c:9aae::1` | -| `grafana` | A | `116.202.176.169` | -| `grafana` | AAAA | `2a01:4f8:1c0c:9aae::1` | -| `udp1` | A | `116.202.176.169` | -| `udp1` | AAAA | `2a01:4f8:1c0c:9aae::1` | -| `udp2` | A | `116.202.176.169` | -| `udp2` | AAAA | `2a01:4f8:1c0c:9aae::1` | +| Subdomain | Type | Value | Notes | +| --------- | ---- | ----------------------- | ------------------------------- | +| `http1` | A | `116.202.176.169` | | +| `http1` | AAAA | `2a01:4f8:1c0c:9aae::1` | | +| `http2` | A | `116.202.176.169` | | +| `http2` | AAAA | `2a01:4f8:1c0c:9aae::1` | | +| `api` | A | `116.202.176.169` | | +| `api` | AAAA | `2a01:4f8:1c0c:9aae::1` | | +| `grafana` | A | `116.202.176.169` | | +| `grafana` | AAAA | `2a01:4f8:1c0c:9aae::1` | | +| `udp1` | A | `116.202.177.184` | Updated 2026-03-06 (issue #407) | +| `udp1` | AAAA | `2a01:4f8:1c0c:828e::1` | Updated 2026-03-06 (issue #407) | +| `udp2` | A | `116.202.176.169` | | +| `udp2` | AAAA | `2a01:4f8:1c0c:9aae::1` | | ### API Approach @@ -297,6 +297,28 @@ udp1: A=116.202.176.169 AAAA=2a01:4f8:1c0c:9aae::1 udp2: A=116.202.176.169 AAAA=2a01:4f8:1c0c:9aae::1 ``` +## Step 4: Update DNS Records for UDP1 (2026-03-06) + +As part of issue #407 (submitting the UDP1 tracker to newTrackon), the `udp1` A and AAAA records +were updated to point to the new dedicated floating IPs: + +| Subdomain | Type | Old value | New value | +| --------- | ---- | ----------------------- | ----------------------- | +| `udp1` | A | `116.202.176.169` | `116.202.177.184` | +| `udp1` | AAAA | `2a01:4f8:1c0c:9aae::1` | `2a01:4f8:1c0c:828e::1` | + +Verified with `dig` (2026-03-06): + +```text +$ dig A udp1.torrust-tracker-demo.com +short +116.202.177.184 + +$ dig AAAA udp1.torrust-tracker-demo.com +short +2a01:4f8:1c0c:828e::1 +``` + +✅ `udp1.torrust-tracker-demo.com` now resolves exclusively to the UDP1 floating IPs. + ✅ All 12 records resolve correctly globally. > DNS propagation with Hetzner's nameservers (`helium.ns.hetzner.de`, `hydrogen.ns.hetzner.com`, @@ -305,8 +327,11 @@ udp2: A=116.202.176.169 AAAA=2a01:4f8:1c0c:9aae::1 ## Outcome -✅ All subdomains resolve to `116.202.176.169` (A) and `2a01:4f8:1c0c:9aae::1` (AAAA). DNS -setup is complete. The next step is [volume-setup.md](volume-setup.md). +✅ All subdomains resolve correctly. After the 2026-03-06 update, `udp1.torrust-tracker-demo.com` +resolves to the dedicated `udp1` floating IPs (`116.202.177.184` / `2a01:4f8:1c0c:828e::1`) +while all other subdomains continue to resolve to `116.202.176.169` (A) and +`2a01:4f8:1c0c:9aae::1` (AAAA). DNS setup is complete. +The next step is [volume-setup.md](volume-setup.md). ## Problems diff --git a/docs/deployments/hetzner-demo-tracker/post-provision/ipv6-udp-tracker-issue.md b/docs/deployments/hetzner-demo-tracker/post-provision/ipv6-udp-tracker-issue.md new file mode 100644 index 00000000..51552639 --- /dev/null +++ b/docs/deployments/hetzner-demo-tracker/post-provision/ipv6-udp-tracker-issue.md @@ -0,0 +1,573 @@ +# IPv6 UDP Tracker — Known Issue + +> **Status**: ✅ Resolved (2026-03-06) — two root causes identified and fixed: +> +> 1. `ufw` was blocking IPv6 UDP 6969 (primary blocker — packets never reached the container) +> 2. Asymmetric routing needed policy routing tables so replies leave via the correct floating IP +> +> ✅ All fixes are now persistent: `ufw` stores rules in `/etc/ufw/`; policy routing rules are +> persisted via netplan (`/etc/netplan/60-floating-ip.yaml`) + +## Context + +During issue #407 (submitting the UDP1 tracker to newTrackon), the tracker was rejected with a +"UDP timeout" error. The newTrackon probe used the AAAA record +(`2a01:4f8:1c0c:828e::1`) to reach the tracker via IPv6. IPv4 probes (tested locally) work fine. + +This document records the investigation, likely root cause, and the fix required. + +## Symptom + +- `udp://udp1.torrust-tracker-demo.com:6969/announce` submitted to newTrackon +- newTrackon probed via IPv6: `2a01:4f8:1c0c:828e::1` +- Result: ❌ **Rejected — UDP timeout** +- Local test via IPv4 (`116.202.177.184`): ✅ Works + +## What Was Ruled Out + +| Hypothesis | Evidence | Verdict | +| --------------------------------- | ------------------------------------------------------------------------ | -------------------------- | +| Docker IPv6 disabled | `ss -ulnp` shows `[::]:6969` — container binds to both IPv4 and IPv6 | ❌ Ruled out | +| Wrong IP in DNS | `dig AAAA` returns `2a01:4f8:1c0c:828e::1` ✅ | ❌ Ruled out | +| Floating IP not on interface | `ip addr show eth0` shows all four IPs with `valid_lft forever` | ❌ Ruled out | +| BEP 34 TXT record missing | `dig TXT udp1.torrust-tracker-demo.com` returns correct value | ❌ Ruled out | +| Caddy proxy intercepting UDP | UDP tracker bypasses reverse proxy entirely | ❌ Ruled out | +| Asymmetric routing (reply source) | Without policy routing, replies left via primary IP, not the floating IP | ✅ Secondary issue — fixed | + +## Investigation — 2026-03-06 + +### Check 1 — Verify Docker IPv6 Port Bindings + +First hypothesis was Docker IPv6 disabled. Ran on the server: + +```bash +sudo ss -ulnp | grep 6969 +``` + +Output: + +```text +UNCONN 0 0 0.0.0.0:6969 0.0.0.0:* users:(("docker-proxy",pid=1533796,fd=7)) +UNCONN 0 0 [::]:6969 [::]:* users:(("docker-proxy",pid=1533806,fd=7)) +``` + +✅ The tracker binds to **both** `0.0.0.0:6969` (IPv4) and `[::]:6969` (IPv6). Docker IPv6 is +enabled — packets arriving on the IPv6 floating IP do reach the container. + +The issue is elsewhere: the packet arrives and the container processes it, but the **reply** goes +out via the wrong source IP. + +### Check 2 — Verify IPv6 Policy Routing Rules + +Policy-based routing forces replies to leave via the same floating IP the probe arrived on. +Checked whether the IPv6 rule was already in place: + +```bash +ip -6 rule list +``` + +Output: + +```text +0: from all lookup local +32765: from 2a01:4f8:1c0c:828e::1 lookup 200 +32766: from all lookup main +``` + +A rule was already present from a previous attempt. Verified the corresponding route table: + +```bash +ip -6 route show table 200 +``` + +Output: + +```text +default via fe80::1 dev eth0 +``` + +✅ IPv6 replies from `2a01:4f8:1c0c:828e::1` route via `fe80::1` (Hetzner's IPv6 gateway on +`eth0`). + +### Check 3 — Add IPv4 Policy Routing Rules + +The IPv4 floating IP (`116.202.177.184`) also needed symmetric routing. Found the IPv4 default +gateway: + +```bash +ip route show default +``` + +Output: + +```text +default via 172.31.1.1 dev eth0 proto dhcp src 46.225.234.201 metric 100 +``` + +Added the IPv4 policy routing rules: + +```bash +ip route add default via 172.31.1.1 dev eth0 table 100 +ip rule add from 116.202.177.184 table 100 +``` + +Verified: + +```bash +ip rule list +``` + +Output: + +```text +0: from all lookup local +32765: from 116.202.177.184 lookup 100 +32766: from all lookup main +32767: from all lookup default +``` + +```bash +ip route show table 100 +``` + +Output: + +```text +default via 172.31.1.1 dev eth0 +``` + +✅ IPv4 replies from `116.202.177.184` now route via `172.31.1.1` (Hetzner's IPv4 gateway). + +### Check 4 — tcpdump During newTrackon Probe (Attempt 2) + +After applying both policy routing rules, resubmitted to newTrackon. While the probe was +running, captured traffic on the server: + +```bash +sudo tcpdump -i eth0 -n udp port 6969 -v +``` + +Output: + +```text +tcpdump: listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes +13:18:22.128306 IP6 (flowlabel 0xb28c9, hlim 56, next-header UDP (17) payload length: 24) 2a01:4f8:1c1a:715::1.37318 > 2a01:4f8:1c0c:828e::1.6969: [udp sum ok] UDP, length 16 +13:18:32.129210 IP6 (flowlabel 0xdf835, hlim 56, next-header UDP (17) payload length: 24) 2a01:4f8:1c1a:715::1.34285 > 2a01:4f8:1c0c:828e::1.6969: [udp sum ok] UDP, length 16 +``` + +**Key observation**: Only **incoming** packets appear (`2a01:4f8:1c1a:715::1` → `2a01:4f8:1c0c:828e::1`). +There are **no outgoing reply lines** from `2a01:4f8:1c0c:828e::1`. This means: + +- ✅ Packets **arrive at eth0** — no Hetzner cloud firewall blocking upstream +- ❌ Packets are **silently dropped** before the container ever processes them +- The container logs showed zero hits on `:6969` — confirmed packets never reached docker-proxy + +This rules out asymmetric routing as the primary cause: the issue is that packets don't reach the +container at all. Something between `eth0` ingress and Docker is dropping them. + +### Check 5 — ufw Firewall Status + +Inspected the ufw rules on the server: + +```bash +sudo ufw status verbose +``` + +Output: + +```text +Status: active +Logging: on (low) +Default: deny (incoming), allow (outgoing), deny (routed) +New profiles: skip + +To Action From +-- ------ ---- +22/tcp ALLOW IN Anywhere # SSH access (configured port 22) +22/tcp (v6) ALLOW IN Anywhere (v6) # SSH access (configured port 22) +``` + +❌ **`6969/udp` is absent from the allow list.** + +`Default: deny (incoming)` means all inbound traffic not explicitly allowed is dropped. +while Docker bypasses the iptables `INPUT` chain for IPv4 published ports (writing its own DNAT +rules), it does **not** manage `ip6tables` by default. As a result: + +- **IPv4 UDP 6969** — Docker DNAT bypasses ufw INPUT chain → works ✅ +- **IPv6 UDP 6969** — No Docker ip6tables DNAT rule exists → hits ufw INPUT → `default: deny` → dropped + +This explains why IPv4 tests always worked and IPv6 failed. + +### Check 6 — iptables FORWARD Chain + +Verified Docker's FORWARD rules were correctly in place for both IPv4 and IPv6: + +```bash +sudo iptables -L FORWARD --line-numbers -n +sudo ip6tables -L FORWARD --line-numbers -n +``` + +Output: + +```text +Chain FORWARD (policy DROP) +num target prot opt source destination +1 DOCKER-USER 0 -- 0.0.0.0/0 0.0.0.0/0 +2 DOCKER-FORWARD 0 -- 0.0.0.0/0 0.0.0.0/0 +... + +Chain FORWARD (policy DROP) +num target prot opt source destination +1 DOCKER-USER 0 -- ::/0 ::/0 +2 DOCKER-FORWARD 0 -- ::/0 ::/0 +... +``` + +✅ `DOCKER-FORWARD` is present at position 2 for both IPv4 and IPv6. Once packets pass the INPUT +chain, forwarding to the container is handled correctly. + +## Confirmed Root Causes + +Two independent issues were both required to be fixed: + +### Root Cause 1 — ufw Blocking IPv6 UDP 6969 (Primary) + +The ufw firewall default policy is `deny (incoming)`. Port `6969/udp` was never added to the +allow list. For IPv6, Docker does not write `ip6tables` INPUT rules, so packets hit ufw's default +deny policy and are silently dropped before reaching docker-proxy. + +For IPv4, Docker writes DNAT rules directly into `iptables` which bypass the ufw INPUT chain — +that is why IPv4 probes always worked. + +```text +IPv6 path (broken): + probe → eth0 → ip6tables INPUT chain → ufw default deny → dropped ❌ + +IPv4 path (always worked): + probe → eth0 → iptables DNAT (Docker) → bypasses INPUT → container ✅ +``` + +### Root Cause 2 — Asymmetric Routing (Secondary) + +When a UDP probe arrives on a floating IP, the Linux kernel routes the reply using the **default +route** — which sends the packet out via the primary server IP (`2a01:4f8:1c19:620b::1` on IPv6, +`46.225.234.201` on IPv4). newTrackon discards the response because it comes from an unexpected +source address. + +```text +Without policy routing: + probe → arrives on floating IP → container processes → reply leaves via primary IP + newTrackon sees reply from wrong source → discards → "UDP timeout" + +With policy routing: + probe → arrives on floating IP → container processes + → kernel matches "from " rule → routes via table 100/200 + → reply leaves via correct floating IP → newTrackon accepts +``` + +## Historical Context + +### Old Demo Tracker (torrust-demo.com, Digital Ocean) + +The previous Torrust demo tracker was deployed on Digital Ocean with a reserved IPv4 +(`144.126.245.19`). That deployment only served **IPv4** — no IPv6 floating IPs were configured. +This means the asymmetric routing / IPv6 Docker issue was never encountered. + +### This Deployment (torrust-tracker-demo.com, Hetzner) + +This is the **first Torrust deployment routing UDP tracker traffic over IPv6 floating IPs**. +The combination of: + +1. Multiple floating IPs (both IPv4 and IPv6) +2. Docker with default network settings +3. UDP tracker on port 6969 + +…is new territory. Both ufw and asymmetric routing needed to be addressed (see above). + +### Proxy Difference (Nginx vs Caddy) + +The old demo used Nginx as a reverse proxy; this deployment uses Caddy. This is **irrelevant +for UDP tracker traffic** — UDP does not go through the reverse proxy (HTTP only). Both +setups are equivalent from the UDP tracker's perspective. + +## Fix Applied (2026-03-06) + +> Fix 1 (`ufw`) was immediately persistent — `ufw` stores rules in `/etc/ufw/` and they survive +> a reboot. Fixes 2 and 3 (policy routing) were runtime only when first applied; they were +> persisted via netplan in [Step 4](#step-4--persist-policy-routing-via-netplan--2026-03-06) below. + +### Fix 1 — Open UDP 6969 in ufw ✅ + +This was the critical fix that allowed IPv6 UDP packets to reach the container. + +```bash +sudo ufw allow 6969/udp +``` + +Verified: + +```bash +sudo ufw status verbose +``` + +Output: + +```text +Status: active +Logging: on (low) +Default: deny (incoming), allow (outgoing), deny (routed) +New profiles: skip + +To Action From +-- ------ ---- +22/tcp ALLOW IN Anywhere # SSH access (configured port 22) +6969/udp ALLOW IN Anywhere +22/tcp (v6) ALLOW IN Anywhere (v6) # SSH access (configured port 22) +6969/udp (v6) ALLOW IN Anywhere (v6) +``` + +✅ Both IPv4 and IPv6 UDP port 6969 are now allowed in. + +### Fix 2 — IPv6 Policy Routing Rule ✅ + +Already present from an earlier investigation step (see Check 2 above). + +| | | +| ----- | ------------------------------------------ | +| Rule | `from 2a01:4f8:1c0c:828e::1 lookup 200` | +| Route | `default via fe80::1 dev eth0` (table 200) | + +### Fix 3 — IPv4 Policy Routing Rule ✅ + +Added to ensure IPv4 replies also leave via the correct floating IP (see Check 3 above). + +```bash +ip route add default via 172.31.1.1 dev eth0 table 100 +ip rule add from 116.202.177.184 table 100 +``` + +### Step 4 — Persist Policy Routing via Netplan ✅ (2026-03-06) + +The `ip rule` and `ip route` commands from Fixes 2 and 3 are runtime only — they are held in +kernel memory and are lost on reboot. `systemd-networkd` (the network renderer used here) +manages persistent network state via `netplan`. The policy routing rules were added to +`/etc/netplan/60-floating-ip.yaml`. + +#### Netplan File Structure on This Server + +Two netplan files exist on this server. The numeric prefix controls load order — networkd +processes them in ascending order: + +| File | Who manages it | Purpose | +| --------------------- | ---------------------- | ------------------------------------------------------------------ | +| `50-cloud-init.yaml` | cloud-init (automatic) | Primary interface: DHCP4, primary IPv6 address, default IPv6 route | +| `60-floating-ip.yaml` | manually managed | Floating IPs and (after this fix) policy routing rules | + +> ⚠️ Never edit `50-cloud-init.yaml` — cloud-init may regenerate it on the next run and +> overwrite your changes. + +The cloud-init file that was already present: + +```bash +sudo cat /etc/netplan/50-cloud-init.yaml +``` + +Output: + +```yaml +network: + version: 2 + ethernets: + eth0: + match: + macaddress: "92:00:07:4f:b3:4f" + addresses: + - "2a01:4f8:1c19:620b::1/64" + nameservers: + addresses: + - 2a01:4ff:ff00::add:2 + - 2a01:4ff:ff00::add:1 + dhcp4: true + set-name: "eth0" + routes: + - on-link: true + to: "default" + via: "fe80::1" +``` + +Key observations: + +- `dhcp4: true` — the primary IPv4 address (`46.225.234.201`) and default IPv4 route are + assigned by DHCP; the gateway (`172.31.1.1`) discovered in Check 3 came from DHCP. +- The primary IPv6 address `2a01:4f8:1c19:620b::1/64` and its default route via `fe80::1` are + statically configured here. +- `fe80::1` is Hetzner's link-local router address on `eth0`. This is why we reuse it as the + gateway in table 200 for the UDP1 floating IP — it is the only IPv6 gateway available on this + interface. + +#### File Before + +The file only contained the static IP address assignments: + +```yaml +network: + version: 2 + renderer: networkd + ethernets: + eth0: + addresses: + # Existing floating IPs (HTTP1) + - 116.202.176.169/32 + - 2a01:4f8:1c0c:9aae::1/64 + # New floating IPs (UDP1) + - 116.202.177.184/32 + - 2a01:4f8:1c0c:828e::1/64 +``` + +#### Changes Made + +Two new blocks were added under `eth0:`: + +**`routing-policy:`** — one entry per floating IP, mapping the source address to a routing table +number. When a packet's source IP matches `from`, the kernel consults that table instead of the +main routing table. + +**`routes:`** — one default route per table, pointing outbound traffic to the correct gateway. +Table 100 uses the IPv4 gateway (`172.31.1.1`); table 200 uses the IPv6 link-local gateway +(`fe80::1`). + +#### File After + +```bash +sudo cat /etc/netplan/60-floating-ip.yaml +``` + +Output: + +```yaml +network: + version: 2 + renderer: networkd + ethernets: + eth0: + addresses: + # Existing floating IPs (HTTP1) + - 116.202.176.169/32 + - 2a01:4f8:1c0c:9aae::1/64 + # New floating IPs (UDP1) + - 116.202.177.184/32 + - 2a01:4f8:1c0c:828e::1/64 + routing-policy: + - from: 116.202.177.184 + table: 100 + - from: 2a01:4f8:1c0c:828e::1 + table: 200 + routes: + - to: default + via: 172.31.1.1 + table: 100 + - to: default + via: fe80::1 + table: 200 +``` + +> **Requirement**: `routing-policy` and per-table `routes` require `renderer: networkd`. +> Verify the renderer is active with: `systemctl status systemd-networkd`. + +#### Apply + +```bash +sudo netplan apply +``` + +No output on success. `netplan apply` instructs `systemd-networkd` to recalculate and apply the +network configuration — including the new routing tables and policy rules — without taking the +interface down. + +#### Verify + +```bash +ip rule list +ip route show table 100 +ip -6 rule list +ip -6 route show table 200 +``` + +Output: + +```text +0: from all lookup local +32764: from 116.202.177.184 lookup 100 proto static +32766: from all lookup main +32767: from all lookup default +default via 172.31.1.1 dev eth0 proto static +0: from all lookup local +32764: from 2a01:4f8:1c0c:828e::1 lookup 200 proto static +32766: from all lookup main +default via fe80::1 dev eth0 proto static metric 1024 pref medium +``` + +Two differences from manually-added rules (as in Check 2 and Check 3): + +- `proto static` — netplan/networkd marks its routes and rules as `proto static`, whereas + `ip rule add` / `ip route add` without a `proto` flag produce untagged entries. This is + cosmetic only; both work identically. +- Priority `32764` instead of `32765` — networkd assigns its own priority numbers. Again, + functionally equivalent. + +✅ Both routing tables are active and will survive a server reboot — they are now managed by +`systemd-networkd` via netplan. + +#### Why Not Just Re-Run the `ip` Commands After Each Reboot? + +The manual `ip rule add` / `ip route add` commands work for a running system but are not +persistent. On Ubuntu/Debian systems with netplan, options for persistence include: + +- **netplan** (used here) — cleanest approach when using `renderer: networkd` +- `/etc/rc.local` — works on older systems but is not idiomatic on modern Ubuntu +- A `systemd` one-shot service — explicit but verbose + +Netplan is the right choice here because it already manages the floating IP addresses on this +interface. Keeping routing policy alongside address configuration in the same file ensures +both are always applied together. + +## Result + +After applying both fixes (ufw + policy routing): + +```text +URL: udp://udp1.torrust-tracker-demo.com:6969/announce +IP: 2a01:4f8:1c0c:828e::1 +Result: ✅ Accepted +Response: {'interval': 300, 'leechers': 0, 'peers': [], 'seeds': 1} +``` + +![newTrackon — UDP1 accepted](../media/newtrackon-submitted-udp1-accepted.png) + +## Impact + +This issue **no longer blocks** the UDP1 tracker. All tracker functionality is operational: + +- HTTP tracker — Caddy → Docker on IPv4 ✅ +- IPv4 UDP tracker ✅ +- IPv6 UDP tracker via floating IP `2a01:4f8:1c0c:828e::1` ✅ +- HTTP1 and UDP1 trackers listed on newTrackon ✅ + +## Cross-Repository Note + +This issue should also be documented in the +[torrust-tracker](https://github.com/torrust/torrust-tracker) repository, as it involves +the tracker's network configuration requirements when running with multiple IPv6 floating IPs. +Any future deployment guide covering IPv6 should mention: + +1. Open firewall port for UDP tracker: `sudo ufw allow /udp` — Docker does **not** manage + `ip6tables` INPUT rules, so ufw's default deny blocks all IPv6 inbound unless explicitly allowed +2. Verify with `sudo ss -ulnp | grep ` that the tracker binds to both `0.0.0.0` and `[::]` +3. Policy-based routing is required for each floating IP to ensure replies leave via the correct + source address (both IPv4 and IPv6) + +## Related + +- [Issue #407 — Submit UDP1 Tracker to newTrackon](../../../issues/407-submit-udp1-tracker-to-newtrackon.md) +- [newTrackon Prerequisites](newtrackon-prerequisites.md) +- [Netplan Configuration](newtrackon-prerequisites.md#step-3--configure-all-floating-ips-permanently-via-netplan-) diff --git a/docs/deployments/hetzner-demo-tracker/post-provision/newtrackon-prerequisites.md b/docs/deployments/hetzner-demo-tracker/post-provision/newtrackon-prerequisites.md new file mode 100644 index 00000000..49796801 --- /dev/null +++ b/docs/deployments/hetzner-demo-tracker/post-provision/newtrackon-prerequisites.md @@ -0,0 +1,502 @@ +# newTrackon Prerequisites + +> **Status**: ✅ Complete — HTTP1 and UDP1 trackers both listed on newTrackon (2026-03-06) + +This document captures the newTrackon prerequisites that were **not** addressed during the initial +tracker submission on 2026-03-04 and the steps being taken to fix them. + +## Context + +During the original deployment (issue #405), we attempted to submit both trackers to +[newTrackon](https://newtrackon.com/): + +- `https://http1.torrust-tracker-demo.com/announce` — **✅ Accepted** +- `udp://udp1.torrust-tracker-demo.com:6969/announce` — **❌ Not accepted** + +The HTTP1 tracker was listed successfully. The UDP1 tracker was not accepted because two +prerequisites were missed: + +1. **BEP 34 DNS TXT records** were not set on the tracker domains. +2. **One tracker per IP policy**: The UDP1 subdomain resolves to the same IPs already used by + HTTP1, violating newTrackon's uniqueness requirement. + +## newTrackon Prerequisites + +### Prerequisite 1 — BEP 34 DNS TXT Record + +[BEP 34](https://www.bittorrent.org/beps/bep_0034.html) defines a DNS TXT record format that +announces which ports a domain is intentionally serving as a BitTorrent tracker. newTrackon uses +this record to validate submissions. + +**Record format**: `"BITTORRENT UDP: TCP:"` + +You only include the protocols that the tracker actually serves. Examples: + +```text +# TCP (HTTP/WebSocket) tracker on port 443 +"BITTORRENT TCP:443" + +# UDP tracker on port 6969 +"BITTORRENT UDP:6969" + +# Both protocols on the same domain +"BITTORRENT UDP:6969 TCP:443" +``` + +**Reference deployment** — the old demo tracker (`tracker.torrust-demo.com`) has: + +```text +dig TXT tracker.torrust-demo.com +;; ANSWER SECTION: +tracker.torrust-demo.com. 3600 IN TXT "BITTORRENT UDP:6969 TCP:443" +``` + +**Records required for this deployment**: + +| Domain | TXT value | Protocol served | +| -------------------------------- | --------------------- | --------------- | +| `http1.torrust-tracker-demo.com` | `BITTORRENT TCP:443` | HTTP tracker | +| `udp1.torrust-tracker-demo.com` | `BITTORRENT UDP:6969` | UDP tracker | + +> **Note**: These TXT records were missing during the initial submission. The HTTP1 tracker was +> accepted without one, but the UDP1 tracker was not. Adding them for both subdomains ensures +> full compliance going forward. + +### Prerequisite 2 — One Tracker Per IP Address + +newTrackon enforces that each listed tracker resolves to at least one IP address that is **not** +already used by another tracker already in the list. + +**Current situation**: + +- `http1.torrust-tracker-demo.com` resolves to: + - IPv4: `116.202.176.169` + - IPv6: `2a01:4f8:1c0c:9aae::1` +- `udp1.torrust-tracker-demo.com` also resolves to the same two IPs (shared with HTTP1). + +Because HTTP1 already occupies both IPs, the UDP1 submission is rejected. + +**Solution**: Provision two new Hetzner floating IPs (one IPv4, one IPv6) and point +`udp1.torrust-tracker-demo.com` exclusively to them. + +**New IPs provisioned (2026-03-06)**: + +| Name | Type | Address | +| ----------- | ---- | ----------------------- | +| `udp1-ipv4` | IPv4 | `116.202.177.184` | +| `udp1-ipv6` | IPv6 | `2a01:4f8:1c0c:828e::1` | + +![Hetzner Console — All four floating IPs](../media/hetzner-console-all-four-floating-ips.png) + +## DNS State Before Changes (2026-03-06) + +Recorded before making any DNS changes so we can verify the effect afterwards. + +Screenshot of the Hetzner DNS panel: + +![Hetzner Console — DNS config before udp1 changes](../media/hetzner-console-dns-config-before-udp1-changes.png) + +`dig` output — all six subdomains resolve to the same two shared IPs, no TXT records exist: + +```text +$ dig A {http1,http2,udp1,udp2,api,grafana}.torrust-tracker-demo.com +short +# All return: 116.202.176.169 + +$ dig AAAA {http1,http2,udp1,udp2,api,grafana}.torrust-tracker-demo.com +short +# All return: 2a01:4f8:1c0c:9aae::1 + +$ dig TXT {http1,http2,udp1,udp2}.torrust-tracker-demo.com +short +# (no output — no TXT records set) +``` + +Expected state after changes: + +| Subdomain | A record | AAAA record | TXT record | +| --------- | ----------------- | ----------------------- | ----------------------- | +| `http1` | `116.202.176.169` | `2a01:4f8:1c0c:9aae::1` | `"BITTORRENT TCP:443"` | +| `http2` | `116.202.176.169` | `2a01:4f8:1c0c:9aae::1` | — | +| `udp1` | `116.202.177.184` | `2a01:4f8:1c0c:828e::1` | `"BITTORRENT UDP:6969"` | +| `udp2` | `116.202.176.169` | `2a01:4f8:1c0c:9aae::1` | — | +| `api` | `116.202.176.169` | `2a01:4f8:1c0c:9aae::1` | — | +| `grafana` | `116.202.176.169` | `2a01:4f8:1c0c:9aae::1` | — | + +## Fix Plan + +### Step 1 — Add BEP 34 TXT Records via Hetzner DNS API ✅ Done (2026-03-06) + +Added directly in the Hetzner DNS panel (TTL 300, consistent with all other records in the zone): + +```text +http1 300 IN TXT "BITTORRENT TCP:443" +udp1 300 IN TXT "BITTORRENT UDP:6969" +``` + +Verified with `dig`: + +```text +$ dig TXT http1.torrust-tracker-demo.com +short +"BITTORRENT TCP:443" + +$ dig TXT udp1.torrust-tracker-demo.com +short +"BITTORRENT UDP:6969" +``` + +**Reference** — the same records can be added via the Hetzner DNS API: + +Add TXT records for both tracker subdomains using the Hetzner DNS API: + +```bash +# HTTP1 — TCP tracker on port 443 +curl -X POST "https://dns.hetzner.com/api/v1/records" \ + -H "Auth-API-Token: $HETZNER_DNS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "zone_id": "", + "type": "TXT", + "name": "http1", + "value": "\"BITTORRENT TCP:443\"", + "ttl": 300 + }' + +# UDP1 — UDP tracker on port 6969 +curl -X POST "https://dns.hetzner.com/api/v1/records" \ + -H "Auth-API-Token: $HETZNER_DNS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "zone_id": "", + "type": "TXT", + "name": "udp1", + "value": "\"BITTORRENT UDP:6969\"", + "ttl": 300 + }' +``` + +Verify with `dig`: + +```bash +dig TXT http1.torrust-tracker-demo.com +dig TXT udp1.torrust-tracker-demo.com +``` + +### Step 2 — Provision New Floating IPs ✅ Done (2026-03-06) + +In the [Hetzner Console](https://console.hetzner.cloud/) under the `torrust-tracker-demo.com` +project: + +1. Go to **Networking → Floating IPs**. +2. Click **Add Floating IP**. +3. Select **Type: IPv4**, region **Nuremberg (nbg1)**, then create. +4. Repeat for **Type: IPv6**, same region. +5. Assign both new IPs to server `torrust-tracker-vm-torrust-tracker-demo`. + +New IPs created and assigned: + +| Name | Type | Address | +| ----------- | ---- | ----------------------- | +| `udp1-ipv4` | IPv4 | `116.202.177.184` | +| `udp1-ipv6` | IPv6 | `2a01:4f8:1c0c:828e::1` | + +### Step 3 — Configure All Floating IPs Permanently via Netplan ✅ Done (2026-03-06) + +The original floating IPs (`116.202.176.169` and `2a01:4f8:1c0c:9aae::1`) were configured +temporarily (using `ip addr add`) and were **not** persisted via netplan. This step fixes that +and adds the new IPs at the same time. + +SSH into the server and updated `/etc/netplan/60-floating-ip.yaml`: + +```yaml +network: + version: 2 + renderer: networkd + ethernets: + eth0: + addresses: + # Existing floating IPs (HTTP1 / http1.torrust-tracker-demo.com) + - 116.202.176.169/32 + - 2a01:4f8:1c0c:9aae::1/64 + # New floating IPs (UDP1 / udp1.torrust-tracker-demo.com) + - 116.202.177.184/32 + - 2a01:4f8:1c0c:828e::1/64 +``` + +> **Note**: IPv6 floating IPs use `/64` prefix (consistent with the existing Hetzner convention), +> not `/128`. + +Applied and verified: + +```bash +sudo chmod 600 /etc/netplan/60-floating-ip.yaml +sudo netplan apply +ip addr show eth0 +``` + +Actual output: + +```text +2: eth0: mtu 1500 qdisc fq_codel state UP group default qlen 1000 + link/ether 92:00:07:4f:b3:4f brd ff:ff:ff:ff:ff:ff + inet 116.202.176.169/32 scope global eth0 + valid_lft forever preferred_lft forever + inet 116.202.177.184/32 scope global eth0 + valid_lft forever preferred_lft forever + inet 46.225.234.201/32 metric 100 scope global dynamic eth0 + valid_lft 86394sec preferred_lft 86394sec + inet6 2a01:4f8:1c0c:828e::1/64 scope global + valid_lft forever preferred_lft forever + inet6 2a01:4f8:1c0c:9aae::1/64 scope global + valid_lft forever preferred_lft forever + inet6 2a01:4f8:1c19:620b::1/64 scope global + valid_lft forever preferred_lft forever + inet6 fe80::9000:7ff:fe4f:b34f/64 scope link + valid_lft forever preferred_lft forever +``` + +✅ All four floating IPs (`116.202.176.169`, `116.202.177.184`, `2a01:4f8:1c0c:9aae::1`, +`2a01:4f8:1c0c:828e::1`) are active on `eth0` with `valid_lft forever`, confirming the +netplan config is persistent across reboots. + +Traffic verified with ping from an external host (2026-03-06): + +```text +# IPv4 — 116.202.177.184 +$ ping -c 3 116.202.177.184 +PING 116.202.177.184 (116.202.177.184) 56(84) bytes of data. +64 bytes from 116.202.177.184: icmp_seq=1 ttl=45 time=71.9 ms +64 bytes from 116.202.177.184: icmp_seq=2 ttl=45 time=71.3 ms +64 bytes from 116.202.177.184: icmp_seq=3 ttl=45 time=70.6 ms +3 packets transmitted, 3 received, 0% packet loss + +# IPv6 — not tested (no IPv6 connectivity on the test machine) +# The address is active on eth0 with valid_lft forever (confirmed via ip addr above) +``` + +### Step 4 — Update DNS for UDP1 Subdomain ✅ Done (2026-03-06) + +Updated via the Hetzner DNS panel directly: + +- A record for `udp1`: `116.202.176.169` → `116.202.177.184` +- AAAA record for `udp1`: `2a01:4f8:1c0c:9aae::1` → `2a01:4f8:1c0c:828e::1` + +![Hetzner Console — DNS config after udp1 changes](../media/hetzner-console-dns-config-after-udp1-changes.png) + +Verified with `dig`: + +```text +$ dig A udp1.torrust-tracker-demo.com +short +116.202.177.184 + +$ dig AAAA udp1.torrust-tracker-demo.com +short +2a01:4f8:1c0c:828e::1 +``` + +✅ Both records resolve to the new floating IPs assigned exclusively to the UDP1 tracker. + +**Reference** — records can also be updated via the Hetzner Cloud API: + +```bash +# Get existing record IDs first +curl -s "https://dns.hetzner.com/api/v1/records?zone_id=" \ + -H "Auth-API-Token: $HETZNER_DNS_TOKEN" \ + | jq '.records[] | select(.name == "udp1")' + +# Update A record +curl -X PUT "https://dns.hetzner.com/api/v1/records/" \ + -H "Auth-API-Token: $HETZNER_DNS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "zone_id": "", + "type": "A", + "name": "udp1", + "value": "116.202.177.184", + "ttl": 300 + }' + +# Update AAAA record +curl -X PUT "https://dns.hetzner.com/api/v1/records/" \ + -H "Auth-API-Token: $HETZNER_DNS_TOKEN" \ + -H "Content-Type: application/json" \ + -d '{ + "zone_id": "", + "type": "AAAA", + "name": "udp1", + "value": "2a01:4f8:1c0c:828e::1", + "ttl": 300 + }' +``` + +### Step 5 — Submit UDP1 to newTrackon + +1. Go to +2. Paste `udp://udp1.torrust-tracker-demo.com:6969/announce` into the submission box +3. Click **Submit** +4. Wait a few minutes while newTrackon probes the tracker +5. Verify acceptance and appearance in the [tracker list](https://newtrackon.com/list) + +Verify via API: + +```bash +curl -s https://newtrackon.com/api/stable | grep udp1.torrust-tracker-demo.com +``` + +#### Attempt 1 — 2026-03-06: Rejected (UDP timeout) + +Submitted `udp://udp1.torrust-tracker-demo.com:6969/announce`. newTrackon probed the tracker +using the new IPv6 address (`2a01:4f8:1c0c:828e::1`) but received no response: + +![newTrackon submitted page — UDP1 rejected with UDP timeout](../media/newtrackon-submitted-udp1-rejected-udp-timeout.png) + +| Field | Value | +| --------- | --------------------------------------------------- | +| URL | `udp://udp1.torrust-tracker-demo.com:6969/announce` | +| IP probed | `2a01:4f8:1c0c:828e::1` | +| Result | ❌ Rejected | +| Error | UDP timeout | + +**Root cause analysis**: The IP is correctly configured on `eth0` and DNS resolves correctly. +"UDP timeout" means packets reached the server but no UDP response was sent back. Likely causes: + +1. **Asymmetric routing**: The tracker responds via the primary IP (`46.225.234.201`) rather + than the floating IP the probe arrived on — newTrackon discards the response because the + source IP doesn't match. This requires policy-based routing (a routing table per floating IP). +2. **Firewall**: The Hetzner firewall or `ufw` may be dropping UDP 6969 on the new IP. +3. **Docker not routing to floating IP**: The tracker container receives the packet on + `0.0.0.0:6969` but the kernel's default route sends the reply out via the wrong interface. + +This is a **new blocker** — see [IPv6 UDP Tracker Issue](ipv6-udp-tracker-issue.md) for the +full investigation and resolution. + +#### Attempt 2 — 2026-03-06: Rejected Again (UDP timeout) + +Resubmitted after applying IPv4 + IPv6 policy routing rules (table 100 and table 200). The +probe still returned "UDP timeout". + +See [IPv6 UDP Tracker Issue](ipv6-udp-tracker-issue.md) for the full investigation. While +the policy routing was correct, a separate blocker remained: the ufw firewall was silently +dropping all IPv6 UDP 6969 packets before they reached the Docker container. + +#### Attempt 3 — 2026-03-06: Accepted ✅ + +During investigation of Attempt 2, ran `tcpdump` on the server while the newTrackon probe was +in progress. Packets arrived at `eth0` but no replies were sent — confirming packets were never +forwarded to the container. + +##### Step 1 — Confirm packets arrive at the server (tcpdump) + +```bash +sudo tcpdump -i eth0 -n udp port 6969 -v +``` + +Output during newTrackon probe: + +```text +tcpdump: listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes +13:18:22.128306 IP6 (flowlabel 0xb28c9, hlim 56, next-header UDP (17) payload length: 24) 2a01:4f8:1c1a:715::1.37318 > 2a01:4f8:1c0c:828e::1.6969: [udp sum ok] UDP, length 16 +13:18:32.129210 IP6 (flowlabel 0xdf835, hlim 56, next-header UDP (17) payload length: 24) 2a01:4f8:1c1a:715::1.34285 > 2a01:4f8:1c0c:828e::1.6969: [udp sum ok] UDP, length 16 +``` + +✅ Packets arrive at `eth0`. ❌ No replies — something is dropping them before the container. + +##### Step 2 — Check iptables FORWARD chain + +```bash +sudo iptables -L FORWARD --line-numbers -n +sudo ip6tables -L FORWARD --line-numbers -n +``` + +Both chains had `DOCKER-FORWARD` at position 2 — forwarding rules are in place. The blockage +is earlier, in the INPUT chain. + +##### Step 3 — Check ufw rules + +```bash +sudo ufw status verbose +``` + +Output: + +```text +Status: active +Logging: on (low) +Default: deny (incoming), allow (outgoing), deny (routed) +New profiles: skip + +To Action From +-- ------ ---- +22/tcp ALLOW IN Anywhere # SSH access (configured port 22) +22/tcp (v6) ALLOW IN Anywhere (v6) # SSH access (configured port 22) +``` + +❌ **Port `6969/udp` was missing.** ufw's default `deny (incoming)` was dropping the packets. + +Docker bypasses the ufw INPUT chain for IPv4 using DNAT rules (which is why IPv4 always worked), +but it does **not** create `ip6tables` INPUT rules. IPv6 UDP hits the INPUT chain and is dropped. + +##### Step 4 — Open UDP 6969 in ufw + +```bash +sudo ufw allow 6969/udp +``` + +Output: + +```text +Rule added +Rule added (v6) +``` + +Verified: + +```bash +sudo ufw status verbose +``` + +Output: + +```text +Status: active +Logging: on (low) +Default: deny (incoming), allow (outgoing), deny (routed) +New profiles: skip + +To Action From +-- ------ ---- +22/tcp ALLOW IN Anywhere # SSH access (configured port 22) +6969/udp ALLOW IN Anywhere +22/tcp (v6) ALLOW IN Anywhere (v6) # SSH access (configured port 22) +6969/udp (v6) ALLOW IN Anywhere (v6) +``` + +Resubmitted to newTrackon immediately after. Result: + +| Field | Value | +| -------- | ----------------------------------------------------------- | +| URL | `udp://udp1.torrust-tracker-demo.com:6969/announce` | +| IP | `2a01:4f8:1c0c:828e::1` | +| Result | ✅ Accepted | +| Response | `{'interval': 300, 'leechers': 0, 'peers': [], 'seeds': 1}` | + +![newTrackon — UDP1 accepted](../media/newtrackon-submitted-udp1-accepted.png) + +## Status + +| Item | Status | Date | +| --------------------------------------- | ----------- | ---------- | +| BEP 34 TXT record for `http1` | ✅ Done | 2026-03-06 | +| BEP 34 TXT record for `udp1` | ✅ Done | 2026-03-06 | +| New IPv4 floating IP provisioned | ✅ Done | 2026-03-06 | +| New IPv6 floating IP provisioned | ✅ Done | 2026-03-06 | +| New IPs assigned to server | ✅ Done | 2026-03-06 | +| All floating IPs configured via netplan | ✅ Done | 2026-03-06 | +| DNS A/AAAA records updated for `udp1` | ✅ Done | 2026-03-06 | +| UDP1 tracker submitted to newTrackon | ✅ Accepted | 2026-03-06 | +| UDP1 tracker listed on newTrackon | ✅ Listed | 2026-03-06 | + +![newTrackon — three trackers listed including http1 and udp1 torrust-tracker-demo.com](../media/newtrackon-home-three-trackers-listed.png) + +## Related + +- [Issue #407 — Submit UDP1 Tracker to newTrackon](../../../issues/407-submit-udp1-tracker-to-newtrackon.md) +- [BEP 34 — DNS Tracker Preferences](https://www.bittorrent.org/beps/bep_0034.html) +- [newTrackon](https://newtrackon.com/) +- [DNS Setup](dns-setup.md) +- [Tracker Registry](../tracker-registry.md) diff --git a/docs/deployments/hetzner-demo-tracker/prerequisites.md b/docs/deployments/hetzner-demo-tracker/prerequisites.md index b8971d69..e8c5b3d6 100644 --- a/docs/deployments/hetzner-demo-tracker/prerequisites.md +++ b/docs/deployments/hetzner-demo-tracker/prerequisites.md @@ -133,10 +133,17 @@ Floating IPs are created in Hetzner Console → project → Networking → Float **Names and addresses**: -| Name | Type | Address | -| --------------------------- | ---- | ------------------------- | -| `torrust-tracker-demo-ipv4` | IPv4 | `116.202.176.169` | -| `torrust-tracker-demo-ipv6` | IPv6 | `2a01:4f8:1c0c:9aae::/64` | +| Name | Type | Address | +| ------------ | ---- | ------------------------- | +| `http1-ipv4` | IPv4 | `116.202.176.169` | +| `http1-ipv6` | IPv6 | `2a01:4f8:1c0c:9aae::/64` | + +> **Note**: These IPs were originally created as `torrust-tracker-demo-ipv4` and +> `torrust-tracker-demo-ipv6` and later renamed to `http1-ipv4` / `http1-ipv6` to +> avoid confusion when provisioning additional floating IPs for the UDP1 tracker +> (see [issue #407](https://github.com/torrust/torrust-tracker-deployer/issues/407)). + +![Hetzner Console — Floating IPs renamed for HTTP1](media/hetzner-console-floating-ips-renamed-for-http1.png) ![Hetzner Console — Create floating IPv4 form](media/hetzner-console-create-floating-ip-ipv4-form.png) @@ -144,8 +151,8 @@ Floating IPs are created in Hetzner Console → project → Networking → Float ![Hetzner Console — Floating IPs list](media/hetzner-console-floating-ips-list.png) -- [x] IPv4 floating IP created in Hetzner project (`torrust-tracker-demo-ipv4`, `nbg1`, `116.202.176.169`) -- [x] IPv6 floating IP created in Hetzner project (`torrust-tracker-demo-ipv6`, `nbg1`, `2a01:4f8:1c0c:9aae::/64`) +- [x] IPv4 floating IP created in Hetzner project (`http1-ipv4`, `nbg1`, `116.202.176.169`) +- [x] IPv6 floating IP created in Hetzner project (`http1-ipv6`, `nbg1`, `2a01:4f8:1c0c:9aae::/64`) - [ ] Both IPs assigned to the server (after provisioning) ### Volume for Storage (⚠️ deferred — do after `release`) diff --git a/docs/deployments/hetzner-demo-tracker/tracker-registry.md b/docs/deployments/hetzner-demo-tracker/tracker-registry.md index c2195456..f8cbab5d 100644 --- a/docs/deployments/hetzner-demo-tracker/tracker-registry.md +++ b/docs/deployments/hetzner-demo-tracker/tracker-registry.md @@ -13,29 +13,95 @@ The previous Torrust demo tracker (`udp://tracker.torrust-demo.com:6969/announce was already listed there. The new Hetzner demo tracker should be submitted as well. -### Which tracker to submit +### Which trackers to submit -Only **UDP Tracker 1** is submitted to public registries: +We submit two trackers from this deployment to public registries: + +| Tracker | URL | Status | +| -------------- | --------------------------------------------------- | --------- | +| HTTP Tracker 1 | `https://http1.torrust-tracker-demo.com/announce` | ✅ Listed | +| UDP Tracker 1 | `udp://udp1.torrust-tracker-demo.com:6969/announce` | ✅ Listed | + +**HTTP Tracker 2**, **UDP Tracker 2**, the REST API, and Grafana are intentionally kept off +all public tracker lists. Once a tracker appears in public lists it receives a continuous stream +of announces from BitTorrent clients worldwide. That background noise makes it very hard to read +logs and debug issues when testing something in production. Keeping `http2` and `udp2` quiet +reserves them as low-traffic endpoints for manual testing and investigation. + +### newTrackon Prerequisites + +Before submitting a tracker to newTrackon, two prerequisites must be met: + +#### 1. BEP 34 DNS TXT Record + +[BEP 34](https://www.bittorrent.org/beps/bep_0034.html) requires a DNS TXT record on the +tracker domain declaring which ports it serves: ```text -udp://udp1.torrust-tracker-demo.com:6969/announce +"BITTORRENT UDP: TCP:" ``` -**UDP Tracker 2** (`udp://udp2.torrust-tracker-demo.com:6868/announce`) is -intentionally kept off all public tracker lists. Once a tracker appears in -public lists it receives a continuous stream of announces from BitTorrent -clients worldwide. That background noise makes it very hard to read logs -and debug issues when testing something in production. Keeping `udp2` quiet -reserves it as a low-traffic endpoint for manual testing and investigation. +For example, the old demo tracker has `"BITTORRENT UDP:6969 TCP:443"` on +`tracker.torrust-demo.com`. Without this record, newTrackon may reject the submission. + +#### 2. Unique IP Address (One Tracker Per IP) + +newTrackon only accepts one tracker per IP address. If two tracker URLs resolve to the same +IP(s), only one can be listed. Each submitted tracker must resolve to at least one IP not +already used by another listed tracker. + +> **What we missed**: During the initial submission (2026-03-04) we did not add BEP 34 TXT +> records for either subdomain, and both `http1` and `udp1` shared the same two floating IPs. +> The HTTP1 tracker was accepted despite the missing TXT record; the UDP1 tracker was not. +> See [post-provision/newtrackon-prerequisites.md](post-provision/newtrackon-prerequisites.md) +> for the complete fix plan and current status. + +### Submission History + +#### HTTP Tracker 1 + +- **URL**: `https://http1.torrust-tracker-demo.com/announce` +- **Submitted**: 2026-03-04 +- **Accepted**: ✅ Yes — listed on newTrackon +- **IPs at submission**: `116.202.176.169` (IPv4), `2a01:4f8:1c0c:9aae::1` (IPv6) + +#### UDP Tracker 1 + +- **URL**: `udp://udp1.torrust-tracker-demo.com:6969/announce` +- **Submitted**: 2026-03-06 (attempt 3) +- **Accepted**: ✅ Yes — listed on newTrackon +- **IPs at submission**: `116.202.177.184` (IPv4), `2a01:4f8:1c0c:828e::1` (IPv6) +- **Notes**: Two blockers required fixing before acceptance: + 1. ufw was blocking IPv6 UDP 6969 — fixed with `sudo ufw allow 6969/udp` + 2. Policy routing tables (100/200) needed to ensure replies leave via the floating IP + + See [post-provision/ipv6-udp-tracker-issue.md](post-provision/ipv6-udp-tracker-issue.md) and + [post-provision/newtrackon-prerequisites.md](post-provision/newtrackon-prerequisites.md). + +### Final State — Both Trackers Listed (2026-03-06) + +![newTrackon — three trackers listed including http1 and udp1 torrust-tracker-demo.com](media/newtrackon-home-three-trackers-listed.png) + +All three trackers visible in the screenshot: + +| Tracker | Notes | +| ----------------------------------------------------- | ------------------------------------- | +| `https://http1.torrust-tracker-demo.com:443/announce` | This deployment — HTTP tracker | +| `udp://udp1.torrust-tracker-demo.com:6969/announce` | This deployment — UDP tracker | +| `udp://tracker.torrust-demo.com:6969/announce` | Previous Torrust demo (Digital Ocean) | ### How to submit -1. Go to -2. Paste `udp://udp1.torrust-tracker-demo.com:6969/announce` into the submission box -3. Click **Submit** -4. Wait a few minutes while newTrackon gathers data -5. Verify it appears in the [Submitted](https://newtrackon.com/submitted) section +1. Ensure all prerequisites are met (see section above and + [post-provision/newtrackon-prerequisites.md](post-provision/newtrackon-prerequisites.md)) +2. Go to +3. Paste the tracker URL into the submission box +4. Click **Submit** +5. Wait a few minutes while newTrackon probes the tracker +6. Verify it appears in the [tracker list](https://newtrackon.com/list) -### Status +You can also verify via the newTrackon API: -✅ Submitted (2026-03-04) — pending appearance in the public list. +```bash +curl -s https://newtrackon.com/api/stable +``` diff --git a/docs/issues/407-submit-udp1-tracker-to-newtrackon.md b/docs/issues/407-submit-udp1-tracker-to-newtrackon.md new file mode 100644 index 00000000..d398c144 --- /dev/null +++ b/docs/issues/407-submit-udp1-tracker-to-newtrackon.md @@ -0,0 +1,187 @@ +# Submit UDP1 Tracker to newTrackon + +**Issue**: #407 +**Parent Epic**: None +**Related**: [#405 - Deploy Hetzner Demo Tracker](405-deploy-hetzner-demo-tracker-and-document-process.md), +[docs/deployments/hetzner-demo-tracker/tracker-registry.md](../deployments/hetzner-demo-tracker/tracker-registry.md) + +## Overview + +After deploying the Hetzner demo tracker (issue #405), the HTTP1 tracker was successfully submitted to +[newTrackon](https://newtrackon.com/). However, the UDP1 tracker submission failed to be accepted. + +Two prerequisites were missed during the initial submission: + +1. **BEP 34 DNS TXT records**: newTrackon requires a DNS TXT record on the tracker's domain following + [BEP 34](https://www.bittorrent.org/beps/bep_0034.html) to announce which ports the tracker uses. +2. **One tracker per IP policy**: newTrackon only accepts one tracker per IP address. The HTTP1 tracker + already occupies the two floating IPs assigned to the server + (`116.202.176.169` and `2a01:4f8:1c0c:9aae::1`), so two new floating IPs (IPv4 + IPv6) must be + provisioned and assigned to support the UDP1 tracker. + +This task documents and resolves both blockers so that the UDP1 tracker can be listed on newTrackon, +and updates the deployment documentation to make the newTrackon prerequisites explicit for future +deployments. + +## Goals + +- [x] Add BEP 34 DNS TXT records for `http1.torrust-tracker-demo.com` (port 443) and + `udp1.torrust-tracker-demo.com` (port 6969) +- [x] Provision two new Hetzner floating IPs (IPv4 + IPv6) and assign them to the existing server +- [x] Configure the new IPs permanently inside the VM (netplan) +- [x] Configure DNS A/AAAA records so `udp1.torrust-tracker-demo.com` resolves to the new IPs +- [x] Retry submission of `udp://udp1.torrust-tracker-demo.com:6969/announce` to newTrackon + (Attempt 3: ✅ Accepted — 2026-03-06; root causes: ufw blocking IPv6 UDP 6969 + asymmetric routing) +- [x] Verify UDP1 tracker appears in the newTrackon public list +- [x] Document the complete process (prerequisites, steps, outcomes) in the deployment docs + +## Specifications + +### BEP 34 DNS TXT Record + +[BEP 34](https://www.bittorrent.org/beps/bep_0034.html) defines a DNS-based method for announcing +tracker availability. newTrackon uses this to validate that a domain is intentionally serving a +BitTorrent tracker on the submitted port. + +The TXT record format is: + +```text +"BITTORRENT UDP: TCP:" +``` + +For example, the old demo tracker (`tracker.torrust-demo.com`) has: + +```text +"BITTORRENT UDP:6969 TCP:443" +``` + +Records to add for the new demo: + +| Domain | TXT value | +| -------------------------------- | --------------------- | +| `http1.torrust-tracker-demo.com` | `BITTORRENT TCP:443` | +| `udp1.torrust-tracker-demo.com` | `BITTORRENT UDP:6969` | + +### One IP Per Tracker (newTrackon Policy) + +newTrackon enforces that each listed tracker resolves to unique IP addresses not already used by +another listed tracker. The HTTP1 tracker already occupies: + +- IPv4: `116.202.176.169` +- IPv6: `2a01:4f8:1c0c:9aae::1` + +To add the UDP1 tracker, two new floating IPs must be provisioned in Hetzner and associated +exclusively with the `udp1` subdomain. + +### Floating IP Configuration + +New floating IPs must be made persistent inside the VM using netplan. The current floating IPs +were **not** configured with netplan — this task also covers making all four floating IPs +permanent (both existing and new ones). + +Netplan configuration path: `/etc/netplan/60-floating-ip.yaml` + +Example netplan stanza for a floating IPv4: + +```yaml +network: + version: 2 + renderer: networkd + ethernets: + eth0: + addresses: + - 116.202.176.169/32 +``` + +> **Note**: Hetzner uses `/64` prefix for IPv6 floating IPs (not `/128`). + +### DNS Records for New IPs + +Once the new floating IPs are provisioned, A and AAAA records must be created for +`udp1.torrust-tracker-demo.com` pointing to those new IPs via the Hetzner DNS API. + +## Implementation Plan + +### Phase 1: DNS BEP 34 TXT Records + +- [x] Task 1.1: Add TXT record `"BITTORRENT TCP:443"` to `http1.torrust-tracker-demo.com` via Hetzner DNS API +- [x] Task 1.2: Add TXT record `"BITTORRENT UDP:6969"` to `udp1.torrust-tracker-demo.com` via Hetzner DNS API +- [x] Task 1.3: Verify both TXT records resolve correctly with `dig TXT ` +- [x] Task 1.4: Create `docs/deployments/hetzner-demo-tracker/post-provision/newtrackon-prerequisites.md` + documenting the BEP 34 requirement and the TXT records added +- [x] Task 1.5: Update `docs/deployments/hetzner-demo-tracker/README.md` to reference the new document + +### Phase 2: Provision New Floating IPs + +- [x] Task 2.1: Book a new IPv4 floating IP in Hetzner Cloud Console (region `nbg1`) — `udp1-ipv4`: `116.202.177.184` +- [x] Task 2.2: Book a new IPv6 floating IP in Hetzner Cloud Console (region `nbg1`) — `udp1-ipv6`: `2a01:4f8:1c0c:828e::1` +- [x] Task 2.3: Assign both new floating IPs to the existing demo server in Hetzner Console +- [x] Task 2.4: Add the "one tracker per IP" policy section to `newtrackon-prerequisites.md` + +### Phase 3: Configure New IPs Inside the VM + +- [x] Task 3.1: SSH into the server +- [x] Task 3.2: Add all four floating IPs to netplan configuration (`/etc/netplan/60-floating-ip.yaml`) + — both existing IPs (which were not previously configured via netplan) and the two new ones +- [x] Task 3.3: Apply netplan configuration (`sudo netplan apply`) and verify all IPs are active +- [x] Task 3.4: Confirm the new IPs receive traffic (IPv4 ping test from external host: ✅; + IPv6 not testable from local machine — confirmed active on `eth0` via `ip addr`) +- [x] Task 3.5: Document the netplan configuration steps and file content in `newtrackon-prerequisites.md` + +### Phase 4: Update DNS for UDP1 Subdomain + +- [x] Task 4.1: Update (or add) A record for `udp1.torrust-tracker-demo.com` pointing to the new IPv4 +- [x] Task 4.2: Update (or add) AAAA record for `udp1.torrust-tracker-demo.com` pointing to the new IPv6 +- [x] Task 4.3: Verify DNS resolution with `dig A udp1.torrust-tracker-demo.com` and + `dig AAAA udp1.torrust-tracker-demo.com` +- [x] Task 4.4: Update `docs/deployments/hetzner-demo-tracker/post-provision/dns-setup.md` with the + new A/AAAA records added for `udp1.torrust-tracker-demo.com` + +### Phase 5: Submit UDP1 Tracker to newTrackon + +- [x] Task 5.1: Go to and submit `udp://udp1.torrust-tracker-demo.com:6969/announce` +- [x] Task 5.2: Verify submission is accepted (no error message from newTrackon) +- [x] Task 5.3: Wait for the tracker to appear in the [newTrackon list](https://newtrackon.com/list) +- [x] Task 5.4: Verify via newTrackon API: `curl https://newtrackon.com/api/stable` +- [x] Task 5.5: Update `docs/deployments/hetzner-demo-tracker/tracker-registry.md` with the final + submission status for the UDP1 tracker and link to `newtrackon-prerequisites.md` + +## Acceptance Criteria + +> **Note for Contributors**: These criteria define what the PR reviewer will check. Use this as your +> pre-review checklist before submitting the PR to minimize back-and-forth iterations. + +**Quality Checks**: + +- [x] Pre-commit checks pass: `./scripts/pre-commit.sh` + +**Task-Specific Criteria**: + +- [x] BEP 34 TXT records are present and correct for both `http1` and `udp1` subdomains + (verified with `dig TXT`) +- [x] Two new floating IPs are provisioned in Hetzner and assigned to the server +- [x] All four floating IPs (existing + new) are configured permanently via netplan +- [x] `udp1.torrust-tracker-demo.com` resolves to the new IPs (A + AAAA records) +- [x] `udp://udp1.torrust-tracker-demo.com:6969/announce` appears in the newTrackon public list +- [x] `docs/deployments/hetzner-demo-tracker/post-provision/newtrackon-prerequisites.md` documents + the prerequisites clearly +- [x] `tracker-registry.md` is updated with the correct submission status + +## Related Documentation + +- [BEP 34 — DNS Tracker Preferences](https://www.bittorrent.org/beps/bep_0034.html) +- [newTrackon](https://newtrackon.com/) +- [Hetzner Demo Tracker — Deployment Journal](../deployments/hetzner-demo-tracker/README.md) +- [Hetzner Demo Tracker — Tracker Registry](../deployments/hetzner-demo-tracker/tracker-registry.md) +- [Hetzner Demo Tracker — DNS Setup](../deployments/hetzner-demo-tracker/post-provision/dns-setup.md) +- [Issue #405 — Deploy Hetzner Demo Tracker](405-deploy-hetzner-demo-tracker-and-document-process.md) + +## Notes + +The HTTP1 tracker (`https://http1.torrust-tracker-demo.com/announce`) was successfully submitted and +accepted by newTrackon on 2026-03-04. The newTrackon API can be used to verify current status: +`curl https://newtrackon.com/api/stable`. + +The UDP2 tracker (`udp://udp2.torrust-tracker-demo.com:6868/announce`) is intentionally **not** +submitted to any public registry — it is reserved as a low-traffic endpoint for manual testing +and debugging. diff --git a/project-words.txt b/project-words.txt index fd4f71f3..84ea7d91 100644 --- a/project-words.txt +++ b/project-words.txt @@ -333,6 +333,8 @@ netfilter netplan networkd newgrp +macaddress +newtrackon newtype newtypes nextest @@ -504,9 +506,15 @@ undertested unergonomic unittests unrepresentable +DNAT +UNCONN +flowlabel +hlim +tcpdump unsubscription userexample usermod +ulnp useroutput userpass userspace