Skip to content

Commit fa73300

Browse files
josecelanojosecelano
committed
docs: [#414] post-deployment manual steps for floating IPs and IPv6
1 parent 588fa47 commit fa73300

4 files changed

Lines changed: 235 additions & 0 deletions

File tree

‎docs/user-guide/providers/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -39,3 +39,4 @@ See the [contributing guide](../../contributing/README.md) for more details.
3939
- [Quick Start Guides](../quick-start/README.md) - Docker and native installation guides
4040
- [Commands Reference](../commands/README.md) - Available commands
4141
- [SSH Keys](../../tech-stack/ssh-keys.md) - SSH key generation and management
42+
- [Hetzner Post-Deployment](hetzner-post-deployment.md) - Manual steps for floating IPs and IPv6

‎docs/user-guide/providers/hetzner-post-deployment.md‎

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
# Hetzner Post-Deployment: Floating IPs and IPv6
2+
3+
This guide documents the manual steps required **after** running the deployer when the server uses
4+
**Hetzner floating IPs** and/or needs **IPv6 UDP tracker support**.
5+
6+
These steps are not planned to be automated by the deployer. They are specific to multi-IP setups
7+
where separate floating IPs are used for separate tracker endpoints (e.g. one IP for the HTTP
8+
tracker, one for the UDP tracker) so that both can be listed independently on
9+
[newTrackon](https://newtrackon.com/), which tracks one tracker per IP.
10+
11+
The reference implementation is
12+
[torrust/torrust-tracker-demo](https://github.com/torrust/torrust-tracker-demo), which uses this
13+
setup with two floating IPs:
14+
15+
- HTTP tracker: `http1.torrust-tracker-demo.com` → `116.202.176.169` / `2a01:4f8:1c0c:9aae::1`
16+
- UDP tracker: `udp1.torrust-tracker-demo.com` → `116.202.177.184` / `2a01:4f8:1c0c:828e::1`
17+
18+
The full incident investigation that led to this documentation is in
19+
[torrust-tracker-demo#2](https://github.com/torrust/torrust-tracker-demo/issues/2).
20+
21+
## Which Steps Are Needed for Which Scenario
22+
23+
| Scenario | Step 1 | Step 2 | Step 3 | Step 4 |
24+
| --------------------------------- | ------ | ------ | ------ | ------ |
25+
| Floating IPv4 only | ✅ | — | — | — |
26+
| IPv6 UDP, primary IP only | — | ✅ | ✅ | — |
27+
| IPv6 UDP, floating IP | — | ✅ | ✅ | ✅ |
28+
| Floating IPv4 + IPv6 UDP floating | ✅ | ✅ | ✅ | ✅ |
29+
30+
---
31+
32+
## Why Floating IPs Require Manual Steps
33+
34+
The deployer configures the tracker to listen on the server's **primary public IP** only. When
35+
traffic arrives on a **Hetzner floating IP**, the kernel's default routing uses the primary IP as
36+
the reply source. The client then receives a reply from a different address than it sent to and
37+
treats it as a timeout (asymmetric routing).
38+
39+
This applies to **both IPv4 and IPv6** floating IPs.
40+
41+
---
42+
43+
## Step 1 — Floating IP Policy Routing
44+
45+
> **Required for**: each floating IP (IPv4 or IPv6)
46+
47+
For each floating IP, add a policy routing rule so that packets arriving on that IP also leave via
48+
that IP.
49+
50+
On Hetzner, this means adding routing tables (e.g. `100` for IPv4, `200` for IPv6) with a default
51+
route via the floating IP gateway, then adding `ip rule` / `ip -6 rule` entries that match source
52+
addresses on those tables.
53+
54+
**Persist via netplan** in `/etc/netplan/60-floating-ip.yaml`:
55+
56+
```yaml
57+
network:
58+
version: 2
59+
renderer: networkd
60+
ethernets:
61+
eth0:
62+
addresses:
63+
- 116.202.177.184/32 # floating IPv4 (UDP1)
64+
- 2a01:4f8:1c0c:828e::1/64 # floating IPv6 (UDP1)
65+
routing-policy:
66+
- from: 116.202.177.184
67+
table: 100
68+
- from: 2a01:4f8:1c0c:828e::1
69+
table: 200
70+
routes:
71+
- to: default
72+
via: 172.31.1.1
73+
table: 100
74+
- to: default
75+
via: fe80::1
76+
table: 200
77+
```
78+
79+
Apply:
80+
81+
```bash
82+
sudo netplan apply
83+
```
84+
85+
Verify:
86+
87+
```bash
88+
ip rule list
89+
ip route show table 100
90+
ip -6 rule list
91+
ip -6 route show table 200
92+
```
93+
94+
> Repeat for every new floating IP pair. Without this, replies from floating IP endpoints leave
95+
> via the wrong source address.
96+
97+
---
98+
99+
## Step 2 — Enable Docker ip6tables Management
100+
101+
> **Required for**: IPv6 UDP tracker
102+
103+
By default, Docker has `ip6tables: false`. This means:
104+
105+
- Docker does not insert ip6tables rules for published ports (unlike IPv4 where it does this
106+
automatically via iptables).
107+
- Every time Docker starts or restarts a container, it rewrites its own chain tables. This flush
108+
wipes ufw's live ip6tables rules from the kernel. ufw does not automatically reload after this,
109+
so IPv6 UDP traffic is silently dropped after every container restart.
110+
111+
**Fix**: create `/etc/docker/daemon.json`:
112+
113+
```json
114+
{
115+
"ip6tables": true
116+
}
117+
```
118+
119+
Apply:
120+
121+
```bash
122+
sudo systemctl restart docker
123+
```
124+
125+
Verify:
126+
127+
```bash
128+
sudo ip6tables -L ufw6-user-input -n
129+
# Must show: ACCEPT 17 -- ::/0 ::/0 udp dpt:6969
130+
```
131+
132+
---
133+
134+
## Step 3 — Enable IPv6 on the Docker Bridge Network
135+
136+
> **Required for**: IPv6 UDP tracker
137+
138+
Even with `ip6tables: true`, native IPv6 UDP still fails. Docker spawns `docker-proxy` processes
139+
for each published port. For IPv6, docker-proxy receives packets on an `::` socket but the
140+
container only has an IPv4 address — docker-proxy cannot relay across address families and silently
141+
drops all native IPv6 UDP.
142+
143+
**Fix**: add `enable_ipv6: true` and a ULA subnet to the bridge network in `docker-compose.yml`:
144+
145+
```yaml
146+
proxy_network:
147+
driver: bridge
148+
enable_ipv6: true
149+
ipam:
150+
config:
151+
- subnet: "fd01:db8:1::/64"
152+
```
153+
154+
With an IPv6 address on the container, Docker creates ip6tables DNAT rules that route native IPv6
155+
traffic directly to the container, bypassing docker-proxy entirely.
156+
157+
Apply:
158+
159+
```bash
160+
cd /opt/torrust
161+
docker compose down
162+
docker compose up -d
163+
```
164+
165+
Verify the container has an IPv6 address:
166+
167+
```bash
168+
docker inspect tracker --format '{{range .NetworkSettings.Networks}}{{.GlobalIPv6Address}} {{end}}'
169+
# Expected: fd01:db8:1::x (non-empty)
170+
```
171+
172+
Verify the DNAT rule exists:
173+
174+
```bash
175+
sudo ip6tables -t nat -L DOCKER -n -v | grep 6969
176+
# Expected: DNAT rule for dpt:6969
177+
```
178+
179+
---
180+
181+
## Step 4 — SNAT for IPv6 UDP Replies via Floating IP
182+
183+
> **Required for**: floating IPv6 UDP
184+
185+
After Step 3, the container has a ULA IPv6 address (`fd01:db8:1::x`). When it replies, Docker's
186+
MASQUERADE rule rewrites the source to the server's **primary** IPv6 address
187+
(`2a01:4f8:1c19:620b::1`). Clients that probed the **floating** IPv6 (`2a01:4f8:1c0c:828e::1`)
188+
receive a reply from the wrong address and time out.
189+
190+
**Fix**: prepend a SNAT rule to `/etc/ufw/before6.rules` **before** the existing `*filter`
191+
section:
192+
193+
```text
194+
# NAT: rewrite source of Docker UDP tracker IPv6 replies to the floating IP
195+
*nat
196+
:POSTROUTING ACCEPT [0:0]
197+
-A POSTROUTING -s fd01:db8:1::/64 -o eth0 -p udp --sport 6969 \
198+
-j SNAT --to-source 2a01:4f8:1c0c:828e::1
199+
COMMIT
200+
```
201+
202+
Apply:
203+
204+
```bash
205+
sudo ufw reload
206+
```
207+
208+
Verify:
209+
210+
```bash
211+
sudo ip6tables -t nat -L POSTROUTING -n -v | grep 6969
212+
# Expected: SNAT ... fd01:db8:1::/64 ... udp spt:6969 to:2a01:4f8:1c0c:828e::1
213+
```
214+
215+
> This rule must be in `before6.rules` (not added via `ufw` CLI) so it persists in the `*nat`
216+
> table. ufw loads this file at startup, before Docker starts. The SNAT fires before Docker's
217+
> MASQUERADE and takes precedence.
218+
>
219+
> If you change the `subnet` in `docker-compose.yml`, update the `-s` match here too.
220+
> If you add a second floating IPv6, add a second SNAT rule for its subnet/address.
221+
222+
---
223+
224+
## References
225+
226+
- [torrust/torrust-tracker-demo](https://github.com/torrust/torrust-tracker-demo) — full working configuration
227+
- [torrust-tracker-demo#2](https://github.com/torrust/torrust-tracker-demo/issues/2) — incident that produced this documentation
228+
- [torrust-tracker-demo/docs/docker-ipv6.md](https://github.com/torrust/torrust-tracker-demo/blob/main/docs/docker-ipv6.md) — detailed explanation with packet-flow diagram
229+
- [torrust-tracker-demo/docs/post-deployment.md](https://github.com/torrust/torrust-tracker-demo/blob/main/docs/post-deployment.md) — step-by-step instructions
230+
- [Hetzner Provider Guide](hetzner.md) — Hetzner Cloud configuration for the deployer
231+
- [IPv6 UDP Tracker Issue Investigation](../../../docs/deployments/hetzner-demo-tracker/post-provision/ipv6-udp-tracker-issue.md) — root-cause analysis

‎docs/user-guide/providers/hetzner.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,3 +193,4 @@ Hetzner deployments configure SSH access through two mechanisms:
193193
- [SSH Keys Guide](../../tech-stack/ssh-keys.md) - SSH key generation
194194
- [SSH Root Access Security](../../security/ssh-root-access-hetzner.md) - Disabling root access
195195
- [LXD Provider](lxd.md) - Local development alternative
196+
- [Post-Deployment: Floating IPs and IPv6](hetzner-post-deployment.md) - Manual steps for floating IPs and IPv6 UDP

‎project-words.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -523,6 +523,8 @@ usize
523523
utmp
524524
vbqajnc
525525
venv
526+
POSTROUTING
527+
SNAT
526528
venvs
527529
versionable
528530
viewmodel

0 commit comments

Comments
 (0)