Skip to content

Commit 23ae5b1

Browse files
davidallendjdavidallendj
committed
docs: added info about cloud-init wireguard
Signed-off-by: David Allen <davidallendj@gmail.com>
1 parent 272f8be commit 23ae5b1

1 file changed

Lines changed: 233 additions & 0 deletions

File tree

‎content/docs/software/cloud-init/wireguard.md‎

Lines changed: 233 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,233 @@
1+
---
2+
title: "WireGuard Integration"
3+
description: "Secure node bootstrapping with WireGuard tunnels for cloud-init traffic."
4+
date: 2026-06-29T00:00:00+00:00
5+
lastmod: 2026-06-29T00:00:00+00:00
6+
draft: false
7+
weight: 500
8+
toc: true
9+
categories: ["Cloud-Init", "WireGuard", "Security"]
10+
tags: ["Cloud-Init", "WireGuard", "VPN", "Security", "Networking"]
11+
---
12+
13+
## Overview
14+
15+
OpenCHAMI's cloud-init server supports **WireGuard** to encrypt and isolate cloud-init traffic between compute nodes and the cloud-init server. This ensures that sensitive metadata, user data, and vendor data exchanged during node provisioning are transmitted over an encrypted tunnel.
16+
17+
The cloud-init server embeds a WireGuard server that dynamically assigns VPN IPs to compute nodes, manages peer configurations, and can optionally restrict cloud-init data access to only those nodes connected via the WireGuard tunnel.
18+
19+
## Architecture
20+
21+
The WireGuard integration works in two phases:
22+
23+
1. **Tunnel establishment** — At boot, the compute node calls the unprotected `/cloud-init/wg-init` endpoint to request a WireGuard IP and peer configuration. This happens before cloud-init itself runs.
24+
2. **Secure data fetch** — Once the tunnel is up, cloud-init fetches `/meta-data`, `/user-data`, `/vendor-data`, and group YAML files over the encrypted WireGuard link.
25+
26+
```text
27+
┌──────────────┐ WireGuard Tunnel ┌──────────────────┐
28+
│ Compute │ ◄────────────────────────► │ cloud-init │
29+
│ Node │ wg0: 100.97.0.x/32 │ Server │
30+
│ │ │ wg0: 100.97.0.1 │
31+
└──────────────┘ └──────────────────┘
32+
│ │
33+
│ 1. POST /cloud-init/wg-init │
34+
│ (before cloud-init starts) │
35+
│ │
36+
│ 2. GET /meta-data, /vendor-data │
37+
│ (over WireGuard tunnel) │
38+
└───────────────────────────────────────────┘
39+
```
40+
41+
## Server-Side Configuration
42+
43+
### Environment Variables
44+
45+
| Variable | Description | Default |
46+
| --- | --- | --- |
47+
| `WIREGUARD_SERVER` | WireGuard server IP and network CIDR (e.g. `100.97.0.1/16`) | (none) |
48+
| `WIREGUARD_ONLY` | Only serve cloud-init data to clients in the WireGuard subnet | `false` |
49+
| `DEBUG` | Enable debug logging | `false` |
50+
51+
### WIREGUARD_ONLY Behavior
52+
53+
When `WIREGUARD_ONLY=true` is set, the cloud-init server restricts access to data-bearing endpoints (`/meta-data`, `/user-data`, `/vendor-data`, `/{group}.yaml`) to clients whose IP falls within the WireGuard subnet **or** whose request arrives on the server's WireGuard interface.
54+
55+
{{< callout context="note" title="Important" icon="info-circle" >}}
56+
The `/cloud-init/wg-init` and `/cloud-init/phone-home/{id}` endpoints are **never** restricted by the WireGuard middleware, even when `WIREGUARD_ONLY=true`. This allows nodes to establish their WireGuard tunnel before fetching cloud-init data.
57+
{{< /callout >}}
58+
59+
### Network Setup
60+
61+
The cloud-init container needs access to both the internal cluster network and an external network for WireGuard traffic. Configure this using Podman quadlet overrides.
62+
63+
{{< details "Example quadlet override (10-override.conf)" >}}
64+
65+
```ini
66+
# /etc/containers/systemd/cloud-init-server.container.d/10-override.conf
67+
[Container]
68+
Image=ghcr.io/openchami/cloud-init:v1.4.2
69+
Environment=WIREGUARD_SERVER=172.16.1.1/24
70+
Environment=WIREGUARD_ONLY=true
71+
Environment=DEBUG=true
72+
AddCapability=NET_ADMIN
73+
AddDevice=/dev/net/tun
74+
PublishPort=58036:58036/udp
75+
Network=openchami-internal.network:alias=cloud-init-int
76+
Network=openchami-external.network:alias=cloud-init-wg
77+
```
78+
79+
{{< /details >}}
80+
81+
### HAProxy Configuration
82+
83+
If using HAProxy in front of the cloud-init server, ensure the backend routes traffic to the internal network alias:
84+
85+
```haproxy
86+
backend cloud-init
87+
server cloud-init-server cloud-init-int:27777
88+
```
89+
90+
## Client-Side Configuration
91+
92+
Compute nodes require two pieces of setup to use WireGuard for cloud-init:
93+
94+
1. A **systemd override** that runs a pre-exec script before cloud-init starts.
95+
2. A **shell script** that configures the WireGuard tunnel and requests a VPN IP from the server.
96+
97+
### Systemd Override
98+
99+
```ini
100+
# /etc/systemd/system/cloud-init.service.d/override.conf
101+
[Service]
102+
Environment=ochami_wg_ip=172.16.0.254
103+
ExecStartPre=/usr/local/bin/ochami-ci-setup.sh
104+
ExecStopPost=/bin/bash -c "ip link delete wg0"
105+
```
106+
107+
The `ochami_wg_ip` variable should point to the cloud-init server's external IP address (the one accessible from the compute node before the tunnel is established).
108+
109+
### WireGuard Setup Script
110+
111+
The pre-exec script performs the following steps:
112+
113+
1. Generate a WireGuard key pair on the compute node.
114+
2. Request a VPN IP from the cloud-init server via the `/cloud-init/wg-init` endpoint.
115+
3. Create and configure the `wg0` interface with the assigned IP and server peer.
116+
4. Add a route to the WireGuard server IP over the tunnel.
117+
118+
{{< details "Example ochami-ci-setup.sh" >}}
119+
120+
```bash
121+
#!/bin/sh
122+
set -e -o pipefail
123+
124+
if [ -z "${ochami_wg_ip}" ]; then
125+
echo "ERROR: Failed to find the 'ochami_wg_ip' environment variable."
126+
if [ -f "/etc/cloud/cloud.cfg.d/ochami.cfg" ]; then
127+
echo "Removing ochami-specific cloud-config; cloud-init will use other defaults"
128+
rm /etc/cloud/cloud.cfg.d/ochami.cfg
129+
else
130+
echo "Not writing ochami-specific cloud-config; cloud-init will use other defaults"
131+
fi
132+
exit 0
133+
fi
134+
135+
echo "Loading WireGuard kernel module"
136+
modprobe wireguard
137+
138+
echo "Generating WireGuard keys"
139+
wg genkey | tee /etc/wireguard/private.key | wg pubkey > /etc/wireguard/public.key
140+
141+
echo "Requesting WireGuard tunnel configuration"
142+
PUBLIC_KEY=$(cat /etc/wireguard/public.key)
143+
PAYLOAD="{ \"public_key\": \"${PUBLIC_KEY}\" }"
144+
WG_PAYLOAD=$(curl -s -X POST -d "${PAYLOAD}" http://${ochami_wg_ip}:8081/cloud-init/wg-init)
145+
146+
CLIENT_IP=$(echo $WG_PAYLOAD | jq -r '."client-vpn-ip"')
147+
SERVER_IP=$(echo $WG_PAYLOAD | jq -r '."server-ip"' | awk -F'/' '{print $1}')
148+
SERVER_PORT=$(echo $WG_PAYLOAD | jq -r '."server-port"')
149+
SERVER_KEY=$(echo $WG_PAYLOAD | jq -r '."server-public-key"')
150+
151+
echo "Setting up WireGuard interface"
152+
ip link add dev wg0 type wireguard
153+
ip address add dev wg0 ${CLIENT_IP}/32
154+
wg set wg0 private-key /etc/wireguard/private.key
155+
ip link set wg0 up
156+
wg set wg0 peer ${SERVER_KEY} allowed-ips ${SERVER_IP}/32 endpoint ${ochami_wg_ip}:${SERVER_PORT}
157+
158+
echo "Adding route to WireGuard server"
159+
ip route replace "${SERVER_IP}/32" dev wg0
160+
161+
rm /etc/wireguard/private.key
162+
rm /etc/wireguard/public.key
163+
```
164+
165+
{{< /details >}}
166+
167+
### Image Building
168+
169+
Inject these files into the compute node image using the OpenCHAMI image builder. Add them under `copyfiles` in the image build configuration:
170+
171+
```yaml
172+
copyfiles:
173+
- src: '/opt/workdir/images/files/cloud-init-override.conf'
174+
dest: '/etc/systemd/system/cloud-init.service.d/override.conf'
175+
- src: '/opt/workdir/images/files/ochami-ci-setup.sh'
176+
dest: '/usr/local/bin/ochami-ci-setup.sh'
177+
```
178+
179+
The image must also include the `wireguard-tools` package:
180+
181+
```yaml
182+
packages:
183+
- wireguard-tools
184+
```
185+
186+
### Boot Parameters
187+
188+
Configure the node's boot parameters in BSS to point to the cloud-init server using the WireGuard subnet address:
189+
190+
```yaml
191+
params: 'nomodeset ro root=live:http://172.16.0.254:7070/boot-images/... ip=dhcp ... cloud-init=enabled ds=nocloud-net;s=http://172.16.1.1:27777/cloud-init'
192+
```
193+
194+
The `ds=nocloud-net` URL should use the WireGuard server IP (e.g. `172.16.1.1:27777`), which is the address the compute node will reach once the tunnel is established.
195+
196+
## How the Middleware Works
197+
198+
The WireGuard middleware enforces access policy based on two criteria:
199+
200+
- **Client IP in WireGuard subnet** — If the connecting client's IP falls within the configured WireGuard CIDR (e.g. `100.97.0.0/16`), the request is allowed.
201+
- **Request arrived on WireGuard interface** — If the connection was received on the server's `wg0` interface, the request is allowed regardless of client IP.
202+
203+
Before the fix in PR [#113](https://github.com/OpenCHAMI/cloud-init/pull/113), the WireGuard middleware was applied globally to **all** routes when `WIREGUARD_ONLY=true`. This created a chicken-and-egg problem: compute nodes needed to call `/cloud-init/wg-init` to establish their WireGuard tunnel, but that endpoint was blocked by the middleware.
204+
205+
The fix scoped the middleware to only the data-bearing routes (`/meta-data`, `/user-data`, `/vendor-data`, `/{group}.yaml`), leaving the tunnel setup endpoint (`/wg-init`) and phone-home endpoint (`/phone-home/{id}`) accessible without restriction.
206+
207+
## Verification
208+
209+
To verify the WireGuard tunnel is functioning correctly from a compute node:
210+
211+
```bash
212+
# Check the WireGuard interface
213+
wg show
214+
215+
# Verify the tunnel is resolving the cloud-init server
216+
ping 100.97.0.1
217+
```
218+
219+
On the cloud-init server, check that the WireGuard interface is active:
220+
221+
```bash
222+
# List WireGuard peers
223+
wg show wg0
224+
225+
# Check debug logs for connection metadata
226+
journalctl -u cloud-init-server
227+
```
228+
229+
If `WIREGUARD_ONLY=true` is set and a non-WireGuard client tries to access a protected endpoint, they will receive:
230+
231+
```text
232+
Access denied: IP 10.89.2.63 not in WireGuard subnet or interface
233+
```

0 commit comments

Comments
 (0)