|
| 1 | +# Factory WireGuard Server: Custom Code Status |
| 2 | + |
| 3 | +## Current Status |
| 4 | + |
| 5 | +**The Proxmox server is running CUSTOM CODE**, not the upstream Foundries code. |
| 6 | + |
| 7 | +## Why Custom Code? |
| 8 | + |
| 9 | +We've made **three critical modifications** that are not in the upstream `foundriesio/factory-wireguard-server` repository: |
| 10 | + |
| 11 | +### 1. Device-to-Device Communication (`--allow-device-to-device`) |
| 12 | + |
| 13 | +**Problem:** By default, Foundries devices can only communicate with the VPN server (`10.42.42.1`), not with each other or client machines. |
| 14 | + |
| 15 | +**Solution:** Added `--allow-device-to-device` flag that: |
| 16 | +- Sets `AllowedIPs = 10.42.42.0/24` for all device peers (instead of `/32`) |
| 17 | +- Enables full subnet communication for development/debugging |
| 18 | + |
| 19 | +**Code Changes:** |
| 20 | +- Added `allow_device_to_device` parameter to `WgServer.__init__()` |
| 21 | +- Modified `_gen_conf()` to use subnet `AllowedIPs` when flag is set |
| 22 | +- Modified `apply_conf()` to remove peers before applying config (ensures `AllowedIPs` persist) |
| 23 | + |
| 24 | +**Status:** ✅ Implemented locally, needs to be deployed to server |
| 25 | + |
| 26 | +### 2. Client Peer Management (`load_client_peers`, `apply_client_peers`) |
| 27 | + |
| 28 | +**Problem:** Client peers (engineers' machines) are NOT automatically managed by the daemon. They must be manually added using `wg set` commands, and they don't persist across daemon restarts. |
| 29 | + |
| 30 | +**Solution:** Added config file-based client peer management: |
| 31 | +- Reads client peers from `/etc/wireguard/factory-clients.conf` |
| 32 | +- Applies client peers on daemon startup and after config changes |
| 33 | +- Client peers persist across restarts |
| 34 | + |
| 35 | +**Code Changes:** |
| 36 | +- Added `load_client_peers()` method |
| 37 | +- Added `apply_client_peers()` method |
| 38 | +- Integrated into `apply_conf()` and `daemon()` startup |
| 39 | + |
| 40 | +**Config File Format:** |
| 41 | +``` |
| 42 | +# /etc/wireguard/factory-clients.conf |
| 43 | +# Format: <public_key> <assigned_ip> [comment] |
| 44 | +mzHaZPGowqqzAa5tVFQJs0zoWuDVLppt44HwgdcPXkg= 10.42.42.10 ajlennon |
| 45 | +7WR3aejgU53i+/MiJcpdboASPgjLihXApnhHj4SRukE= 10.42.42.11 engineer2 |
| 46 | +``` |
| 47 | + |
| 48 | +**Status:** ✅ Implemented locally, needs to be deployed to server |
| 49 | + |
| 50 | +### 3. AllowedIPs Persistence Fix |
| 51 | + |
| 52 | +**Problem:** WireGuard clears `AllowedIPs` when peers reconnect with active endpoints, causing device-to-device communication to break. |
| 53 | + |
| 54 | +**Solution:** Modified `apply_conf()` to: |
| 55 | +- Remove existing device peers before applying config |
| 56 | +- Wait 10 seconds for peers to fully disconnect |
| 57 | +- Apply config (which adds peers without endpoints initially) |
| 58 | +- Explicitly set `AllowedIPs` using `wg set` after interface is up |
| 59 | + |
| 60 | +**Status:** ✅ Implemented locally, needs to be deployed to server |
| 61 | + |
| 62 | +## Repository Structure |
| 63 | + |
| 64 | +The `factory-wireguard-server` directory is a **git submodule** with three remotes: |
| 65 | + |
| 66 | +``` |
| 67 | +dynamicdevices -> git@github.com:DynamicDevices/factory-wireguard-server.git (fork) |
| 68 | +origin -> git@github.com:ajlennon/factory-wireguard-server.git (personal fork) |
| 69 | +upstream -> git@github.com:foundriesio/factory-wireguard-server.git (upstream) |
| 70 | +``` |
| 71 | + |
| 72 | +**Current State:** |
| 73 | +- Local code has custom modifications |
| 74 | +- DynamicDevices fork may or may not have these changes |
| 75 | +- Upstream does NOT have these changes |
| 76 | + |
| 77 | +## What's Running on Proxmox Server? |
| 78 | + |
| 79 | +**Unknown** - Need to check what version is actually running on the server. |
| 80 | + |
| 81 | +To check: |
| 82 | +```bash |
| 83 | +# SSH to server via Foundries VPN |
| 84 | +ssh root@10.42.42.1 -p 5025 |
| 85 | + |
| 86 | +# Check daemon code |
| 87 | +cat /root/factory-wireguard-server/factory-wireguard.py | grep -E "allow_device_to_device|load_client_peers|apply_client_peers" |
| 88 | + |
| 89 | +# Check daemon command line |
| 90 | +systemctl cat factory-vpn-dynamic-devices.service | grep ExecStart |
| 91 | +``` |
| 92 | + |
| 93 | +## Deployment Status |
| 94 | + |
| 95 | +**Local Code:** ✅ All custom modifications implemented |
| 96 | +**Server Code:** ❓ Unknown - needs verification and deployment |
| 97 | + |
| 98 | +## Should We Contribute Back to Upstream? |
| 99 | + |
| 100 | +### Option 1: Contribute to Upstream (Recommended) |
| 101 | + |
| 102 | +**Pros:** |
| 103 | +- Benefits entire Foundries community |
| 104 | +- Easier maintenance (upstream updates) |
| 105 | +- Standardized solution |
| 106 | + |
| 107 | +**Cons:** |
| 108 | +- Requires PR review/approval |
| 109 | +- May take time to merge |
| 110 | +- May need to justify use cases |
| 111 | + |
| 112 | +**Changes to Contribute:** |
| 113 | +1. ✅ `--allow-device-to-device` flag (useful for development) |
| 114 | +2. ✅ Client peer management (useful for all factories) |
| 115 | +3. ✅ AllowedIPs persistence fix (bug fix) |
| 116 | + |
| 117 | +### Option 2: Maintain Custom Fork |
| 118 | + |
| 119 | +**Pros:** |
| 120 | +- Full control |
| 121 | +- No upstream dependencies |
| 122 | +- Can customize freely |
| 123 | + |
| 124 | +**Cons:** |
| 125 | +- Must maintain fork |
| 126 | +- Must manually merge upstream updates |
| 127 | +- Duplicate effort |
| 128 | + |
| 129 | +**Recommendation:** **Contribute to upstream** - These are useful features that benefit everyone. |
| 130 | + |
| 131 | +## Next Steps |
| 132 | + |
| 133 | +1. **Verify Server Code:** |
| 134 | + - Check what's actually running on Proxmox server |
| 135 | + - Compare with local custom code |
| 136 | + |
| 137 | +2. **Deploy Custom Code:** |
| 138 | + - Copy updated `factory-wireguard.py` to server |
| 139 | + - Restart daemon |
| 140 | + - Test functionality |
| 141 | + |
| 142 | +3. **Decide on Upstream Contribution:** |
| 143 | + - Create PR for `--allow-device-to-device` flag |
| 144 | + - Create PR for client peer management |
| 145 | + - Create PR for AllowedIPs persistence fix |
| 146 | + |
| 147 | +4. **Document Custom Changes:** |
| 148 | + - Keep this document updated |
| 149 | + - Document any additional customizations |
| 150 | + |
| 151 | +## Summary |
| 152 | + |
| 153 | +**Why Custom Code?** |
| 154 | +- Upstream doesn't support device-to-device communication |
| 155 | +- Upstream doesn't manage client peers |
| 156 | +- Upstream has AllowedIPs persistence issues |
| 157 | + |
| 158 | +**What We've Added:** |
| 159 | +- Device-to-device communication flag |
| 160 | +- Client peer config file management |
| 161 | +- AllowedIPs persistence fixes |
| 162 | + |
| 163 | +**Status:** |
| 164 | +- ✅ Code implemented locally |
| 165 | +- ❓ Server status unknown (needs check) |
| 166 | +- ⏳ Deployment pending |
| 167 | +- ⏳ Upstream contribution pending |
| 168 | + |
0 commit comments