Skip to content

Commit 95054be

Browse files
teovlteovl
committed
Release: v1.9.0 notes + macOS ad-hoc codesign in workflow
Add ad-hoc codesign + xattr -cr step on darwin builds in release.yml. Without this, downloaded macOS tarballs trigger "killed: 9" or Gatekeeper "cannot be opened because Apple cannot check it for malicious software" on first run. Ad-hoc signing (no Apple Developer account, no notarization) is enough to make the binary launch. RELEASE_NOTES_v1.9.0.md: per-theme summary covering registry hardening, P1-010 closure, multi-beacon discovery, peer-IP privacy, network-member-count gate, runtime banner endpoint, pilotctl updates, async IPC, pubsub retry, and integration-suite stability fixes. CONTRIBUTING.md: lock-discipline rule.
1 parent a68e434 commit 95054be

3 files changed

Lines changed: 210 additions & 0 deletions

File tree

‎.github/workflows/release.yml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,24 @@ jobs:
7272
go build -ldflags "$LDFLAGS" -o dist/$bin ./cmd/$bin
7373
done
7474
75+
# Ad-hoc codesign + drop quarantine on darwin builds so downloaded
76+
# binaries don't trigger "killed: 9" or Gatekeeper "cannot be opened
77+
# because Apple cannot check it for malicious software". Ad-hoc
78+
# signing is enough to make the binary launch — full notarization
79+
# would need APPLE_ID + APP_SPECIFIC_PASSWORD secrets and a paid
80+
# developer cert; out of scope for this release.
81+
- name: macOS ad-hoc codesign + strip quarantine
82+
if: matrix.goos == 'darwin'
83+
run: |
84+
for bin in dist/*; do
85+
echo "codesigning $bin..."
86+
codesign --force --deep --sign - "$bin"
87+
xattr -cr "$bin" || true
88+
done
89+
for bin in dist/*; do
90+
codesign -dv "$bin" 2>&1 | grep -E "Signature|Authority|TeamIdentifier" | head -1
91+
done
92+
7593
- name: Smoke test binaries
7694
if: matrix.goos == 'linux' && matrix.goarch == 'amd64' || matrix.goos == 'darwin'
7795
run: |

‎CONTRIBUTING.md‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,32 @@ docs/ # Documentation
8181
WHITEPAPER.pdf # Protocol whitepaper (LaTeX source: WHITEPAPER.tex)
8282
```
8383

84+
## Lock discipline (required reading for registry/daemon contributors)
85+
86+
The registry holds several mutexes covering different scopes. Past
87+
contention incidents have shown that running signature-verification
88+
work — or any operation that can take longer than a few microseconds —
89+
while holding a global lock can produce contention queues large enough
90+
to drop the registry over a cliff under load.
91+
92+
The 3-phase pattern is: **RLock → unlock → verify (signatures, args,
93+
caller identity) → Lock for mutation only**. The verify phase must not
94+
hold any registry mutex.
95+
96+
Concrete rules:
97+
98+
1. **Never call `crypto`/`subtle.ConstantTimeCompare`/JSON-marshal
99+
under `s.mu`.** These can take microseconds-to-milliseconds; a
100+
queue of contended writers piles up behind them.
101+
2. **Bracket `s.mu` write windows tightly** — only the actual map
102+
mutation goes under `s.mu.Lock()`. Read inputs first under RLock,
103+
verify, then take the write lock.
104+
3. **Snapshot/replication paths build their deep-copy outside `s.mu`.**
105+
See `apply_snapshot_test.go` for the lock-hold regression test.
106+
4. **List endpoints (`list_nodes`, `list_networks`) go through the
107+
singleflight cache.** Re-marshalling JSON for every caller while
108+
`s.mu` is held was the root cause of the 2026-04-28 outage.
109+
84110
## Contributing to the Python SDK
85111

86112
See the **[Python SDK Contributing Guide](sdk/python/CONTRIBUTING.md)**.

‎RELEASE_NOTES_v1.9.0.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# v1.9.0 — Release Notes
2+
3+
**Scope:** 96 commits since `v1.8.0` (2026-04-21 → 2026-04-30).
4+
5+
This is the stable v1.9.0 cut. The headline themes are: registry
6+
lock-contention closed end-to-end, P1-010 tunnel-desync recovery,
7+
registry hardening (panic recovery, WAL replay, 3-phase
8+
rotate-key/set-key-expiry), multi-beacon discovery, and a privacy pass —
9+
peer IPs hidden by default in pilotctl output and network member counts
10+
admin-gated at the registry.
11+
12+
## What's new
13+
14+
### Registry hardening
15+
16+
- **Panic recovery** in registry handlers (`pkg/registry/panic_recovery.go`).
17+
A panicking handler no longer takes the whole server down; the panic is
18+
logged with a stack trace and the connection is closed.
19+
- **WAL replay + size cap** (`pkg/registry/wal_replay.go`,
20+
`wal_size_cap_test.go`). The write-ahead log replays cleanly on startup
21+
and self-bounds to a configured size cap so a stuck `flushSave` cannot
22+
let the WAL grow until the volume fills.
23+
- **3-phase pattern** applied to `handleRotateKey` and
24+
`handleSetKeyExpiry` — RLock for verify, unlock, Lock only for the
25+
mutation. Eliminates the lock-hold during signature checks.
26+
- **Heartbeat-ack pre-build** to keep the hot path off `json.Marshal`.
27+
- **`list_nodes` cache-invalidation hooks** wired through every membership
28+
mutation (5 per-net + 5 admin sites).
29+
- **`apply_snapshot` outside-lock build** — the registry constructs the
30+
snapshot deep-copy without holding `s.mu`, dropping the lock-hold from
31+
multi-second peaks to milliseconds.
32+
- **Snapshot lock-discipline tests** (`apply_snapshot_test.go`,
33+
`snapshot_lock_test.go`).
34+
35+
### Tunnel + dataexchange (P1-010 closure)
36+
37+
1. **Salvage replay** — per-peer plaintext ring buffer, re-encrypted with
38+
the new key after rekey. Recovers fire-and-forget RPCs sent under
39+
stale crypto during the rekey window. Webhook event:
40+
`tunnel.desync_salvage`.
41+
2. **Rekey retransmit loop** — per-peer `pendingRekey` state and a
42+
bounded retransmit goroutine (every 4 s, capped at 5 retries). Closes
43+
the case where our `key_exchange` or the peer's reply was dropped.
44+
Webhook event: `tunnel.rekey_gave_up`.
45+
3. **Salvage cap of 4 entries** — original 32 caused a replay-storm on
46+
rekey; 4 covers the realistic shape and is ~6 KiB/peer instead of
47+
~48 KiB.
48+
4. **Preserve nonce/replay state across duplicate `key_exchange`** —
49+
replace `tm.crypto[N]` only when the entry doesn't exist or the peer's
50+
pubkey actually changed. Pinned by
51+
`TestHandleKeyExchangeDuplicatePreservesCryptoState`.
52+
53+
### Multi-beacon discovery
54+
55+
`pkg/daemon/beacon_select.go`, `beacon_discovery.go`. Daemons can be
56+
configured with a comma-separated `-beacon` list and FNV-of-pubkey-hash
57+
to one of the live beacons. Foundation for autoscaling beacon meshes —
58+
per-pair tunnels pin to the same beacon for ordering.
59+
60+
### Privacy: peer IPs no longer in pilotctl output by default
61+
62+
`pilotctl peers`, `pilotctl info`, `pilotctl lookup` strip
63+
endpoint-bearing fields (`endpoint`, `real_addr`, `lan_addrs`,
64+
`public_addr`, `stun_addr`, `observed_addr`) from their output by
65+
default. Pass `--show-endpoints` to bring them back for ops/debug.
66+
Searches that match endpoint substrings only resolve when
67+
`--show-endpoints` is set, so a search prompt cannot leak IP existence.
68+
69+
Pinned by `cmd/pilotctl/redact_test.go::TestRedactPeerEndpointsRemovesIPFields`.
70+
71+
### Privacy: network member counts now admin-gated
72+
73+
`pkg/registry/server.go::handleListNetworks` omits the per-network
74+
`members` count from `list_networks` responses unless the request
75+
carries a valid `admin_token`. Network identity (id, name, join_rule,
76+
enterprise) stays visible — daemons can still discover what to join —
77+
but population per network is treated as a privacy-sensitive aggregate.
78+
Pinned by `tests/network_test.go::TestListNetworksMembersAdminGated`.
79+
80+
The pilotctl `network list` view shows `—` in the MEMBERS column when
81+
the count is hidden, so the redaction is visible-by-policy rather than
82+
mistaken for broken.
83+
84+
`pkg/registry/client.go::ListNetworks` takes a variadic `adminToken` to
85+
opt into seeing the count.
86+
87+
### Operations: runtime banner endpoint
88+
89+
The dashboard's amber maintenance notice (rendered above the release
90+
banner) is now mutable at runtime via an admin-only HTTP endpoint —
91+
`PUT /api/banner` on the rendezvous's HTTP port. Persisted to disk,
92+
survives restart, no reconnect storm. Both `GET` and `PUT` require the
93+
admin token; the public still sees the banner via
94+
`/api/stats.maintenance_banner` and the dashboard HTML.
95+
96+
Pinned by `tests/dashboard_test.go::TestDashboardBannerEndpoint` —
97+
8 cases (no-token GET/PUT, wrong-token, valid round-trip,
98+
`/api/stats` surfaces banner, persistence file written, empty-body
99+
clears, 405 on DELETE).
100+
101+
### `pilotctl updates`
102+
103+
New command to fetch the public changelog feed:
104+
105+
```
106+
pilotctl updates [--count N] [--scope <category>] [--refresh]
107+
pilotctl --json updates
108+
```
109+
110+
Reads `https://teoslayer.github.io/pilot-changelog/feed.xml`, caches at
111+
`~/.pilot/updates-cache.xml` for 5 min, falls back to stale cache when
112+
offline. No external deps — RSS parsed with `encoding/xml`.
113+
114+
Pinned by 11 tests in `cmd/pilotctl/updates_test.go` (cold fetch, cache
115+
hit, `--refresh` bypass, stale-cache refetch, offline fallback,
116+
no-fallback-without-cache, scope filter, count truncation, combined).
117+
118+
### Daemon stability
119+
120+
- **Async IPC write queue** — buffered `sendCh` per ipcConn + single
121+
writer goroutine, done-channel pattern, `ErrIPCBackpressure` on slow
122+
clients. Eliminates the IPC-write-mutex contention bottleneck observed
123+
under stress. Pinned by `pkg/daemon/ipc_async_write_test.go`.
124+
- **Pubsub retry resilience** — failed publish attempts retry once
125+
before raising; per-peer failure counter exposed via webhook.
126+
`pkg/daemon/pubsub_resilience_test.go`.
127+
- **Lock-ordering invariants** documented in `CONTRIBUTING.md`.
128+
129+
### Integration suite
130+
131+
- **Webhook readiness helper** — `webhook_helpers.sh` waits for the
132+
webhook sink to be HTTP-bound before bringing up agents.
133+
- **NAT IPAM retry** — `test_nat_dual_symmetric.sh` and the shared
134+
`nat_test_common.sh` sweep stale Docker networks on `pool overlaps`
135+
and retry the compose-up.
136+
- **Bench harness** — `bench_relay.sh` + `docker-compose.multi.bench.yml`
137+
characterize relay throughput without the chaos overlay's tmpfs cap
138+
used by disk-full tests.
139+
- **Lane bump 3 → 5** for parallel NAT tests.
140+
141+
### Release engineering
142+
143+
- **macOS ad-hoc codesign** in `.github/workflows/release.yml`. darwin
144+
builds get `codesign --force --deep --sign -` plus `xattr -cr` before
145+
packaging. Without this, downloaded macOS tarballs trigger
146+
`killed: 9` or Gatekeeper "cannot be opened because Apple cannot check
147+
it for malicious software" on first run. Ad-hoc signing is enough to
148+
let the binary launch — full notarization would need an Apple
149+
Developer ID and is out of scope for this release.
150+
151+
## Migration notes
152+
153+
- **External SDKs / dashboards** that call `list_networks` and read
154+
`members` need to either pass `admin_token` in the request or handle
155+
the field being absent. Pre-v1.9.0 registries keep returning the
156+
count to anyone; after upgrading, they don't.
157+
- **Scripts / tooling** that grepped pilotctl output for IP fragments
158+
need `--show-endpoints` (or, preferably, switch to node IDs).
159+
160+
## Acceptance evidence
161+
162+
- `go vet ./...` clean.
163+
- `go test ./...` — 15/15 packages pass.
164+
- `go test ./... -race -count=1 -timeout 600s` — clean, zero data races.
165+
- 8 binaries cross-build clean for `linux/{amd64,arm64}` and
166+
`darwin/{amd64,arm64}` with `-X main.version=v1.9.0`.

0 commit comments

Comments
 (0)