Skip to content

Commit dcdfec1

Browse files
committed
release(v1.10.2): fix compat-mode dial to fresh peers
In v1.10.1, a compat-mode (WSS-only) daemon's first SYN to a fresh peer went raw through the WSS pipe because routing.WriteFrame only wrapped frames in BeaconMsgRelay after the blackhole heuristic flipped the peer to relay (3 misses, ~8s silence). For brand-new peers the unwrapped frame reached the beacon as an unknown protocol byte and was dropped silently — every outbound-initiated dial from a managed claw timed out. Add routing.Manager.forceRelay, set by TunnelManager.ConnectCompat, so every outbound write in compat mode is BeaconMsgRelay-wrapped regardless of blackhole state. Verified live against beacon.pilotprotocol.network: a fresh compat daemon (UDP-free, only TCP/443 + TCP/9000) successfully fetched the weather-agent list from list-agents. Test surface added: - pkg/daemon/routing/zz_routing_test.go::TestWriteFrame_ForceRelayWrapsFreshPeer - tests/compat/zz_real_beacon_test.go: 4 integration tests against the real beacon binary's WSS bridge - tests/zz_compat_dial_test.go: end-to-end DialConnection from a compat daemon to a UDP peer through an in-process beacon Also moves the unworkable go-test and go-coverage pre-commit hooks to the manual stage (still one command away, no longer 5-min gate on every commit). Pre-push retains the full release gauntlet. Scope (per CHANGELOG platform matrix): unblocks Render-class environments (UDP-blocked, arbitrary TCP allowed). Railway/Fly/Lambda/Cloud Run/Modal/ E2B/Daytona were already fine in UDP mode. Replit Agent / Docker AI Sandbox / locked-down Cursor sandboxes still need HTTPS_PROXY + a registry-over-WSS bridge (tracked for v1.10.3).
1 parent a4265a2 commit dcdfec1

9 files changed

Lines changed: 768 additions & 4 deletions

File tree

.pre-commit-config.yaml

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -32,17 +32,28 @@ repos:
3232
files: \.go$
3333
pass_filenames: false
3434

35+
# `go-test` and `go-coverage` are too slow for the pre-commit
36+
# stage (the tests/ package's integration suite alone is ~2 min,
37+
# `make coverage` is 5+ min). Pre-push already runs the full
38+
# release gauntlet on v* tag pushes (.git/hooks/pre-push), and
39+
# CI runs it on every PR. Leaving these here as `manual` stages
40+
# so they're still one command away locally:
41+
#
42+
# pre-commit run --hook-stage manual go-test
43+
# pre-commit run --hook-stage manual go-coverage
44+
3545
- id: go-test
36-
name: go test
37-
entry: bash -c 'cd tests && go test -v -short -timeout 30s'
46+
name: go test (manual)
47+
entry: bash -c 'go test -parallel 4 -count=1 -timeout 300s ./tests/ ./pkg/beacon/'
3848
language: system
3949
files: \.go$
4050
pass_filenames: false
51+
stages: [manual]
4152

4253
- id: go-coverage
43-
name: update coverage badge
54+
name: update coverage badge (manual)
4455
entry: make coverage
4556
language: system
4657
files: \.go$
4758
pass_filenames: false
48-
stages: [pre-commit]
59+
stages: [manual]

CHANGELOG.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,62 @@ project uses [Semantic Versioning](https://semver.org/).
77
Detailed per-release notes are on the
88
[GitHub Releases page](https://github.com/TeoSlayer/pilotprotocol/releases).
99

10+
## [1.10.2] - 2026-05-19
11+
12+
### Fixed
13+
- **Compat mode: outbound-initiated dials to fresh peers**. In v1.10.1, a
14+
compat-mode (WSS-only) daemon's first SYN to a peer went raw through the WSS
15+
pipe because `routing.WriteFrame` only wrapped frames in `BeaconMsgRelay`
16+
after the blackhole heuristic flipped the peer to relay mode (3 misses, ~8s
17+
silence). For brand-new peers, the unwrapped frame reached the beacon as an
18+
unknown protocol byte and was dropped silently — every outbound-initiated
19+
dial timed out. Managed claws (Docker on Render/Railway/Lambda, UDP-blocked
20+
corp networks) could RECEIVE traffic via the bridge but couldn't INITIATE
21+
connections to UDP-only peers. Fix: a `forceRelay` flag on `routing.Manager`
22+
set by `TunnelManager.ConnectCompat`, so every outbound write in compat mode
23+
is BeaconMsgRelay-wrapped regardless of blackhole state.
24+
25+
### Added
26+
- `tests/zz_compat_dial_test.go` — end-to-end regression test: a compat daemon
27+
dials a UDP peer's echo service through an in-process beacon and asserts the
28+
three-way handshake completes and echo data flows both ways.
29+
- `tests/compat/zz_real_beacon_test.go` — 4 integration tests exercising the
30+
production beacon binary's WSS↔UDP bridge with real `bwss.Server` +
31+
`dwss.Transport` (not the synthetic in-memory bridge used by the existing
32+
4-cell matrix).
33+
- `beacon.Server.WSSAddr()` and `WSSIsConnected(nodeID)` — exposed for tests
34+
that bind to `:0` and need to wait for post-handshake WSS registration.
35+
36+
### Platform compatibility
37+
38+
Researched egress policies of common managed-claw / agent-sandbox platforms.
39+
Verified live: a fresh compat daemon (`node_id=205787`) fetched the
40+
weather-agent list from `list-agents` over **TCP/443 (beacon WSS) + TCP/9000
41+
(registry), zero UDP**. From that, the picture is:
42+
43+
| Platform | UDP egress | TCP arbitrary port | UDP mode works | Compat mode works |
44+
|-------------------------|--------------------|----------------------|-----------------|-------------------|
45+
| Render | ❌ blocked || No | **✅ Yes** |
46+
| Railway |||||
47+
| Fly.io |||||
48+
| AWS Lambda | ✅ (port 25 blkd) ||||
49+
| Google Cloud Run | ✅ (via VPC) ||||
50+
| Vercel Functions | (ephemeral; wrong runtime for a persistent daemon) |||
51+
| Modal Sandboxes | ✅ default-allow ||||
52+
| E2B Sandboxes | ✅ (IP rules only) ||||
53+
| Daytona Sandboxes | ✅ default-allow ||||
54+
| Cursor Cloud Agents | allowlist-driven | allowlist-driven | only if allowed | only if allowed |
55+
| Replit Agent / Docker AI Sandbox | ❌ hard-blocked | ❌ hard-blocked | No | ❌ No (needs `HTTPS_PROXY`) |
56+
| Devin (Cognition) | undocumented; likely Docker-AI-class || No | likely No |
57+
58+
**v1.10.2 fully unblocks**: Render (the original Garry Tan report case).
59+
**Already worked in v1.10.0 UDP mode**: Railway, Fly.io, Lambda, Cloud Run,
60+
Modal, E2B, Daytona.
61+
**Still blocked in v1.10.2**: platforms enforcing the Docker AI Sandbox
62+
HTTP-proxy-only egress model (Replit Agent, Devin, locked-down Cursor
63+
sandboxes). These need an `HTTP_PROXY`/`HTTPS_PROXY`-honoring transport and
64+
a registry-over-WSS bridge — tracked for v1.10.3.
65+
1066
## [1.10.1] - 2026-05-19
1167

1268
### Fixed

pkg/beacon/server.go

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,27 @@ func (s *Server) WSSMetrics() bwss.Metrics {
199199
return s.wssServer.Metrics()
200200
}
201201

202+
// WSSAddr returns the actual listen address of the compat WSS
203+
// bridge. Empty string if EnableCompatWSS was never called. Used by
204+
// tests that bind to :0 and need to discover the real port.
205+
func (s *Server) WSSAddr() string {
206+
if s.wssServer == nil {
207+
return ""
208+
}
209+
return s.wssServer.Addr()
210+
}
211+
212+
// WSSIsConnected reports whether a compat-mode daemon is currently
213+
// connected via the WSS bridge for the given nodeID. False if the
214+
// bridge isn't enabled. Used by tests that need to wait for the
215+
// post-handshake registration to land.
216+
func (s *Server) WSSIsConnected(nodeID uint32) bool {
217+
if s.wssServer == nil {
218+
return false
219+
}
220+
return s.wssServer.IsConnected(nodeID)
221+
}
222+
202223
// CloseCompatWSS shuts down the WSS bridge. Idempotent. Used by
203224
// graceful shutdown paths and tests.
204225
func (s *Server) CloseCompatWSS() error {

pkg/daemon/routing/routing.go

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,28 @@ type Manager struct {
132132

133133
// localNodeIDFn supplies our own node ID for relay-wrapping headers.
134134
localNodeIDFn LocalNodeIDFn
135+
136+
// forceRelay, when true, makes WriteFrame always wrap outbound
137+
// frames in BeaconMsgRelay regardless of blackhole heuristics or
138+
// pinning. Set by TunnelManager.ConnectCompat for compat-mode
139+
// daemons whose only path to peers is the beacon's WSS bridge.
140+
// Without this, fresh peers receive raw L2 frames through the WSS
141+
// pipe; the beacon's handlePacket sees no BeaconMsg prefix and
142+
// drops them silently — outbound-initiated handshakes never
143+
// complete and the daemon looks dead from the peer's side.
144+
forceRelay atomic.Bool
145+
}
146+
147+
// SetForceRelay toggles the every-send-must-be-relayed flag. Called
148+
// by TunnelManager.ConnectCompat. Idempotent.
149+
func (m *Manager) SetForceRelay(v bool) {
150+
m.forceRelay.Store(v)
151+
}
152+
153+
// ForceRelay reports the current forceRelay flag. Exposed for tests
154+
// and the dashboard.
155+
func (m *Manager) ForceRelay() bool {
156+
return m.forceRelay.Load()
135157
}
136158

137159
// New returns a fresh Manager with empty state. The Socket may be nil

pkg/daemon/routing/writeframe.go

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,15 @@ func (m *Manager) WriteFrame(nodeID uint32, addr *net.UDPAddr, frame []byte, cou
3131
_ = silentFor
3232
_ = misses
3333

34+
// Compat-mode daemons have no usable direct UDP path — the only
35+
// way for an outbound packet to reach a peer is wrapped in
36+
// BeaconMsgRelay so the beacon's relayWorker can route it (over
37+
// UDP if the peer is a regular UDP node, over WSS if the peer
38+
// is itself in compat mode).
39+
if m.forceRelay.Load() {
40+
shouldRelay = true
41+
}
42+
3443
m.mu.RLock()
3544
bAddr := m.beaconAddr
3645
sock := m.sock

pkg/daemon/routing/zz_routing_test.go

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -599,6 +599,68 @@ func TestWriteFrame_RelayWhenPinned(t *testing.T) {
599599
}
600600
}
601601

602+
// TestWriteFrame_ForceRelayWrapsFreshPeer pins compat-mode behavior:
603+
// when forceRelay is set, every outbound — even to a fresh peer with
604+
// no prior blackhole observations — must be wrapped in BeaconMsgRelay
605+
// and addressed to the beacon. Regression test for the v1.10.1 bug
606+
// where managed-claw daemons silently failed to talk to UDP-only
607+
// peers because the first send went raw through the WSS pipe and the
608+
// beacon dropped it as an unknown protocol packet.
609+
func TestWriteFrame_ForceRelayWrapsFreshPeer(t *testing.T) {
610+
t.Parallel()
611+
m := routing.New()
612+
const ourID uint32 = 0xC0FFEE01
613+
const peerID uint32 = 0xBADCAFE0
614+
m.SetLocalNodeIDFn(func() uint32 { return ourID })
615+
beacon := &net.UDPAddr{IP: net.ParseIP("198.51.100.20"), Port: 9001}
616+
m.SetBeaconAddrUDP(beacon)
617+
sock := &fakeSocket{}
618+
m.SetSocket(sock)
619+
620+
// Critical: peer is FRESH — no SetRelayPeer / SetRelayPeerPinned /
621+
// blackhole flips. Only forceRelay is on.
622+
m.SetForceRelay(true)
623+
if !m.ForceRelay() {
624+
t.Fatal("ForceRelay() returned false after SetForceRelay(true)")
625+
}
626+
627+
// addr is nil — caller doesn't even know the peer's direct UDP
628+
// endpoint yet. This mirrors the real compat-mode caller, where
629+
// peers maps only get populated after the first relay reply.
630+
frame := []byte{0xDE, 0xAD, 0xBE, 0xEF, 0xCA, 0xFE}
631+
out, err := m.WriteFrame(peerID, nil, frame, routing.CounterTarget{})
632+
if err != nil {
633+
t.Fatalf("WriteFrame with forceRelay should not error on fresh peer: %v", err)
634+
}
635+
if !out.WasRelay {
636+
t.Fatal("expected relay-wrapped send under forceRelay, got direct")
637+
}
638+
639+
got := sock.snapshot()
640+
if len(got) != 1 {
641+
t.Fatalf("expected 1 send, got %d", len(got))
642+
}
643+
if !addrsEqual(got[0].dst, beacon) {
644+
t.Fatalf("dst: got %v want beacon %v", got[0].dst, beacon)
645+
}
646+
wire := got[0].frame
647+
if len(wire) != 1+4+4+len(frame) {
648+
t.Fatalf("wire len=%d want %d", len(wire), 1+4+4+len(frame))
649+
}
650+
if wire[0] != protocol.BeaconMsgRelay {
651+
t.Fatalf("wire[0]=0x%02x want 0x%02x (BeaconMsgRelay)", wire[0], protocol.BeaconMsgRelay)
652+
}
653+
if got := binary.BigEndian.Uint32(wire[1:5]); got != ourID {
654+
t.Fatalf("sender nodeID=0x%08x want 0x%08x", got, ourID)
655+
}
656+
if got := binary.BigEndian.Uint32(wire[5:9]); got != peerID {
657+
t.Fatalf("dst nodeID=0x%08x want 0x%08x", got, peerID)
658+
}
659+
if !equalBytes(wire[9:], frame) {
660+
t.Fatalf("payload mismatch: got %x want %x", wire[9:], frame)
661+
}
662+
}
663+
602664
func TestWriteFrame_NoAddressNoBeacon(t *testing.T) {
603665
t.Parallel()
604666
m := routing.New()

pkg/daemon/tunnel.go

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -719,6 +719,11 @@ func (tm *TunnelManager) ConnectCompat(ctx context.Context, cfg ConnectCompatCon
719719
}
720720
tm.sock = wssTr
721721
tm.routing.SetSocket(wssTr)
722+
// In compat mode every outbound L2 frame must travel beacon-wrapped:
723+
// the WSS pipe terminates at the beacon, not at peers, so raw frames
724+
// would be received as unknown beacon-protocol packets and dropped.
725+
// Force the routing manager to wrap every send in BeaconMsgRelay.
726+
tm.routing.SetForceRelay(true)
722727

723728
tm.readWg.Add(1)
724729
go tm.readLoop()

0 commit comments

Comments
 (0)