Commit 327c542
committed
release(v1.10.4): auto-recover compat-mode WSS connection
In v1.10.2/v1.10.3, when the daemon's WSS connection to the beacon
died (network blip, nginx restart, peer hangup), the transport
stayed permanently dead — every subsequent Send returned "wss send:
failed to write msg: use of closed network connection" and a manual
daemon restart was the only recovery. Observed in production twice
in one hour today.
New design: a supervisor goroutine owns both the read loop and the
reconnect lifecycle.
1. drainReads(conn) — block on conn.Read with lifetimeCtx
2. on read error: tear down conn, sleep with exp backoff
(250 ms → 30 s cap), poll closed flag at 100 ms
3. dialAndAuth a fresh conn (parented on lifetimeCtx so Close
interrupts an in-flight dial)
4. install + resume read loop
Send returns the new wss.ErrReconnecting (instead of a raw conn
write error) when no live conn is installed — caller's higher-level
retransmit (key-exchange retx, dial-retry, writeFrame retry)
refires naturally on the next conn.
Transient read errors during the reconnect window are NOT surfaced
to Recv(), so the daemon's tunnel readLoop doesn't tear itself down
over a temporary blip.
Tests:
- TestReconnect_AfterServerCloseRestoresSendAndRecv: fakeBeacon
CloseNow's the conn after the second incoming frame; supervisor
reconnects; a probe-loop confirms ErrReconnecting during the gap
and a successful echo on the freshly-established conn
- TestServer_LastWriterWinsOnReconnect updated to not poll Recv
(the test's previous "tr1.Recv returns ErrClosed after kick"
assertion is incompatible with auto-reconnect, which is the
intended new behavior; the server-side IsConnected check still
validates last-writer-wins)1 parent 822e12d commit 327c542
5 files changed
Lines changed: 396 additions & 54 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
10 | 46 | | |
11 | 47 | | |
12 | 48 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
13 | 12 | | |
14 | 13 | | |
15 | 14 | | |
| |||
22 | 21 | | |
23 | 22 | | |
24 | 23 | | |
25 | | - | |
26 | 24 | | |
27 | 25 | | |
28 | 26 | | |
| |||
303 | 301 | | |
304 | 302 | | |
305 | 303 | | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
306 | 311 | | |
307 | 312 | | |
308 | 313 | | |
309 | 314 | | |
310 | | - | |
311 | | - | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
317 | | - | |
318 | | - | |
319 | | - | |
320 | | - | |
321 | 315 | | |
| 316 | + | |
322 | 317 | | |
323 | 318 | | |
324 | 319 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
82 | 82 | | |
83 | 83 | | |
84 | 84 | | |
85 | | - | |
86 | | - | |
87 | | - | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
88 | 88 | | |
89 | 89 | | |
90 | 90 | | |
| |||
0 commit comments