Skip to content

Commit 7d7834d

Browse files
committed
TC-85: document tunnel relay contract additions from the review round
- Sequence coordination is per-name, not per-operation: claim/delete/cert/ tunnel-auth all share one counter, single source of truth on the node. - Max frame (1MB) and body (25MB, TUNNEL_MAX_BODY_BYTES) size limits, and the requestBody/responseBody chunking contract this implies. - Frames carrying an unrecognized request id are silently ignored. - Headers are ordered [name, value] pairs so duplicate header names (e.g. Set-Cookie) survive. - Connection-attempt limits enforced before the WS handshake (per-IP, per-name churn, global concurrent-tunnel cap via TUNNEL_MAX_CONCURRENT).
1 parent 6e45067 commit 7d7834d

1 file changed

Lines changed: 9 additions & 5 deletions

File tree

README.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,8 @@ Public, unsigned. Returns `{ name, subject, lanIps, updatedAt }` or `404`.
111111

112112
There is one name registry with two surfaces: a name claimed via `PUT /v1/names/:name` gets LAN A/AAAA records (`<name>.local.tinycloud.link`) *and* the right to open a tunnel for `<name>.tinycloud.link`. Same owner, same signing key, same sequence counter.
113113

114+
**Sequence coordination is per-name, not per-operation**: `claim`/`delete` (`PUT`/`DELETE /v1/names/:name`), `cert` (`POST /v1/certs/:name`), and `tunnel` (the WS auth frame) all read and bump the *same* `sequence` column on the name's row in `names`. There is exactly one counter per name, shared across every action type -- a node must treat it as a single source of truth (e.g. keep one in-memory counter per name and increment it before every signed write, regardless of which endpoint), not maintain a separate counter per action. Reusing a sequence across two different action types for the same name is exactly as stale/rejected as reusing it for the same action twice.
115+
114116
### Lifecycle
115117

116118
```
@@ -144,24 +146,26 @@ There is one name registry with two surfaces: a name claimed via `PUT /v1/names/
144146
- **Close codes on rejection** (RFC 6455 private-use range): `4400` malformed auth frame or frame `name` != URL name, `4401` invalid signature, `4403` subject does not own the name, `4404` name not claimed, `4408` auth timeout, `4409` stale sequence, `4410` superseded by a newer connection for the same name.
145147
- **One socket per name, newest wins** (`src/tunnel/registry.ts`): a second authenticated connection for the same name evicts the first (close `4410`) rather than being rejected -- a node reconnecting after a network blip must be able to take over from its own half-dead previous socket. (Each reconnect needs a fresh auth frame with a higher sequence.)
146148
- **Keep-alive**: the relay pings every 30s and terminates the socket if the previous ping got no pong. Standard WS libraries answer pings automatically; a Rust client must confirm its library does (tungstenite does).
149+
- **Connection-attempt limits** (`src/tunnel/upgrade.ts`, checked in the raw HTTP `upgrade` handler, before the WS handshake and before any Postgres read): at most 30 upgrade attempts per remote IP per minute, at most 10 upgrade attempts per tunnel name per minute (bounds reconnect/eviction churn on one name), and a global cap of 1000 concurrently registered tunnels (env-tunable via `TUNNEL_MAX_CONCURRENT`, see below). An attempt over any of these limits gets the raw TCP socket destroyed -- no WS handshake, no close code, nothing for the node to parse.
147150

148151
### Frame protocol
149152

150153
Defined in `src/tunnel/protocol.ts` -- that file is the source of truth for the Rust node client. Every frame is one JSON **text** message (binary WS messages are never used); body bytes travel base64-encoded inside the JSON. After the ack, the frames are:
151154

152155
| frame | direction | fields |
153156
|---|---|---|
154-
| `request` | relay → node | `id` (UUID string), `method`, `path` (path + query, always starts with `/`), `headers` (string map; `Host` and hop-by-hop headers stripped) |
157+
| `request` | relay → node | `id` (UUID string), `method`, `path` (path + query, always starts with `/`), `headers` (ordered `[name, value]` pairs; `Host` and hop-by-hop headers stripped) |
155158
| `requestBody` | relay → node | `id`, `chunk` (base64, may be `""`), `done` (bool) |
156-
| `response` | node → relay | `id`, `status` (int), `headers` (string map) |
159+
| `response` | node → relay | `id`, `status` (int), `headers` (ordered `[name, value]` pairs) |
157160
| `responseBody` | node → relay | `id`, `chunk` (base64, may be `""`), `done` (bool) |
158161
| `error` | either direction | `message`, optional `id` (fails just that request without closing the socket) |
159162

160163
Rules a node client must follow:
161164

162-
- Requests are multiplexed: frames for different `id`s interleave freely on the one socket. Echo the request's `id` on every frame you send for it.
163-
- The relay sends `request` first, then at least one `requestBody` (currently exactly one carrying the whole body with `done: true`, but a client must tolerate multiple chunks and treat `done: true` as the boundary).
164-
- The node replies with exactly one `response` frame, then one or more `responseBody` frames ending with `done: true`. An empty body is still one frame: `{"type":"responseBody","id":...,"chunk":"","done":true}`.
165+
- Requests are multiplexed: frames for different `id`s interleave freely on the one socket. Echo the request's `id` on every frame you send for it. A frame carrying an `id` the relay/node doesn't recognize (e.g. arriving after that request already failed or timed out) is silently ignored, never treated as a protocol error.
166+
- **Headers are ordered `[name, value]` pairs, not an object** -- so a header repeated on the wire (most importantly `Set-Cookie`, but any header) survives as multiple array entries instead of colliding on one object key. A client must be able to both emit and parse duplicate entries for the same header name.
167+
- The relay sends `request` first, then one or more `requestBody` frames ending with `done: true`; the node replies with exactly one `response` frame, then one or more `responseBody` frames ending with `done: true`. An empty body is still exactly one frame: `{"...Body","id":...,"chunk":"","done":true}`.
168+
- **Size limits**: no single WS frame may exceed `MAX_FRAME_PAYLOAD_BYTES` (1MB) -- the relay's `WebSocketServer` is configured with this as `maxPayload` and will close a connection that sends a larger one (WS close code `1009`). A body larger than `BODY_CHUNK_BYTES` (256KB, chosen to stay well under 1MB once base64-inflated) must be split across multiple body frames; only the last carries `done: true`. The relay itself follows this when sending `requestBody` frames and expects a node to do the same for `responseBody`. Independent of frame size, the full reassembled body (request or response) is capped at `DEFAULT_MAX_BODY_BYTES` (25MB, overridable via the `TUNNEL_MAX_BODY_BYTES` env var, see below): a request over the cap gets `413` directly from the relay before it ever reaches the tunnel; a response over the cap is aborted with `502` to the HTTP caller and an `{"type":"error","id":...}` frame is sent back to the node so it knows to stop sending.
165169
- If the node cannot serve a request it sends `{"type":"error","id":...,"message":...}`; the relay answers the HTTP caller with `502`. The relay applies a 30s per-request timeout (caller gets `504`).
166170
- HTTP callers with no live tunnel for their Host get `502` directly from the relay.
167171

0 commit comments

Comments
 (0)