Skip to content

Commit 098e02d

Browse files
ccie18643claude
andcommitted
feat(tcp): receive-buffer Dynamic Right-Sizing (autotune step 4 / R2-R4)
Fourth phase of the Tier-3 buffer auto-tuning plan (docs/refactor/tcp_buffer_autotuning.md §4 R2-R4 / §10 step 4): grow the advertised receive window toward the bandwidth-delay product, mirroring Linux tcp_rcv_space_adjust. Builds on the R1 receiver-RTT estimator. R2 (state): - New RcvSpaceState (state/tcp__state__rcv_space.py): space (high-water per-RTT copied bytes) + copied_anchor + time_ms — the measurement window. Seeded self._rcv_space with space = initial rcv_wnd_max in TcpSession.__init__, plus self._rcv_copied_total bumped by byte_count in receive() so the policy can compute per-RTT throughput. R3 + R4 (trigger + policy): - TcpSession._maybe_adjust_rcv_space() runs from receive() after the drain (the app-thread analogue of Linux calling it from tcp_recvmsg). Once per receiver-RTT, when copied exceeds the last measurement, it computes rcvwin = 2*copied + 16*advmss plus a sender-rate headroom term, clamps at min(tcp.rmem.max, 0xFFFF << rcv_wsc) — the R0 WSCALE ceiling so the grown window stays expressible under the negotiated shift — and calls the existing grow-only grow_rcv_wnd_max. per_mss is the bare advmss (no tcp_adv_win_scale overhead — the locked Tier-1 deviation), so window == buffer size. - No-op until the receiver RTT exists, before one RTT elapsed, when tcp.moderate_rcvbuf is off, or when SO_RCVBUF is set (SOCK_RCVBUF_LOCK — makes DRS and the A3 mid-connection resize mutually-exclusive writers of rcv_wnd_max, preserving the single-writer invariant). DRS is now functionally complete. It stays inert on the common path until per-RTT throughput exceeds the start window; no behaviour change for connections that never fill it or that pin SO_RCVBUF. Tests-first: test__tcp__session__drs.py (7 cases — grow toward BDP, cadence gate, no-RTT skip, SO_RCVBUF lock, moderate_rcvbuf off, rmem.max clamp, receive()-path trigger). RED pre-impl: AttributeError (_rcv_space absent). Two grow tests initially failed because the default handshake negotiates no WSCALE, so the R0 ceiling (0xFFFF << 0) correctly capped at 65535 — the RED step surfaced that the grow tests must negotiate WSCALE; fixed by establishing with peer_wscale=7. Plan §4 / §10 step 4 marked done. Reference: RFC 9293 §3.8.6 (receive window management). Reference: RFC 7323 §2 (window scale bounds the grown window). Reference: Linux tcp_rcv_space_adjust / net.ipv4.tcp_moderate_rcvbuf. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent cc74c38 commit 098e02d

4 files changed

Lines changed: 454 additions & 1 deletion

File tree

docs/refactor/tcp_buffer_autotuning.md

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -209,6 +209,23 @@ for a pure receiver.
209209
timestamps-gated for the first pass (Linux DRS is materially weaker
210210
without timestamps too).
211211

212+
**R2 + R3 + R4 are DONE** (autotune step 4). `RcvSpaceState`
213+
(`state/tcp__state__rcv_space.py`: `space` / `copied_anchor` / `time_ms`)
214+
+ `self._rcv_copied_total` bumped in `receive()`; `_rcv_space.space`
215+
seeded from `rcv_wnd_max` at construction.
216+
`TcpSession._maybe_adjust_rcv_space()` runs from `receive()` after the
217+
drain: gates on `_rcv_rtt.rtt_ms` present and one RTT elapsed, computes
218+
`2*copied + 16*advmss` + the sender-rate term, clamps at
219+
`min(tcp.rmem.max, 0xFFFF << rcv_wsc)` (the R0 WSCALE ceiling), and calls
220+
the grow-only `grow_rcv_wnd_max`; a no-op under
221+
`_so_rcvbuf is not None` (SOCK_RCVBUF_LOCK) or `tcp.moderate_rcvbuf == 0`.
222+
Tests: `test__tcp__session__drs.py` (grow, cadence gate, no-RTT skip,
223+
SO_RCVBUF lock, moderate_rcvbuf off, rmem.max clamp, receive()-path
224+
trigger). The grow tests establish with WSCALE negotiated — without a
225+
non-zero `rcv_wsc` the R0 ceiling correctly caps the window at 65535.
226+
227+
Original plan text (R2/R3/R4):
228+
212229
### R2 — the `RcvSpaceState` struct (small)
213230

214231
- New `state/tcp__state__rcv_space.py` `@dataclass(slots=True)`
@@ -513,7 +530,13 @@ CLAUDE.md "Linux as tiebreaker" precedence.
513530
`tcp__session__ack.py` phase-5 TSecr hook; timestamps-gated. Tests:
514531
`test__tcp__state__rcv_rtt.py`, `test__tcp__session__rcv_rtt.py`.
515532
4. **R2 + R3 + R4** (DRS struct, trigger, grow policy) — the large piece;
516-
lands on top of R1.
533+
lands on top of R1. **DONE**`RcvSpaceState` + copied counter +
534+
`_maybe_adjust_rcv_space()` fired from `receive()`, driving the
535+
grow-only `grow_rcv_wnd_max`, clamped at `tcp.rmem.max` / the WSCALE
536+
ceiling, gated on `tcp.moderate_rcvbuf` + `SO_RCVBUF` unset. Tests:
537+
`test__tcp__session__drs.py`. **DRS is functionally complete** (still
538+
inert on the common path until step 5 raises throughput enough to
539+
grow, but it fires whenever per-RTT throughput exceeds the window).
517540
5. **The small-default switch** (§2 / §7) — flip the unset `SO_SNDBUF`
518541
(and optionally `rcv_wnd_max`) default to the Tier-2 `.default`,
519542
making auto-tuning observable. Behaviour-changing → its own deliberate

packages/pytcp/pytcp/protocols/tcp/session/tcp__session.py

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -63,6 +63,7 @@
6363
from pytcp.protocols.tcp.state.tcp__state__persist import PersistState
6464
from pytcp.protocols.tcp.state.tcp__state__rack_tlp import RackTlpState
6565
from pytcp.protocols.tcp.state.tcp__state__rcv_rtt import RcvRttState
66+
from pytcp.protocols.tcp.state.tcp__state__rcv_space import RcvSpaceState
6667
from pytcp.protocols.tcp.state.tcp__state__recv_seq import RecvSeqState
6768
from pytcp.protocols.tcp.state.tcp__state__rtt_sample import RttSampleState
6869
from pytcp.protocols.tcp.state.tcp__state__send_seq import SendSeqState
@@ -344,6 +345,13 @@ def __init__(
344345
# cadence). Fed from the inbound TSecr echo, so it produces an
345346
# RTT even on a pure receiver. See 'state/tcp__state__rcv_rtt.py'.
346347
self._rcv_rtt: RcvRttState = RcvRttState()
348+
# Tier-3 Track R DRS measurement window. 'space' seeds from the
349+
# initial 'rcv_wnd_max' (set above from SO_RCVBUF) so the first
350+
# grow fires only once per-RTT throughput exceeds the start
351+
# window. '_rcv_copied_total' is the cumulative count of bytes
352+
# the application has drained via 'receive()'.
353+
self._rcv_space: RcvSpaceState = RcvSpaceState(space=self._win.rcv_wnd_max)
354+
self._rcv_copied_total: int = 0
347355
self._retransmit_count: int = 0
348356
# RFC 6298 §5.7 second-clause SYN-retransmit counter.
349357
# Decoupled from '_retransmit_count' (which
@@ -1304,6 +1312,9 @@ def receive(self, *, byte_count: int | None = None, timeout: float | None = None
13041312

13051313
rx_buffer = self._rx_buffer[:byte_count]
13061314
del self._rx_buffer[:byte_count]
1315+
# Tier-3 Track R: count bytes the application copies out so
1316+
# the DRS measurement can compute per-RTT throughput.
1317+
self._rcv_copied_total += byte_count
13071318

13081319
# Clear the event only when the buffer is fully drained
13091320
# AND the remote end is still open. When the remote
@@ -1325,8 +1336,61 @@ def receive(self, *, byte_count: int | None = None, timeout: float | None = None
13251336
# half (matches BSD select-on-read-EOF semantics).
13261337
self._socket._drain_readable()
13271338

1339+
# Tier-3 Track R DRS: the app just drained the buffer, so run
1340+
# the once-per-RTT receive-window adjust (Linux calls
1341+
# 'tcp_rcv_space_adjust' from 'tcp_recvmsg'). Outside the
1342+
# rx-buffer lock — it touches only app-thread DRS state and the
1343+
# lockless grow-only 'grow_rcv_wnd_max'.
1344+
self._maybe_adjust_rcv_space()
1345+
13281346
return bytes(rx_buffer)
13291347

1348+
def _maybe_adjust_rcv_space(self) -> None:
1349+
"""
1350+
Receive-buffer Dynamic Right-Sizing (Tier-3 Track R, mirroring
1351+
Linux 'tcp_rcv_space_adjust'). At most once per receiver-RTT,
1352+
when the application has drained more than the previously
1353+
measured per-RTT bytes ('space'), grow 'rcv_wnd_max' toward the
1354+
estimated bandwidth-delay product:
1355+
1356+
rcvwin = 2 * copied + 16 * advmss # BDP + loss slack
1357+
rcvwin += 2 * rcvwin * (copied - space) // space # sender-rate headroom
1358+
target = min(rcvwin, tcp.rmem.max, 0xFFFF << rcv_wsc)
1359+
1360+
'per_mss' is the bare advertised MSS (no skb-truesize / no
1361+
'tcp_adv_win_scale' overhead — the same deviation locked for
1362+
Tier-1), so the window value maps directly to the buffer size.
1363+
The WSCALE-ceiling clamp keeps the grown window expressible under
1364+
the shift negotiated at handshake (RFC 7323).
1365+
1366+
A no-op until the receiver RTT estimate exists, before one RTT
1367+
has elapsed since the last measurement, when 'tcp.moderate_rcvbuf'
1368+
is off, or when SO_RCVBUF is set (SOCK_RCVBUF_LOCK). Runs on the
1369+
application thread; the actuator 'grow_rcv_wnd_max' is grow-only
1370+
and lockless.
1371+
"""
1372+
1373+
rtt_ms = self._rcv_rtt.rtt_ms
1374+
if rtt_ms is None:
1375+
return
1376+
now_ms = stack.timer.now_ms
1377+
if now_ms - self._rcv_space.time_ms < rtt_ms:
1378+
return
1379+
1380+
copied = self._rcv_copied_total - self._rcv_space.copied_anchor
1381+
if copied > self._rcv_space.space:
1382+
if self._socket._so_rcvbuf is None and tcp__constants.TCP__MODERATE_RCVBUF:
1383+
advmss = self._win.rcv_mss
1384+
rcvwin = 2 * copied + 16 * advmss
1385+
rcvwin += 2 * (rcvwin * (copied - self._rcv_space.space) // self._rcv_space.space)
1386+
target = min(rcvwin, tcp__constants.TCP__RMEM__MAX, 0xFFFF << self._win.rcv_wsc)
1387+
if target > self._win.rcv_wnd_max:
1388+
self.grow_rcv_wnd_max(target)
1389+
self._rcv_space.space = copied
1390+
1391+
self._rcv_space.copied_anchor = self._rcv_copied_total
1392+
self._rcv_space.time_ms = now_ms
1393+
13301394
def close(self) -> None:
13311395
"""
13321396
The 'CLOSE' syscall.
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
################################################################################
2+
## ##
3+
## PyTCP - Python TCP/IP stack ##
4+
## Copyright (C) 2020-present Sebastian Majewski ##
5+
## ##
6+
## This program is free software: you can redistribute it and/or modify ##
7+
## it under the terms of the GNU General Public License as published by ##
8+
## the Free Software Foundation, either version 3 of the License, or ##
9+
## (at your option) any later version. ##
10+
## ##
11+
## This program is distributed in the hope that it will be useful, ##
12+
## but WITHOUT ANY WARRANTY; without even the implied warranty of ##
13+
## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ##
14+
## GNU General Public License for more details. ##
15+
## ##
16+
## You should have received a copy of the GNU General Public License ##
17+
## along with this program. If not, see <https://www.gnu.org/licenses/>. ##
18+
## ##
19+
## Author's email: ccie18643@gmail.com ##
20+
## Github repository: https://github.com/ccie18643/PyTCP ##
21+
## ##
22+
################################################################################
23+
24+
25+
"""
26+
This module contains the per-session receive-buffer Dynamic Right-Sizing
27+
(DRS) measurement state — the Tier-3 Track R analogue of Linux's
28+
'tcp_sock.rcvq_space'. It records the high-water-mark of bytes the
29+
application copied out in one receiver-RTT ('space') plus the anchor the
30+
next measurement is taken from ('copied_anchor' / 'time_ms'), so the
31+
grow policy can compare the current per-RTT throughput against the last.
32+
33+
pytcp/protocols/tcp/state/tcp__state__rcv_space.py
34+
35+
ver 3.0.8
36+
"""
37+
38+
from dataclasses import dataclass
39+
40+
41+
@dataclass(slots=True)
42+
class RcvSpaceState:
43+
"""
44+
Per-session DRS measurement window. Owned by 'TcpSession' and
45+
touched only on the application thread (inside 'receive()'), so it
46+
needs no lock. 'space' is seeded from the initial 'rcv_wnd_max' so
47+
the first grow fires only once per-RTT throughput exceeds the
48+
starting window.
49+
"""
50+
51+
# High-water-mark of bytes copied to the application in a single
52+
# receiver-RTT. The grow policy fires when the latest per-RTT
53+
# 'copied' exceeds this; then 'space' rises to that 'copied'.
54+
space: int = 0
55+
56+
# Value of the session's cumulative copied-bytes counter at the
57+
# last measurement — 'copied = total - copied_anchor' is the bytes
58+
# drained in the window just ending.
59+
copied_anchor: int = 0
60+
61+
# 'stack.timer.now_ms' at the last measurement; the next adjust is
62+
# gated until one receiver-RTT has elapsed past this.
63+
time_ms: int = 0

0 commit comments

Comments
 (0)