Skip to content

drpcstream: per-stream receive window and read-path grant wiring#72

Open
suj-krishnan wants to merge 2 commits into
sujatha/flow-control-send-gatefrom
sujatha/flow-control-recv-window
Open

drpcstream: per-stream receive window and read-path grant wiring#72
suj-krishnan wants to merge 2 commits into
sujatha/flow-control-send-gatefrom
sujatha/flow-control-recv-window

Conversation

@suj-krishnan

@suj-krishnan suj-krishnan commented Jul 1, 2026

Copy link
Copy Markdown

1. Per-stream receive-window grant policy (recvWindow).
Tracks buffered bytes (in-progress reassembly + completed, not-yet-consumed) and decides when to return credit: withholds grants while buffered is at/above the high-water mark, otherwise releases accrued returnable credit once it reaches the threshold. dispatched/consumed return the credit delta to emit; consuming reopens a gate the high-water mark had closed.

2. Read-path wiring.

  • HandleFrame intercepts an incoming KindWindowUpdate before the assembler and applies it to the send window. Intercepting early keeps a grant that interleaves with an in-progress message (grants are emitted off the write lock) from disturbing reassembly.
  • Discarded partials release their bytes: the assembler silently can drop an unfinished message when a higher message id supersedes it. Those bytes were dispatch-counted and would inflate buffered forever, causing the high-water gate to shut. So, the assembler now reports the drop (TakeDiscarded) and HandleFrame releases the bytes as consumed, so their accrued credit flows back.

@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from 58a5fed to 30ffb52 Compare July 13, 2026 07:00
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch from f6bc0d6 to e72ebc5 Compare July 13, 2026 07:00
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from 30ffb52 to 9b6d87b Compare July 13, 2026 07:06
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch from e72ebc5 to 4953332 Compare July 13, 2026 07:06
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from 9b6d87b to 2512021 Compare July 13, 2026 08:41
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch from 4953332 to 25af70a Compare July 13, 2026 08:41
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from 2512021 to c086e23 Compare July 13, 2026 08:47
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch from 25af70a to 85794ad Compare July 13, 2026 08:47
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from c086e23 to b14cb1f Compare July 13, 2026 08:55
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch from 85794ad to e6a5378 Compare July 13, 2026 08:55
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from b14cb1f to 6594c14 Compare July 13, 2026 08:59
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch from e6a5378 to 74ad6c3 Compare July 13, 2026 08:59
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch 7 times, most recently from fcfa2e4 to 10b863f Compare July 14, 2026 10:25
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch 10 times, most recently from 2b5a3d4 to c07fef5 Compare July 15, 2026 11:03
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from 10b863f to 7d03098 Compare July 15, 2026 11:03
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch 3 times, most recently from 1e6988a to baf780b Compare July 15, 2026 13:57
@suj-krishnan
suj-krishnan marked this pull request as ready for review July 15, 2026 13:59
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from 7d03098 to 336c61c Compare July 16, 2026 09:08
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch 2 times, most recently from be3886d to 1f8204f Compare July 16, 2026 10:16
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from 336c61c to ef342ce Compare July 16, 2026 10:16
suj-krishnan and others added 2 commits July 17, 2026 11:08
Add recvWindow, the per-stream receive side of flow control. It tracks
buffered bytes (in-progress reassembly plus completed, not-yet-consumed
data) and decides when to return credit to the sender: grants are
withheld while buffered is at or above the high-water mark, and the
accrued returnable credit is otherwise released once it reaches the
threshold, coalescing many frames into a single grant. Consuming reopens
a gate that the high-water mark had closed.

dispatched/consumed return the credit delta the caller should emit as a
KindWindowUpdate. Emitting the frame, wiring into the read path
(HandleFrame/MsgRecv), and the connection-level receive budget come in
later commits.

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
…ad path

Install a per-stream recvWindow and drive it from the read/recv path:

  - HandleFrame intercepts an incoming KindWindowUpdate before the
    assembler and applies it to the send window. Intercepting early keeps
    a grant that interleaves with an in-progress message (grants are
    emitted off the write lock) from disturbing reassembly.
  - A dispatched KindMessage frame returns credit via recvWindow, emitting
    a KindWindowUpdate once the accrued amount crosses the threshold and
    the high-water gate is open.
  - RawRecv/MsgRecv call recvWindow after dequeuing; consuming can reopen a
    gate the high-water mark had closed and flush the accrued grant.

Grants are control frames, appended without blocking, so emitting them
from the reader goroutine is safe. The window is opt-in: with no recvw
installed no grants are emitted, and an incoming grant with no send
window is ignored, so default behavior is unchanged.

Stream-level only; the connection-level receive budget and the enablement
path that installs the windows come in later commits.

The packet assembler can silently drop an unfinished message when a
higher message id supersedes it (a legitimate sequence: a send preempted
mid-message, then a cancel). Those bytes were counted into the receive
window at dispatch and would inflate buffered forever, wedging the
high-water gate shut and deadlocking the sender once its credit ran
out. The assembler now records the drop (TakeDiscarded) and HandleFrame
releases the bytes as consumed, letting their credit flow back. The
release runs last (deferred past the replacement frame's accounting and
handlePacket), so the grant decision sees the final buffered state
rather than the transient dip, and a terminal packet that supersedes a
partial suppresses the grant via termination.

emitGrant is also a no-op once the stream has terminated: the ring
drains queued messages after close, and returning credit behind the
terminal frame would invite the peer to send more doomed data.

Co-Authored-By: roachdev-claude <roachdev-claude-bot@cockroachlabs.com>
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-recv-window branch from 1f8204f to 848a5a5 Compare July 17, 2026 05:38
@suj-krishnan
suj-krishnan force-pushed the sujatha/flow-control-send-gate branch from ef342ce to 0d5cf31 Compare July 17, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant