perf(tcp): disable Nagle's algorithm on accepted connections#210
Open
EddieHouston wants to merge 1 commit intoBlockstream:new-indexfrom
Open
perf(tcp): disable Nagle's algorithm on accepted connections#210EddieHouston wants to merge 1 commit intoBlockstream:new-indexfrom
EddieHouston wants to merge 1 commit intoBlockstream:new-indexfrom
Conversation
Set TCP_NODELAY on all accepted TCP connections for both the Electrum RPC server and the REST HTTP server. Nagle's algorithm interacts poorly with TCP delayed ACKs on request/response protocols, adding up to 200ms of latency per round trip on otherwise idle connections. Unix socket path in rest.rs is correctly left unchanged as TCP_NODELAY does not apply to Unix domain sockets.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disable Nagle's algorithm on accepted TCP connections
Set
TCP_NODELAYon all accepted connections for both the Electrum RPC server (server.rs) and the REST HTTP server (rest.rs).Why
Nagle's algorithm coalesces small outgoing segments, waiting up to 200ms for more data before flushing. On request/response protocols this interacts badly with TCP delayed ACKs on the peer: the server holds a small response waiting to batch it, while the client holds its ACK waiting for more data, producing an unnecessary stall on every round trip. Disabling Nagle ensures responses are sent immediately without waiting for the coalescing timer.
The Unix socket path in
rest.rsis left unchanged —TCP_NODELAYis not applicable to Unix domain sockets.Benchmark methodology
A BDK wallet sync (
ElectrumBlockchain::sync,stop_gap=20, 2 descriptors) against a signet electrs node. The sync issues 204 sequential Electrum RPC calls on a single persistent TCP connection(
blockchain.scripthash.get_history×200,blockchain.transaction.get×2,blockchain.headers.subscribe×1,blockchain.block.header×1).The gain comes from eliminating the Nagle + delayed-ACK stall: without
TCP_NODELAY, the server holds small responses for up to 40ms waiting to coalesce a full TCP segment. With 200 sequential request/response pairs this compoundssignificantly under any meaningful RTT.
Simulated WAN latency (toxiproxy), 3 runs each
TCP_NODELAYTCP_NODELAYReal network (separate laptop over WiFi, ~9ms RTT), 4 runs each
TCP_NODELAY(Nagle ON)TCP_NODELAYThe WiFi result is stronger because at 9ms RTT the 40ms Nagle stall represents a larger fraction of each round trip than at 50ms, so it fires more often relative to total time.
Changes
src/electrum/server.rs— setTCP_NODELAYon accepted streams before handing off to the connection acceptorsrc/rest.rs— setTCP_NODELAYon accepted streams before spawning the connection handler; Unix socket path unchanged