What is the issue?
I was looking at the perf of requests & responses sent via h2 on a TCP stream versus the perf of raw messages sent back and forth over a TCP stream,
and found throughput with h2 to be reduced by an average of around 50%.
The following table compares the perf that I observed on the test VM that I used with h2 requests & responses versus raw messages:
OS: linux (kernel version 6.6)
Arch: arm64
TCP_NODELAY: True
Data Payload Size | Message Kind | Avg Throughput | Avg Round-Trip Latency
4kiB | h2 over TCP | 0.97 GB/s | 8.43 µs
4kiB | raw over TCP | 1.84 GB/s | 4.45 µs
16kiB | h2... | 2.68 GB/s | 12.23 µs
... | raw... | 5.31 GB/s | 6.17 µs
64kiB | h2... | 3.54 GB/s | 37.05 µs
... | raw... | 8.43 GB/s | 15.54 µs
256kiB | h2... | 4.64 GB/s | 113.00 µs
... | raw... | 9.43 GB/s | 55.57 µs
1024kiB | h2... | 4.28 GB/s | 490.00 µs
... | raw... | 9.22 GB/s | 227.40 µs
Primary root causes
Currently, the encoder can only hold a single pending DATA frame, and blocks the acceptance of any new frame until the
previous one is fully flushed to the socket and reclaimed.
This means every DATA frame triggers its own write() syscall and TCP segment — even when the TCP send buffer could carry far more.
With TCP_NODELAY enabled, there is no Nagle buffering to coalesce these small writes, so the per-frame overhead is fully realized.
On the receiver side, the recv buffer used by the frame decoder is small by default, and even if its buffer size was increased,
the effective capacity of that buffer would degrade over time and never recover, causing small read() calls.
This is because:
- The buffer is repeatedly split to yield decoded frames.
- If the recv buffer runs out of capacity, BytesMut::reserve is called, & if that buffer is still in use (likely),
the current buffer is abandoned & a new buffer with a maximum capacity of 64kB would be created
due to how BytesMut encodes its original capacity to a max of 64kB & caps new reservations
based on that encoded original capacity.
Together, these things inflate the syscall-to-payload ratio on both ends.
What is the issue?
I was looking at the perf of requests & responses sent via h2 on a TCP stream versus the perf of raw messages sent back and forth over a TCP stream,
and found throughput with h2 to be reduced by an average of around 50%.
The following table compares the perf that I observed on the test VM that I used with h2 requests & responses versus raw messages:
Primary root causes
Currently, the encoder can only hold a single pending DATA frame, and blocks the acceptance of any new frame until the
previous one is fully flushed to the socket and reclaimed.
This means every DATA frame triggers its own write() syscall and TCP segment — even when the TCP send buffer could carry far more.
With TCP_NODELAY enabled, there is no Nagle buffering to coalesce these small writes, so the per-frame overhead is fully realized.
On the receiver side, the recv buffer used by the frame decoder is small by default, and even if its buffer size was increased,
the effective capacity of that buffer would degrade over time and never recover, causing small
read()calls.This is because:
the current buffer is abandoned & a new buffer with a maximum capacity of 64kB would be created
due to how BytesMut encodes its original capacity to a max of 64kB & caps new reservations
based on that encoded original capacity.
Together, these things inflate the syscall-to-payload ratio on both ends.