iperf_sctp: apply default socket buffer to fix slow single-CPU throughput Linux SCTP#2043
Open
nshopik wants to merge 1 commit into
Open
iperf_sctp: apply default socket buffer to fix slow single-CPU throughput Linux SCTP#2043nshopik wants to merge 1 commit into
nshopik wants to merge 1 commit into
Conversation
…hput Linux SCTP, unlike TCP, does not auto-tune SO_RCVBUF/SO_SNDBUF (there is no equivalent of tcp_moderate_rcvbuf). Without an explicit setting the per-association receive window collapses to ~40 KB, which on single-CPU systems gates single-stream loopback throughput to roughly 1 Gbps even after the delayed-SACK fix from esnet#1815: sender and softirq compete for the same core, so the small rwnd is never reopened in time. Apply DEFAULT_SCTP_SOCKBUF (512 KB) to SO_RCVBUF and SO_SNDBUF on the client connect socket, the server listen socket, and the per- association accept socket whenever --window is not given. The kernel silently caps at net.core.{r,w}mem_max where smaller, and the explicit --window path is unchanged. Failures on the default path are non-fatal (best-effort, not user-requested). Measured on Debian 13 (kernel 6.12.48), loopback, single stream, 30 s runs, median of 3: 1-vCPU box TCP default : 25.5 Gbps (unchanged) SCTP default before : 0.94 Gbps SCTP default after : 25.5 Gbps (~27x) 2-vCPU box SCTP default before/after: ~22 Gbps (within run-to-run variance) The gap only manifests when sender and softirq share a CPU; on multi- core systems the kernel already works around the small rwnd and the patch is a no-op there.
Contributor
|
Thanks for the PR! Will test this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unlike TCP, it does not auto-tune SO_RCVBUF/SO_SNDBUF (there is no equivalent of tcp_moderate_rcvbuf). Without an explicit setting the per-association receive window collapses to ~40 KB, which on single-CPU systems gates single-stream loopback throughput to roughly 1 Gbps even after the delayed-SACK fix from #1815: sender and softirq compete for the same core, so the small rwnd is never reopened in time.
Apply DEFAULT_SCTP_SOCKBUF (512 KB) to SO_RCVBUF and SO_SNDBUF on the client connect socket, the server listen socket, and the per- association accept socket whenever --window is not given. The kernel silently caps at net.core.{r,w}mem_max where smaller, and the explicit --window path is unchanged. Failures on the default path are non-fatal (best-effort, not user-requested).
Measured on Debian 13 (kernel 6.12.48), loopback, single stream, 30 s runs, median of 3:
The gap only manifests when sender and softirq share a CPU; on multi- core systems the kernel already works around the small rwnd and the patch is a no-op there.
PLEASE NOTE the following text from the iperf3 license. Submitting a
pull request to the iperf3 repository constitutes "[making]
Enhancements available...publicly":
The complete iperf3 license is available in the
LICENSEfile in thetop directory of the iperf3 source tree.
Version of iperf3 (or development branch, such as
masteror3.1-STABLE) to which this pull request applies: masterIssues fixed (if any):
Brief description of code changes (suitable for use as a commit message):