Skip to content

iperf_sctp: apply default socket buffer to fix slow single-CPU throughput Linux SCTP#2043

Open
nshopik wants to merge 1 commit into
esnet:masterfrom
nshopik:sctp-default-sockbuf
Open

iperf_sctp: apply default socket buffer to fix slow single-CPU throughput Linux SCTP#2043
nshopik wants to merge 1 commit into
esnet:masterfrom
nshopik:sctp-default-sockbuf

Conversation

@nshopik

@nshopik nshopik commented May 24, 2026

Copy link
Copy Markdown
Contributor

Unlike TCP, it does not auto-tune SO_RCVBUF/SO_SNDBUF (there is no equivalent of tcp_moderate_rcvbuf). Without an explicit setting the per-association receive window collapses to ~40 KB, which on single-CPU systems gates single-stream loopback throughput to roughly 1 Gbps even after the delayed-SACK fix from #1815: sender and softirq compete for the same core, so the small rwnd is never reopened in time.

Apply DEFAULT_SCTP_SOCKBUF (512 KB) to SO_RCVBUF and SO_SNDBUF on the client connect socket, the server listen socket, and the per- association accept socket whenever --window is not given. The kernel silently caps at net.core.{r,w}mem_max where smaller, and the explicit --window path is unchanged. Failures on the default path are non-fatal (best-effort, not user-requested).

Measured on Debian 13 (kernel 6.12.48), loopback, single stream, 30 s runs, median of 3:

1-vCPU box
  TCP default          : 25.5 Gbps (unchanged)
  SCTP default before  :  0.94 Gbps
  SCTP default after   : 25.5 Gbps  (~27x)

2-vCPU box
  SCTP default before/after: ~22 Gbps (within run-to-run variance)

The gap only manifests when sender and softirq share a CPU; on multi- core systems the kernel already works around the small rwnd and the patch is a no-op there.

PLEASE NOTE the following text from the iperf3 license. Submitting a
pull request to the iperf3 repository constitutes "[making]
Enhancements available...publicly":

You are under no obligation whatsoever to provide any bug fixes, patches, or
upgrades to the features, functionality or performance of the source code
("Enhancements") to anyone; however, if you choose to make your Enhancements
available either publicly, or directly to Lawrence Berkeley National
Laboratory, without imposing a separate written license agreement for such
Enhancements, then you hereby grant the following license: a non-exclusive,
royalty-free perpetual license to install, use, modify, prepare derivative
works, incorporate into other computer software, distribute, and sublicense
such enhancements or derivative works thereof, in binary and source code form.

The complete iperf3 license is available in the LICENSE file in the
top directory of the iperf3 source tree.

  • Version of iperf3 (or development branch, such as master or
    3.1-STABLE) to which this pull request applies: master

  • Issues fixed (if any):

  • Brief description of code changes (suitable for use as a commit message):

…hput

Linux SCTP, unlike TCP, does not auto-tune SO_RCVBUF/SO_SNDBUF (there
is no equivalent of tcp_moderate_rcvbuf).  Without an explicit setting
the per-association receive window collapses to ~40 KB, which on
single-CPU systems gates single-stream loopback throughput to roughly
1 Gbps even after the delayed-SACK fix from esnet#1815: sender and softirq
compete for the same core, so the small rwnd is never reopened in time.

Apply DEFAULT_SCTP_SOCKBUF (512 KB) to SO_RCVBUF and SO_SNDBUF on the
client connect socket, the server listen socket, and the per-
association accept socket whenever --window is not given.  The kernel
silently caps at net.core.{r,w}mem_max where smaller, and the explicit
--window path is unchanged.  Failures on the default path are non-fatal
(best-effort, not user-requested).

Measured on Debian 13 (kernel 6.12.48), loopback, single stream, 30 s
runs, median of 3:

    1-vCPU box
      TCP default          : 25.5 Gbps (unchanged)
      SCTP default before  :  0.94 Gbps
      SCTP default after   : 25.5 Gbps  (~27x)

    2-vCPU box
      SCTP default before/after: ~22 Gbps (within run-to-run variance)

The gap only manifests when sender and softirq share a CPU; on multi-
core systems the kernel already works around the small rwnd and the
patch is a no-op there.
@bmah888

bmah888 commented May 29, 2026

Copy link
Copy Markdown
Contributor

Thanks for the PR! Will test this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants