Commit cc3f33b
committed
fix(net): defer a would-block on-behalf send off the notification loop
Seccomp notifications are processed sequentially on one loop, and the
on-behalf sends (sendmsg / sendto in send_msghdr_on_behalf,
send_named_unix_msghdr, sendto_on_behalf) ran inline on it. Because
pidfd_getfd shares the child's open file description, the supervisor
inherited the child's blocking mode, so a send to a peer that stops
draining blocked the whole loop — every trapped syscall from every
sandboxed process hung until the peer read or closed. A child could
trigger it with no network permission at all (fill one end of a
socketpair, then sendmsg). #125 widened the reach by routing connected
AF_UNIX stream sendmsg (Wayland, D-Bus) through this path.
The send is now non-blocking and, when it would block, deferred:
- Each on-behalf message is materialized into an owned MaterializedMsg
(flattened iovec payload, translated control buffer with its
SCM_RIGHTS fds, any named-unix inode pin, destination sockaddr), so it
can be replayed from a byte offset on a worker without borrowing
supervisor state. Batches are materialized one entry at a time, never
whole (vlen x MAX_SEND_BUF would be the OOM the cap prevents).
- resolve_send does the first send with MSG_DONTWAIT on the loop (never
blocking there). For a child that wants blocking (socket is O_NONBLOCK-
clear AND the call didn't pass MSG_DONTWAIT) whose message didn't fully
fit, push_until_done finishes it off the loop: it awaits writability via
the Tokio IO driver's epoll (never blocking a worker thread) and pushes
the rest, advancing the offset past each partial send, until the full
length is delivered — preserving the kernel's "a blocking send of N
returns N" contract. Control/address are attached only at offset 0, so
SCM_RIGHTS transmits once. A non-blocking socket, or a per-call
MSG_DONTWAIT, gets the single-attempt result (short count / EAGAIN), as
the kernel would return.
- sendmmsg keeps blocking semantics per entry: an entry that would block
entirely (first entry) or partially sends a stream is completed off the
loop via the shared complete_batch_entry tail (writing the full
per-entry msg_len so a caller ignoring it isn't silently truncated),
then reports the message count. A would-block at a later entry with
nothing sent is a contract-legal short count the child retries.
Verified end-to-end: a single blocking send() of 2 MiB to a peer that
stalls 2 s then drains returns the full 2 MiB in ~2 s (not truncated to
one SO_SNDBUF); the same with MSG_DONTWAIT returns a short count
immediately; and a concurrent thread's sends stay at ~1 ms latency while a
blocking send is stalled (the loop is not wedged). Adds an integration
test that one blocking send() of 4 MiB to a stall-then-drain peer returns
the full count and delivers every byte.1 parent 732b147 commit cc3f33b
2 files changed
Lines changed: 528 additions & 158 deletions
0 commit comments