[#1570] Add iceoryx2-dmabuf crate with SCM_RIGHTS fd sidecar and DMA-BUF wrapper#1572
Open
julienzarka wants to merge 3 commits into
Open
[#1570] Add iceoryx2-dmabuf crate with SCM_RIGHTS fd sidecar and DMA-BUF wrapper#1572julienzarka wants to merge 3 commits into
julienzarka wants to merge 3 commits into
Conversation
…channel Introduce a cross-platform role marker trait for out-of-band transports that travel alongside pub/sub metadata. The trait is deliberately free of Linux-specific syscall surface (fd passing, DMA-BUF) so that it compiles on every target; downstream crates extend it with platform-specific primitives. The canonical downstream implementation is `iceoryx2-dmabuf`, which implements `SideChannel` via a Unix domain socket with `SCM_RIGHTS` ancillary data for zero-copy DMA-BUF file-descriptor delivery.
…idecar and DMA-BUF wrapper
Introduce a new workspace crate `iceoryx2-dmabuf` that extends iceoryx2
with out-of-band delivery of kernel-owned file descriptors alongside
typed pub/sub metadata samples.
Two clearly scoped APIs coexist in the crate:
* Transport (`FdSidecarPublisher` / `FdSidecarSubscriber`) — passes any
`OwnedFd` (memfd, eventfd, pipe end, DMA-BUF fd) via `SCM_RIGHTS` over
a Unix domain socket, synchronised to the iceoryx2 sample sequence
number through an `FdSidecarToken` user-header. Linux-only runtime;
non-Linux targets compile and return `UnsupportedPlatform`.
* Payload (`DmaBufPublisher` / `DmaBufSubscriber`, feature `dma-buf`) —
newtype wrappers over the transport that accept and yield
`dma_buf::DmaBuf` values. CPU-sync ioctls (`DMA_BUF_IOCTL_SYNC`
start/end) are delegated to the audited upstream `dma-buf` crate
(v0.5) via `MappedDmaBuf::read/write/readwrite` — `iceoryx2-dmabuf`
does not reimplement the ioctl surface.
The transport implements `iceoryx2::port::side_channel::SideChannel`
(added in the preceding commit) and also a Linux-specific internal
`FdSideChannel` extension trait that documents the `send_fd` /
`recv_fd_matching` contract.
Cargo features (all off by default except `std`):
* `memfd` — `memfd_create`-based helpers for tests and examples
* `peercred` — `SO_PEERCRED` UID verification before fd delivery
* `dma-buf` — `DmaBufPublisher` / `DmaBufSubscriber` typed layer
(Linux only; pulls `dma-buf` 0.5 as optional dep)
* `test-utils` — injection APIs for error-path integration tests
Tested with:
* `cargo test` on Linux aarch64 (OrbStack) and macOS aarch64 — unit,
integration, and property tests all green (Linux-only tests skip on
macOS, `/dev/dma_heap/system` heap test skips when unavailable).
* `cargo clippy --all-features --all-targets -- -D warnings` clean on
both hosts, including `--no-default-features`.
* `cargo fmt --check` clean.
* `cargo doc --no-deps --all-features` emits zero broken-intra-doc-link
warnings.
Examples:
* `publish_subscribe_with_fd/{publisher,subscriber}.rs` — transport
layer with memfd payloads.
* `publish_subscribe_dmabuf/{publisher,subscriber}.rs` — payload
layer with `dma-heap` allocation + `MappedDmaBuf::read`.
Add a `benchmarks-dmabuf` workspace member that measures three
characteristics of the `FdSidecar*` transport from the preceding
`iceoryx2-dmabuf` crate commit:
* `bench_latency` — single publisher/subscriber end-to-end per-frame
latency for 1080p fd passing (p50/p95/p99).
* `bench_throughput` — sustained frames-per-second ceiling with a warm
socket buffer.
* `bench_fanout` — slowest-consumer p95 across 1..N subscribers,
exercising the `SCM_RIGHTS` multi-accept path.
The three binaries live under one `benchmarks-dmabuf` crate that picks
the bench to run via `cargo run -p benchmarks-dmabuf --bin <name>`.
A shared `main.rs` parses common flags (frame count, warm-up frames,
subscriber count) so every bench has a uniform CLI.
Linux only at runtime (same constraint as the transport). Non-Linux
hosts build the crate but the binaries return `UnsupportedPlatform`
immediately.
julienzarka
added a commit
to julienzarka/iceoryx2
that referenced
this pull request
May 5, 2026
…ce variant Captures the design pivot from the standalone sidecar crate (this branch) to a fully-integrated dmabuf::Service variant per maintainer roadmap feedback on PR eclipse-iceoryx#1572. Three deliverables: - specs/arch-dmabuf-service-variant.adoc — Design C architecture (8 decisions D1-D8 + Mermaid component diagram + trade-off matrix) - specs/spec-dmabuf-service-variant.adoc — 49 numbered MUST/SHOULD/MAY requirements + commit structure + acceptance criteria - docs/superpowers/plans/2026-05-05-dmabuf-service-variant.md — 13 TDD tasks (0a/0b through 11) with mos-agent dispatch points, including Task 5b for back-channel + pool-ack integration in the same PR Plus archives of the original sidecar PR's spec/plan/PR-message under iceoryx2-dmabuf/{specs,docs}/ for historical reference. The new branch (feat/dmabuf-service-variant) will cherry-pick these docs from this commit then execute Task 0a (architecture validation spike) before any cal-layer code is written. Refs eclipse-iceoryx#1570
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[#1570] Add
iceoryx2-dmabuf- SCM_RIGHTS fd sidecar and typed DMA-BUF wrapperCloses #1570.
Summary
Adds a new workspace crate
iceoryx2-dmabufthat extends iceoryx2 pub/subwith out-of-band delivery of kernel-owned file descriptors (DMA-BUF frames
from V4L2 ISP, DRM scanout, Vulkan external memory, or generic memfds).
The crate exposes two clearly scoped APIs:
FdSidecarPublisher/FdSidecarSubscriberOwnedFd(memfd, eventfd, pipe end, DMA-BUF fd) - no CPU-sync semanticsdma-buf)DmaBufPublisher/DmaBufSubscriberdma_buf::DmaBufwith safe CPU access viaMappedDmaBuf::read/write/readwriteThe transport carries fds via
SCM_RIGHTSover a per-service Unix-domainsocket, synchronised to the iceoryx2 sample sequence through an
FdSidecarTokenuser-header. The payload layer is a newtype over thetransport that delegates every
DMA_BUF_IOCTL_SYNCto the auditeddma-bufcrate (v0.5, by@mripard) - this PR does not reimplement the
ioctl surface.
Motivation (from #1570)
iceoryx2's typed SHM pool delivers value-type payloads efficiently but
cannot represent kernel-owned DMA-BUF allocations produced by V4L2 ISP,
DRM scanout, or Vulkan external-memory exports. Those frames are
identified by a file descriptor whose numeric value is meaningless
outside the producing process; cross-process transfer requires
SCM_RIGHTSover a Unix-domain socket.iceoryx2-dmabufadds a thinsidecar channel that delivers fds in lockstep with the normal iceoryx2
metadata flow, preserving zero-copy semantics end-to-end.
Changes
iceoryx2(core)iceoryx2::port::side_channel::SideChanneltrait - a cross-platformrole marker for out-of-band transports alongside pub/sub. Deliberately
free of Linux-specific syscall surface so it compiles everywhere.
iceoryx2-dmabuf(new crate)FdSidecarPublisher<S, Meta>/FdSidecarSubscriber<S, Meta>- thetransport layer. Linux-only at runtime; non-Linux targets compile and
return
UnsupportedPlatform.DmaBufPublisher<S, Meta>/DmaBufSubscriber<S, Meta>- typednewtypes over the transport, feature-gated on
dma-buf. Accept a&dma_buf::DmaBufonsend; yield adma_buf::DmaBufonrecv.FdSidecarError::DmaBuf(dma_buf::BufferError)- wraps upstreamBufferErrorinto our non-exhaustive error enum rather thanre-exporting it.
SO_PEERCREDUID check (featurepeercred), sequence-number correlation viaFdSidecarTokenin theiceoryx2 sample user-header, and a 50 ms timeout on fd delivery.
test-utils) for error-path tests.benchmarks/dmabuf(new crate)bench_latency- single-publisher/subscriber per-frame latency for1080p fd passing (p50/p95/p99).
bench_throughput- sustained frames-per-second ceiling.bench_fanout- slowest-consumer p95 across N subscribers.Cargo features (all off by default except
std)memfdmemfd_create-based helpers for tests and examplespeercredSO_PEERCREDUID verification before fd deliverydma-bufDmaBufPublisher/DmaBufSubscribertyped layerdma-buf0.5)test-utilsCommit structure
Each commit compiles, clippies, and tests in isolation (
git bisectgreen).Test evidence
macOS aarch64 (Darwin 25.4, cargo 1.88):
cargo fmt --check✓cargo clippy -p iceoryx2-dmabuf --all-features --all-targets -- -D warnings✓cargo clippy -p iceoryx2-dmabuf --no-default-features --target aarch64-apple-darwin -- -D warnings✓cargo test -p iceoryx2-dmabuf --features "dma-buf memfd peercred test-utils"✓ (Linux-gated tests report 0 run)Linux aarch64 (Ubuntu 25.10, cargo 1.93):
cargo fmt --check✓cargo clippy -p iceoryx2-dmabuf --all-features --all-targets -- -D warnings✓cargo clippy -p iceoryx2-dmabuf --no-default-features -- -D warnings✓cargo test -p iceoryx2-dmabuf --features "dma-buf memfd peercred test-utils"✓ - 28 pass / 0 fail / 2 ignored/dev/dma_heap/systempresent → realDmaBufheap round-trip passes withDMA_BUF_IOCTL_SYNCfiring on both publisher and subscriberTest suite highlights
tests/dmabuf_roundtrip.rs- in-process memfd roundtriptests/it_fanout.rs- 1 publisher × 3 subscribers × 100 framestests/it_crash_midsend.rs- producerSIGSTOPbetween sidecar sendand iceoryx2 send → subscriber gets
NoFdInMessagetests/it_service_gone.rs,tests/it_socket_gone.rs- recovery pathstests/refcount_survival.rs- fd refcount survives N-1 consumer closestests/it_fd_identity.rs- two-processfstatshows same inodetests/error_paths.rs-TokenMismatchandNoFdInMessagepaths(test-utils injection)
tests/prop_roundtrip.rs- proptest random payload sizes 1 B - 16 MiBtests/it_dmabuf_fd_identity.rs- inode preserved acrossDmaBufwraptests/it_dmabuf_heap_roundtrip.rs- real/dev/dma_heap/systemallocation,
MappedDmaBuf::readwith CPU-sync ioctls, byte-patternverification
Examples
examples/publish_subscribe_with_fd/{publisher,subscriber}.rs-transport layer with memfd payloads.
examples/publish_subscribe_dmabuf/{publisher,subscriber}.rs-payload layer with
dma-heapallocation +MappedDmaBuf::read.Out of scope for this PR
sendmsg/recvmsgare synchronous.sync_file/ GPU fence propagation - DMA-BUF only.DXGI_KEYED_MUTEXhandles - Linux only.pidfd_getfd(2)alternative transport - future work if benchmarks show benefit.Checklist
iox2-1570-dmabuf-sidecarper CONTRIBUTING)[#1570]on every commit.rsfiledoc/release-notes/iceoryx2-unreleased.mdQuestions for maintainers
three? The trait + crate + benchmarks split keeps the diff reviewable
in chunks; happy to squash.
iceoryx2-dmabufthe right crate name, or should it be e.g.iceoryx2-fd-sidecarto reflect that the transport works witharbitrary fds and DMA-BUF is one application?
dma-bufdependency should be adefault feature or remain opt-in?
Thanks for the review.