feat(moq-video): native V4L2 capture, NVENC via dlopen, drop VAAPI + nokhwa(Linux)#1704
Merged
Conversation
…erage Continues the native-codecs work from #1691 by writing the last missing piece (the V4L2 dmabuf capture) and making CI actually compile the Linux-only hardware backends so they can't silently rot. Capture: new src/capture/v4l2.rs exports NV12 MMAP buffers as persistent dmabufs (REQBUFS -> EXPBUF -> QBUF/STREAMON -> DQBUF) and hands them to the VAAPI encoder zero-copy, re-queuing one frame late since the capture/encode loop is synchronous. It is opt-in: used only when VAAPI is chosen by name (backend::requires_dmabuf -> capture::open's want_dmabuf), never for Auto. That matters because the generic Linux release binary enables both nvenc and vaapi; making the vaapi feature unconditionally switch capture to dmabuf would break an NVIDIA-only box, where neither NVENC nor openh264 can consume a dmabuf frame. The CPU webcam + software/NVENC path is unchanged. CI: cargo check --workspace --all-features (run by `just rs ci`) was broken for these backends. nvidia-video-codec-sdk's build.rs panics without libnvidia-encode.so, so a new `nvenc-ci` feature enables its `ci-check` escape hatch (additive, so it also neutralizes the panic under --all-features), and libva is added to the nix devShell so cros-libva's build script can probe it. Both backends now type-check on Linux. Compile-verified on Linux (cargo check + clippy -D warnings) via a podman aarch64 VM; the V4L2 capture and the VAAPI encode path are not yet runtime-tested (need a real /dev/video* + VAAPI device). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cros-codecs 0.0.6 (the latest) caret-pins cros-libva 0.0.12, which does not compile against libva >= 2.23: newer headers add seg_id_block_size / va_reserved8 to VAEncPictureParameterBufferVP9 and cros-libva 0.0.12's struct literal omits them. cros-libva 0.0.13 fixes it but cros-codecs 0.0.6 won't accept it. Modern distros and the nix toolchain ship libva 2.23+, so the VAAPI backend (added in #1691) could neither build nor ship, and CI's --all-features could not be made green while the `vaapi` feature pulled this dependency. This only surfaced now because wiring the hardware backends into CI is what first compiled VAAPI against a real, modern libva. cros-codecs is the only pure-Rust VAAPI H.264 encoder and reintroducing ffmpeg for VAAPI defeats this effort, so VAAPI is parked until cros-codecs releases against cros-libva >= 0.0.13. Intel/AMD Linux falls back to the openh264 software encoder; NVIDIA uses NVENC; macOS uses VideoToolbox. Removed: backend/vaapi.rs, capture/v4l2.rs (the dmabuf capture that only fed VAAPI), the `vaapi` feature and its cros-codecs/v4l/libc deps, Frame::DmaBuf and the requires_dmabuf capture coupling, and libva from the nix devShell. Kept from this PR's original scope: the `nvenc-ci` feature, which enables nvidia-video-codec-sdk/ci-check so `cargo check --workspace --all-features` type-checks the NVENC backend on Linux without an NVIDIA driver (its build.rs otherwise panics looking for libnvidia-encode.so). NVENC remains compile-verified only; runtime validation still needs a Linux + GPU box. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kixelated
changed the title
`cargo deny` (run by `just rs ci`) rejected the IJG license carried by mozjpeg-sys, which nokhwa pulls for MJPEG decode in moq-video. This was latent: `just rs ci` runs `cargo check --workspace --all-features` before `cargo deny`, and that check failed at compile time until now, so the deny step was never reached. With the hardware backends compiling, deny runs and surfaces it. IJG (Independent JPEG Group) is a standard permissive license; add it to the allow list. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kixelated
enabled auto-merge (squash)
June 13, 2026 05:06
`just rs ci` ran `cargo check/test --workspace --all-features`, which force-enables moq-video's `nvenc` feature. That can't work in a GPU-less CI: the SDK's build.rs panics without libnvidia-encode.so (cargo check), and even with the `ci-check` escape hatch a test/build still fails to link its NVENC symbols. This is why --all-features has been red since NVENC landed (#1691). Exclude moq-video from the workspace-wide --all-features check and test, and cover it explicitly instead: a Linux-only check-only pass with `nvenc-ci` (ci-check skips the link), plus `cargo test -p moq-video` on default features. moq-video is still built (without nvenc) as moq-cli's dependency, so its default path keeps full coverage; only the vendor-lib encoder feature is carved out of the blanket runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Collaborator
Author
|
CI note: the remaining red Check is a pre-existing libmoq bug, not this PR.
The last failure is a SIGSEGV in libmoq's tests ( (Written by Claude) |
4 tasks
Replaces nokhwa's Linux webcam path with a native V4L2 capture (src/capture/v4l2.rs) built on the `v4l` crate's MMAP streaming. It negotiates YUYV or MJPEG (the formats essentially all UVC webcams expose) and converts each frame to CPU I420 for the encoder: YUYV is resampled 4:2:2 -> 4:2:0 directly, MJPEG is decoded with the pure-Rust `zune-jpeg` (MIT/Apache/Zlib, no libjpeg/IJG) then converted. The stream owns the device's Arc<Handle>, so no self-referential borrow. nokhwa is now Windows-only (Media Foundation), kept until a native MF capture lands; macOS already uses AVFoundation/ScreenCaptureKit. This drops nokhwa, mozjpeg, and image from the Linux and macOS builds (they remain in the lockfile only for the Windows target). frame.rs gains `from_rgb` (MJPEG path) and `from_yuyv` (raw path) alongside the existing `from_rgba`, sharing a small `pack` helper; the two new ones are Linux-gated since only the V4L2 capture uses them. Capture is compile-verified on Linux (clippy -D warnings) but not runtime-tested (no camera in CI / the sandbox); the V4L2 format negotiation and YUYV/MJPEG conversions need a real /dev/video* device to confirm. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kixelated
changed the title
Collaborator
Author
|
CI update: the moq-video/plumbing work here now compiles and passes everything it should. With the libmoq fix from
This is the same category as the libmoq segfault and is being fixed separately on (Written by Claude) |
kixelated
disabled auto-merge
June 13, 2026 21:33
2 tasks
NVENC now dlopens everything at runtime, so an `--features nvenc` binary links
on a GPU-less builder and loads on machines without the NVIDIA driver, falling
back to software. This unblocks shipping capture/encode in a portable moq-cli,
and lets CI build/link/test NVENC normally instead of the previous ci-check
compile-only workaround.
- `nvidia-video-codec-sdk/dynamic-loading`: a small fork feature (root
[patch.crates-io] -> kixelated/nvidia-video-codec-sdk) that dlopens
libnvidia-encode for the two entry points the SDK uses to build its function
table, and skips the link directives in build.rs. Upstream + drop the patch
once merged.
- cudarc 0.16 -> 0.19 with `fallback-dynamic-loading` (dlopen libcuda) +
`cuda-12020` (pin the CUDA API version so no toolkit is needed at build).
DriverError lost its Display impl in 0.19, so format it with `{e:?}`.
- `just rs ci`: drop the ci-check incantation; build + test moq-video with real
`--all-features` on Linux (still excluded from the workspace --all-features
because the SDK has no macOS bindings).
- deny.toml allows the fork git source; DESIGN + the feature comment updated.
Verified on a GPU-less Linux box (podman): builds, links, `readelf -d` shows no
libnvidia-encode / libcuda DT_NEEDED, and `cargo test -p moq-video --features
nvenc` passes (falls back to openh264). Real-GPU encode still needs a Linux+GPU
box.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
kixelated
changed the title
This was referenced Jun 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
moq-video native-capture/codec overhaul, continuing the drop-the-abstraction-crates effort (ffmpeg already gone). Compile-verified on Linux via podman; runtime-validated where possible without a GPU.
Changes
NVENC via dlopen — portable + shippable. NVENC now dlopens everything at runtime (
cudarc/fallback-dynamic-loadingfor libcuda;nvidia-video-codec-sdk/dynamic-loadingfor libnvidia-encode — a small fork feature, see root[patch.crates-io]). So an--features nvencbinary links on a GPU-less builder and loads on machines without the NVIDIA driver, falling back to software. Verified:readelf -dshows nolibnvidia-encode/libcudaDT_NEEDED, andcargo test --features nvencpasses on a GPU-less box. cudarc bumped 0.16 → 0.19. This is the prerequisite for shipping capture/encode in a portablemoq-cli.Native V4L2 capture, replacing nokhwa on Linux.
capture/v4l2.rsstreams MMAP buffers via thev4lcrate, negotiates YUYV or MJPEG, and converts to CPU I420 (MJPEG decoded with pure-Rustzune-jpeg— no IJG/libjpeg). nokhwa is now Windows-only (until a native Media Foundation capture lands); macOS already uses AVFoundation/ScreenCaptureKit.Drop VAAPI (cros-codecs).
cros-codecs 0.0.6/cros-libva 0.0.12don't compile against libva ≥ 2.23 (modern distros + nixpkgs). Removed the backend + its dmabuf capture; parked untilcros-codecssupportscros-libva ≥ 0.0.13. Intel/AMD Linux uses openh264 meanwhile.CI.
just rs cibuilds + tests moq-video with real--all-featureson Linux (dlopen needs no GPU/driver/flags); excluded from the workspace--all-featuresonly because the NVENC SDK has no macOS bindings.IJGallowed indeny.toml(mozjpeg via nokhwa, Windows), plus the fork git source. Includes a merge ofdevfor the libmoq test fix (#1710).Test plan
-D warningsdefault +--all-features;cargo test -p moq-video --features nvenc(7 pass, NVENC falls back);readelfconfirms no driverDT_NEEDEDcargo fmt/sort/shear/doc/denygreen/dev/video*; NVENC encode on a Linux+GPU boxFollow-ups (
DESIGN-native-codecs.md)nvidia-video-codec-sdkdynamic-loadingfeature; drop the fork patch.cros-codecssupports modern libva.(Written by Claude)