Skip to content

feat(moq-video): reintroduce VAAPI via discord/cros-codecs + NV12 surface upload#1720

Merged
kixelated merged 2 commits into
devfrom
claude/youthful-mirzakhani-2a98f3
Jun 14, 2026
Merged

feat(moq-video): reintroduce VAAPI via discord/cros-codecs + NV12 surface upload#1720
kixelated merged 2 commits into
devfrom
claude/youthful-mirzakhani-2a98f3

Conversation

@kixelated

@kixelated kixelated commented Jun 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reintroduces the Intel/AMD VAAPI H.264 encoder for moq-video, dropped in #1704 because cros-codecs 0.0.6 pins cros-libva 0.0.12, which fails to compile against libva >= 2.23.

The unblock is discord/cros-codecs (branch discord-0.0.5), an actively-maintained fork (Discord ships it for Go Live) that bumps to cros-libva 0.0.13 and hardens the H.264 VAAPI encoder (packed SPS/PPS + slice headers, frame_num, rate control). It's consumed as plain git dependencies, with no fork of our own.

  • dlopen, like NVENC. The vaapi feature enables cros-codecs' vaapi_dlopen, which loads libva at runtime via the cros-libva dlopen feature (discord/cros-libva#discord-0.0.13, pulled through a root [patch.crates-io]). So a vaapi-enabled binary links on a libva-less builder and loads on a libva-less machine, falling back to openh264. The build still needs libva headers for bindgen, so libva is back in the nix devShell.
  • Input is an NV12 surface upload, not zero-copy dmabuf. VAAPI's H.264 encoder wants an NV12 VA surface, but UVC webcams deliver YUYV/MJPEG (decoded to CPU I420) and rarely offer NV12 to import zero-copy. So backend/vaapi.rs drives new_native_vaapi with a VaSurfacePool and uploads each I420 frame into a pooled VA surface as NV12. This works with the existing CPU V4L2 capture, no new capture code.

Zero-copy dmabuf (Frame::DmaBuf + a VIDIOC_EXPBUF capture + DrmPrime2 surface import) for the rare NV12-capable V4L2 source is a deliberate follow-up.

Public API

No public API or wire change. The vaapi feature is off by default and Linux-gated; the codec-agnostic moq-video surface (Encoder, Config, Kind) is untouched. Additive only.

Test plan

  • Default macOS build green (cargo check -p moq-video; vaapi is cfg'd out).
  • Git deps resolve and lock to the discord forks; the cros-libva patch applies (no unused-patch warning).
  • rustfmt clean (incl. the cfg-gated vaapi.rs).
  • Linux cargo clippy -p moq-video --all-targets --all-features -- -D warnings green (nvenc + vaapi both active), verified in a Linux container. Caught one missing field (FrameMetadata.force_idr), now fixed.
  • cargo deny check licenses sources ok (the new transitive licenses are already allow-listed).
  • Hardware validation on Linux + Intel/AMD GPU still pending: the low_power entrypoint (recent Intel iHD may need true), and the NV12 upload pitch handling round-trip.

Targets dev because it adds git-patched dependencies and belongs to the native-codecs effort.

(Written by Claude)

kixelated and others added 2 commits June 13, 2026 21:47
…face upload

VAAPI was dropped in #1704 because cros-codecs 0.0.6 pins cros-libva 0.0.12,
which fails to compile against libva >= 2.23. Reintroduce it on top of
discord/cros-codecs (branch discord-0.0.5), an actively-maintained fork that
bumps to cros-libva 0.0.13 and hardens the H.264 VAAPI encoder (packed SPS/PPS +
slice headers, frame_num, rate control). Consumed as plain git dependencies, no
fork of our own.

- vaapi feature -> cros-codecs `vaapi_dlopen`, which loads libva at runtime via
  the cros-libva `dlopen` feature (discord/cros-libva#discord-0.0.13, pulled
  through a root [patch.crates-io]). So a vaapi-enabled binary links without
  libva and loads on machines without it, falling back to openh264. The build
  still needs libva headers for bindgen, so libva is back in the nix devShell.
- Input is an NV12 surface upload, not zero-copy dmabuf: webcams deliver
  YUYV/MJPEG (decoded to CPU I420) and rarely offer NV12 to import zero-copy, so
  backend/vaapi.rs drives new_native_vaapi with a VaSurfacePool and uploads each
  I420 frame into a pooled VA surface as NV12. This works with the existing CPU
  V4L2 capture, no new capture code.
- deny.toml allows both discord git sources; doc/bin/cli.md and the design doc
  note VAAPI on Linux Intel/AMD.

Zero-copy dmabuf (Frame::DmaBuf + a VIDIOC_EXPBUF capture + DrmPrime2 surface
import) for the rare NV12-capable V4L2 source is a follow-up.

Compiles on Linux with libva headers (covered by CI --all-features); the default
macOS build is unaffected (vaapi is cfg'd out). Not yet validated on Intel/AMD
hardware: the low_power entrypoint, the NV12 upload pitch handling, and the new
transitive licenses still need a real GPU pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
discord/cros-codecs' FrameMetadata has both force_keyframe and force_idr; only
the former was set, so the backend failed to compile on Linux. Force an IDR
(not just a keyframe) when a keyframe is requested, so a re-subscribing viewer
gets a clean random-access point with in-band SPS/PPS.

Caught by a Linux compile-check; clippy --all-features -D warnings now passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kixelated
kixelated merged commit d0a00fb into dev Jun 14, 2026
1 check failed
@kixelated
kixelated deleted the claude/youthful-mirzakhani-2a98f3 branch June 14, 2026 04:53
kixelated added a commit that referenced this pull request Jun 14, 2026
Dev landed #1720 (VAAPI reintroduction), which reworked the same moq-video
files. The merge is clean: the Windows Media Foundation encoder and the Linux
VAAPI encoder sit on separate cfg-gated candidate lines, and the Frame enum
gains both Surface/Texture without collision.

Re-ran taplo afterward, which normalized two column-width-150 arrays the merge
surfaced: deny.toml's allow-git list (collapsed) and the cros-codecs dependency
features in moq-video's Cargo.toml (expanded).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
kixelated added a commit that referenced this pull request Jun 14, 2026
…ecs git dep

dev's #1720 added cros-codecs as a git dependency for the `vaapi` feature, which
cargo-deny's wildcard lint bans for published crates (crates.io disallows git
deps). moq-video can't be published to crates.io with that dep anyway, so
`publish = false` is the honest fix and lets `allow-wildcard-paths` exempt the
dep. Revert once the cros-codecs fork is published to crates.io under its own
name.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant