feat(moq-video,libmoq): native H.264 decode (drop ffmpeg dependency)#1796
Merged
Conversation
Add a native H.264 decode path mirroring the existing native encode side, so consumers (the OBS plugin) no longer need ffmpeg to decode subscribed video. - moq-video: new `decode` module mirroring `encode`. `decode::Consumer` (the counterpart to moq-audio's AudioConsumer) subscribes to a moq-mux H.264 track and returns raw I420 frames. Backends are VideoToolbox (macOS) and openh264 (portable software fallback). avc1 payloads are converted to Annex-B with SPS/PPS injected ahead of keyframes; avc3 passes through. - libmoq: new `moq_consume_video_raw` (+ `_close`/`_frame`/`_frame_free`) C API, the video counterpart to `moq_consume_audio_raw`. Decoding now happens inside the staticlib, with no ffmpeg. Only H.264 is supported (symmetric with what moq-video encodes); a non-H.264 rendition surfaces UnsupportedCodec. Verified on macOS: encode/decode round-trips for both backends, plus an end-to-end libmoq test that publishes real H.264 and decodes it via the C API. The OBS plugin rewrite + CI/docs (removing ffmpeg from cpp/obs) is a follow-up on top of #1787. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…module `cargo doc -D warnings` rejects the intra-doc link (rustdoc::private-intra-doc-links); drop it to plain text. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…coder Replace the hand-rolled Annex-B start-code scanner (and its trailing-zero heuristic) in the VideoToolbox backend with `moq_mux::codec::annexb::NalIterator`, the same tested helper the rest of the codebase uses. The backend `decode` now takes an owned `Bytes`, so NAL slices (incl. SPS/PPS) come out zero-copy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts: # rs/libmoq/CHANGELOG.md
kixelated
added a commit
that referenced
this pull request
Jun 20, 2026
…-32ccb0 Brings the branch up to date with dev (HEVC #1802, native H.264 decode #1796, async device capture #1807, srt/rtmp/rtc egress) and keeps this PR's deltas: NVENC/VAAPI always-on for Linux, openh264 always-on (no `software` feature), publish=false removed, the nvidia-video-codec-sdk fork repinned to the hardened dynamic-loading commit (default-features off), and the NVIDIA-probe fallback. Note: restores dev's H.265/HEVC codec-aware encode path (Codec enum, codec-aware backend selection, hev1 producer) that the previous branch merge had dropped by taking "ours" wholesale. Verified: clippy -D clean, 19 moq-video tests pass (incl. H.265 roundtrips), sort/taplo/fmt clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a native H.264 decode path mirroring moq-video's existing native encode side, and exposes it through libmoq. This is the symmetric counterpart to how audio already works (
moq-audiodecodes Opus natively →moq_consume_audio_raw), and the prerequisite for removing ffmpeg from the in-tree OBS plugin (cpp/obs), which currently decodes subscribed video withlibavcodec+libswscale.This is Phase 1 (the Rust side) of that effort. The OBS plugin rewrite + CI/docs is a follow-up on top of #1787 (it can't compile against this API until it lands).
rs/moq-video— newdecodemodule (mirror ofencode)decode::Consumer— the direct counterpart tomoq_audio::AudioConsumer: subscribes to a moq-mux H.264 track and returns raw packed-I420decode::Frames.videotoolbox— macOS hardware (VTDecompressionSession, hand-written on theobjc2-video-toolboxbindings).openh264— portable software fallback (the path that lets a Linux build ship without ffmpeg).decode::Decoderdoes the access-unit prep: avc1 (length-prefixed + out-of-band avcC) → Annex-B with SPS/PPS injected ahead of keyframes (reusingmoq_mux::codec::{annexb,h264}helpers); avc3 passes through. Keyframe-gated.Error::UnsupportedCodec. Native decode of HEVC/VP9/AV1/VP8 (which ffmpeg did) is dropped.rs/libmoq— new video-decode C APImoq_consume_video_raw(+_close/_frame/_frame_free), the video twin ofmoq_consume_audio_raw. Decoding happens inside the staticlib, so callers get a native decoder with no ffmpeg. cbindgen auto-emits the new types/functions intomoq.h.Public API changes
rs/moq-video(additive): newpub mod decodewithdecode::{Consumer, Config, Kind, Frame}; newError::{NoDecoder, UnsupportedCodec}variants (enum is#[non_exhaustive]). No existing signatures changed.rs/libmoq(additive C ABI):moq_consume_video_raw{,_close,_frame,_frame_free},moq_video_frame,moq_video_decoder_output; newError::Videovariant (code-36).Both are additive, so non-breaking, but this targets
devper the branch-targeting rules (touchesrs/moq-video/rs/libmoqpublic surface).Test plan
decoderound-trip unit tests (openh264 encode → decode) for both backends, incl. a macOS-gated VideoToolbox test.video_raw_decode): publishes real H.264 and decodes it through the C API, asserting a 320×240 packed-I420 frame.cargo clippyclean,cargo fmt/taploclean (vianix develop).cargo check --workspace --all-targetsgreen.Follow-up (out of scope here)
cpp/obs/src/moq-source.cppontomoq_consume_video_raw(hand OBSVIDEO_FORMAT_I420directly, drop avcodec/swscale), strip ffmpeg fromcpp/obs/CMakeLists.txt, re-enable a portable Linux binary in theobs-buildmatrix, and updatedoc/bin/obs.md.🤖 Generated with Claude Code
(Written by Claude)