Experimental MoQ server input. by JBRS307 · Pull Request #1998 · software-mansion/smelter

JBRS307 · 2026-05-18T14:17:27Z

Experimental MoQ server input #2009

Based on the moq-lite crate, however adds a few more dependencies for QUIC server and CMAF handling.

Currently supports only H264 video and AAC audio.

Yet to be done:

Properly close all sessions on /reset
Properly close session in input removal
Sequential catalog handling
Refactor for new moq-mux version

What will NOT be done in the experimental:

Handling of catalog changes (catalog currently read only once per session)
Handling other containers than CMAF

Copilot

Pull request overview

This PR introduces an experimental MoQ (Media over QUIC) input path (based on moq-lite) by adding a WebTransport server to smelter-core, wiring it into pipeline initialization/config, exposing a new moq_server input type in the public API, and surfacing MoQ in status/stats and integration-test demos.

Changes:

Add MoQ server + pipeline integration (WebTransport accept loop, origin announce loop, input registration, and broadcast-to-queue decode path for H264/AAC).
Extend configuration + pipeline options to enable/disable MoQ server and configure port/TLS.
Add API models/conversions + status/stats reporting for MoQ inputs, plus integration-test/demo support.

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
`src/state.rs`	Plumbs MoQ server options into `PipelineOptions` derived from config.
`src/routes/status.rs`	Adds `"moq"` protocol label in status output.
`src/routes/register_request.rs`	Adds `moq_server` input registration route handling via `MoqInput`.
`src/config.rs`	Adds MoQ env-driven config (port/enable/TLS cert+key).
`smelter-core/src/stats/input/moq.rs`	Implements MoQ input stats state/events (bitrate tracking).
`smelter-core/src/stats/input/mod.rs`	Registers MoQ stats state/event wiring and report mapping.
`smelter-core/src/stats/input_reports.rs`	Adds serializable MoQ stats report structs + enum variant.
`smelter-core/src/protocols/moq.rs`	Defines MoQ input options and MoQ-specific error types.
`smelter-core/src/protocols.rs`	Exposes the new `protocols::moq` module.
`smelter-core/src/pipeline/moq/state.rs`	Tracks registered MoQ inputs and active broadcast connection handles.
`smelter-core/src/pipeline/moq/server.rs`	Implements MoQ WebTransport server startup, accept loop, and announce loop.
`smelter-core/src/pipeline/moq/moq_input.rs`	Implements MoQ server input registration and teardown hooks.
`smelter-core/src/pipeline/moq/mod.rs`	Declares MoQ pipeline submodules and re-exports.
`smelter-core/src/pipeline/moq/connection.rs`	Implements catalog parsing, track subscription, decoding, queue feeding, and stats events.
`smelter-core/src/pipeline/instance.rs`	Instantiates MoQ pipeline state and starts/stores MoQ server handle.
`smelter-core/src/pipeline/input.rs`	Adds MoQ as a pipeline input variant and external input constructor branch.
`smelter-core/src/pipeline.rs`	Adds `PipelineMoqServerOptions` + threads MoQ state through `PipelineCtx`.
`smelter-core/src/input.rs`	Adds MoQ to `RegisterInputOptions` and `InputProtocolKind`.
`smelter-core/src/error.rs`	Adds MoQ server init error and MoQ input init error variants.
`smelter-core/Cargo.toml`	Adds MoQ/WebTransport/CMAF-related dependencies to core crate.
`smelter-api/src/input/moq.rs`	Adds public API schema for `MoqInput`.
`smelter-api/src/input/moq_into.rs`	Adds conversion from API `MoqInput` into core `RegisterInputOptions`.
`smelter-api/src/input.rs`	Re-exports the new MoQ input API types.
`integration-tests/src/bin/benchmark/utils.rs`	Updates benchmark pipeline options to include MoQ server option.
`integration-tests/examples/demo/smelter_state.rs`	Adds MoQ input flow to the interactive demo app.
`integration-tests/examples/demo/inputs/moq.rs`	Adds a demo MoQ input builder + registration JSON + publish instructions.
`integration-tests/examples/demo/inputs.rs`	Registers MoQ in demo input selection and handle enum.
`Cargo.toml`	Adds workspace dependency versions for MoQ/WebTransport/TLS helpers.
`Cargo.lock`	Locks newly introduced transitive dependencies.

Comments suppressed due to low confidence (1)

smelter-core/src/pipeline/moq/connection.rs:332

process_audio_config will panic if the catalog's AAC rendition has description: None (it calls .expect(...)). Since description is optional in the catalog, this should be handled as an error (e.g., return a dedicated MoqConnectionError or treat the track as unsupported/missing config) instead of crashing the broadcast task.

    let aac_decoder_options = {
        let asc = audio
            .description
            .as_ref()
            .expect("process_audio_config called with description present")
            .clone();
        AudioDecoderOptions::FdkAac(FdkAacDecoderOptions { asc: Some(asc) })
    };

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+        .description
+        .clone()
+        .ok_or(MoqConnectionError::UnsupportedVideoCodec)?;
+    let h264_config = H264AvcDecoderConfig::parse(avcc_bytes).unwrap();


JBRS307 · 2026-05-19T09:23:13Z

+    let video = match catalog.video.renditions.first_key_value() {
+        Some((name, config)) if let HangVideoCodec::H264(_) = config.codec => {
+            match Hang::try_from(&config.container) {
+                Ok(container) => Some(DiscoveredVideo {
+                    name: name.clone(),
+                    container,
+                    description: config.description.clone(),
+                }),
+                Err(error) => {
+                    warn!(track=%name, "Unsupported video container, skipping: {error}");
+                    None
+                }
+            }
+        }
+        Some((name, config)) => {
+            warn!(track=%name, codec=%config.codec, "Unsupported video codec, skipping track");
+            None
+        }
+        None => None,
+    };
+
+    let audio = match catalog.audio.renditions.first_key_value() {
+        Some((name, config)) if let HangAudioCodec::AAC(_) = config.codec => {
+            match Hang::try_from(&config.container) {
+                Ok(container) => Some(DiscoveredAudio {
+                    name: name.clone(),
+                    container,
+                    description: config.description.clone(),
+                }),
+                Err(error) => {


Aware of that, this code is not meant to be production ready for now

+    pub fn on_after_registration(&self) -> Result<()> {
+        println!("Publish to this input using moq-cli:");
+        println!(
+            "  moq-cli publish https://localhost:4443/{}",


+    let aac_decoder_options = {
+        let asc = audio
+            .description
+            .as_ref()
+            .expect("process_audio_config called with description present")
+            .clone();
+        AudioDecoderOptions::FdkAac(FdkAacDecoderOptions { asc: Some(asc) })
+    };
+
+    match aac_decoder_options {
+        AudioDecoderOptions::FdkAac(decoder_options) => {
+            let options = AudioDecoderThreadOptions {
+                ctx: ctx.clone(),
+                decoder_options,
+                samples_sender,
+                input_buffer_size: MOQ_MAX_BUFFER,
+            };
+            AudioDecoderThread::<FdkAacDecoder>::spawn(input_ref.clone(), options)
+                .map_err(MoqConnectionError::InitAudioDecoder)
+        }
+        _ => Err(MoqConnectionError::UnsupportedAudioCodec),
+    }


JBRS307 · 2026-05-18T14:27:41Z

+        // XXX: I am not sure if this is the best solution from semantic POV, using `rtmp::TlsConfig` in
+        // MoQ
+        tls_config: Option<TlsConfig>,


That is a good question

agree, create separate type

JBRS307 · 2026-05-18T14:28:40Z

All stats related code should be put to another PR, remove it from here.

wkozyra95 · 2026-05-18T14:45:09Z

+        // XXX: I am not sure if this is the best solution from semantic POV, using `rtmp::TlsConfig` in
+        // MoQ
+        tls_config: Option<TlsConfig>,


agree, create separate type

wkozyra95 · 2026-05-18T14:51:32Z

+        Err(_) => true,
+    };
+
+    let moq_tls_cert_file = match env::var("SMELTER_MOQ_TLS_CERT_FILE") {


does the moq server even make sense without certs? If no then you can skip enable option, by default do not enable it unless certs are provided

wkozyra95 · 2026-05-18T14:54:58Z

+    // HACK: This is needed because even though `.with_certificate()` is sync it runs async code
+    // underneath and requires a runtime


It's worth preserving the comment that with_certificate requires runtime, but instead of hack, I would just make this function async

wkozyra95 · 2026-05-18T14:59:33Z

+    }
+}
+
+fn start_wt_server(


I don't know what wt is

wkozyra95 · 2026-05-18T15:03:49Z

+        tokio::spawn(async move {
+            let session = match request.ok().await {
+                Ok(session) => {
+                    info!("WebTransport session created.");


Is it correct to refer to web transport? My understanding is that WebTransport is a browser API for QUICK, but if you run outside of browser it does not have anything to do with it, you just use quick then.

Copilot

Pull request overview

Copilot reviewed 27 out of 28 changed files in this pull request and generated 4 comments.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

It was a leftover from before branch split

JBRS307 requested a review from Copilot May 18, 2026 14:17

JBRS307 self-assigned this May 18, 2026

JBRS307 force-pushed the @jbrs/moq branch from c4e9397 to 82d1bf4 Compare May 18, 2026 14:17

Copilot started reviewing on behalf of JBRS307 May 18, 2026 14:18 View session

Copilot AI reviewed May 18, 2026

View reviewed changes

JBRS307 commented May 18, 2026

View reviewed changes

Comment thread Cargo.toml Outdated

JBRS307 commented May 18, 2026

View reviewed changes

JBRS307 requested a review from wkozyra95 May 18, 2026 14:35

wkozyra95 reviewed May 18, 2026

View reviewed changes

JBRS307 force-pushed the @jbrs/moq branch 13 times, most recently from f30de37 to 33d1397 Compare May 20, 2026 12:07

JBRS307 requested a review from Copilot May 20, 2026 12:13

Copilot started reviewing on behalf of JBRS307 May 20, 2026 12:14 View session

Copilot AI reviewed May 20, 2026

View reviewed changes

Comment thread smelter-core/src/stats/input/mod.rs

Comment thread smelter-core/src/pipeline/moq/state.rs Outdated

Comment thread smelter-core/src/pipeline/moq/connection.rs

Comment thread smelter-core/src/pipeline/moq/connection.rs Outdated

JBRS307 force-pushed the @jbrs/moq branch 2 times, most recently from 3730c61 to f641055 Compare May 20, 2026 15:12

JBRS307 mentioned this pull request May 21, 2026

Experimental MoQ server input #2009

Open

3 tasks

JBRS307 force-pushed the @jbrs/moq branch from f641055 to 8bcb9ee Compare May 21, 2026 10:35

JBRS307 and others added 28 commits May 28, 2026 16:09

Robotic CR suggestions

b78d58d

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Removed new input stats event on new input

bca44c9

It was a leftover from before branch split

Added retry to the moq server startup

77f1fc9

Fixed /reset

d1fba56

Sequential hang and msf catalog handling

fc7b138

Better demo command

a3de47f

Openapi schema regen

f769a27

Removing session from MoQ server state after it is closed

375d544

Got rid of the direct moq-lite dependency. Using reexports

3c9acad

Catalog handling comment

74f65a0

Removed support for Legacy container for moq-msf

2596137

Removed support for Hang::Legacy container

987e6fe

better warn! log

fb12b96

Removed redundant loop in MoqServer::drop()

0332b4d

Disabled MoQ server in integration tests instance

ad78907

Changed type in the demo

6b4430d

Renamed files in smelter-api to moq_server

907ee34

broadcast_path parameter must be unique

676a6b1

Fixed enum variant InputProtocolKind name from Moq to MoqServer

93c4782

Renamed moq_input to moq_server_input

eb1b950

Removed required option from demo for moq input

65069c6

WIP: Updated dependencies, adjusted hang catalog reading

3ef3d2c

Finished new way of catalog parsing

39b775b

Parsing codec description from init bytes for msf catalog

c505baa

Changed drain to clear

f7669e9

Changed latency and offset values

20cedc3

Bringing back baseline implementation

18f8520

Decoder map force ffmpeg

1bf69d2

JBRS307 force-pushed the @jbrs/moq branch from 5b2708c to 1bf69d2 Compare May 28, 2026 14:09

Side channel

1fedc30

		// HACK: This is needed because even though `.with_certificate()` is sync it runs async code
		// underneath and requires a runtime

Conversation

JBRS307 commented May 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JBRS307 commented May 18, 2026 •

edited

Loading