diff --git a/CLAUDE.md b/CLAUDE.md index e0f4fa621..a89b39bd9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,64 +39,29 @@ Key architectural rule: The CDN/relay does not know anything about media. Anythi ## Project Structure -``` -/rs/ # Rust crates - moq-net/ # Core networking layer (published as moq-net; negotiates moq-lite or moq-transport) - moq-native/ # QUIC/WebTransport connection helpers for native apps; clock example lives in examples/clock.rs - moq-relay/ # Clusterable relay server (binary: moq-relay) - moq-token/ # JWT authentication library - moq-token-cli/ # JWT token CLI tool (binary: moq-token-cli) - moq-cli/ # CLI tool for media operations (binary: moq) - moq-bench/ # Load generator for benchmarking relays (binary: moq-bench) - moq-mux/ # Media muxers/demuxers (fMP4, CMAF, HLS) - moq-audio/ # Native PCM ↔ Opus encode/decode on top of moq-mux - hang/ # Media encoding/streaming (catalog/container format) - libmoq/ # C bindings (staticlib) - moq-ffi/ # UniFFI bindings for Python/Swift/Kotlin (cdylib + staticlib) - moq-boy/ # MoQ Boy emulator publisher (binary: moq-boy) - moq-gst/ # GStreamer plugin (moqsink/moqsrc elements) - -/js/ # TypeScript/JavaScript packages - net/ # Core networking layer for browsers (published as @moq/net) - signals/ # Reactive signals library (published as @moq/signals) - token/ # JWT token generation (published as @moq/token) - clock/ # Clock example (published as @moq/clock) - hang/ # Core media layer: catalog, container, support (published as @moq/hang) - watch/ # Watch/subscribe to streams + UI (published as @moq/watch) - publish/ # Publish media to streams + UI (published as @moq/publish) - moq-boy/ # MoQ Boy web viewer (published as @moq/boy) - -/py/ # Python packages (uv workspace) - moq-ffi/ # Maturin project: rs/moq-ffi cdylib + uniffi bindings. - # Distribution `moq-ffi` (PyPI); import `moq_ffi`. One - # wheel covers every crate exposed via moq-ffi because - # uniffi-linked libraries can't be split across separate - # wheels. Version tracks rs/moq-ffi (release-py-ffi.yml - # fires on moq-ffi-v* tags). Most callers want `moq-rs`. - moq-rs/ # Pure-python ergonomic wrapper. Distribution `moq-rs` - # (PyPI, since `moq` is taken); import `moq`. Depends on - # moq-ffi via a compatible-release pin (~=0.2.x) so it - # floats to the latest moq-ffi patch. Versioned - # independently: bump py/moq-rs/pyproject.toml by hand; on - # merge to main release-py.yml publishes to PyPI if that - # version isn't already there (registry is the gate). - -/swift/ # Swift wrapper over rs/moq-ffi (SwiftPM) -/kt/ # Kotlin wrapper over rs/moq-ffi (Gradle, KMP) -/go/ # Go wrapper over rs/moq-ffi (uniffi-bindgen-go) - # swift/kt/go are in-tree source skeletons. - # CI mirrors them to moq-dev/moq-{swift,kotlin,go} - # on each moq-ffi-v* tag. - -/demo/ # Demos and test media - boy/ # MoQ Boy demo (ROM hosting, orchestration justfile) - relay/ # Relay server configs (relay.toml, root.toml, leaf*.toml) - pub/ # Media hosting (vid.moq.dev) - web/ # Web demo (watch/publish examples) - throttle/ # Network throttle script for testing - -/doc/ # Documentation site (VitePress, deployed via Cloudflare) -``` +Top-level layout only. Per-crate and per-package detail lives in the nested guides (see [Per-Directory Guides](#per-directory-guides)), which sit next to the code and don't rot here. + +- `/rs/` - Rust crates: core networking (`moq-net`), native helpers, the relay, CLIs, media muxing/codecs, and the FFI/C bindings. See `rs/CLAUDE.md`. +- `/js/` - TypeScript/JavaScript packages for the browser, published as `@moq/*`. See `js/CLAUDE.md`. +- `/py/`, `/swift/`, `/kt/`, `/go/` - language wrappers over `rs/moq-ffi` (see [Language Bindings](#language-bindings)). `/py/` has `py/CLAUDE.md`; the others defer to their `README.md`. +- `/demo/` - demos and test media: relay configs, the web demo, MoQ Boy, media hosting, and a network throttle script. +- `/doc/` - documentation site (VitePress, deployed via Cloudflare). + +## Language Bindings + +`rs/moq-ffi` is the single UniFFI core that every non-Rust binding is generated from. The wrappers under `/py`, `/swift`, `/kt`, and `/go` are thin layers over it, and `rs/libmoq` exposes the same core as a C staticlib. So one `moq-ffi` change ripples out to all of them (and their docs) per the [Cross-Package Sync](#cross-package-sync) table. CI mirrors the `swift`/`kt`/`go` source skeletons to `moq-dev/moq-{swift,kotlin,go}` on each `moq-ffi-v*` tag. For Python, most callers want the ergonomic `moq-rs` wrapper rather than the generated `moq-ffi` bindings directly. + +## Per-Directory Guides + +Language-specific conventions, crate/package maps, and patterns live in nested `CLAUDE.md` files that load automatically when you work under that directory. Before writing code in one of these areas, read its guide (your editor loads it for you, but check it explicitly if you are reasoning about the area without opening a file in it): + +- **`rs/CLAUDE.md`** - Rust workspace: crate map, Producer/Consumer model, `poll_*` plumbing, error handling, config/TOML merge, Version matching, testing. +- **`js/CLAUDE.md`** - TypeScript/JS workspace: package map, the signals + Effect reactivity model and its lifecycle rules, Web Components UI, `bun`/Biome tooling. +- **`py/CLAUDE.md`** - Python wrappers: the `moq-ffi` (generated bindings) vs `moq-rs` (ergonomic) split and the `moq` public surface. + +The `swift/`, `kt/`, and `go/` directories are thin wrappers over `rs/moq-ffi` (mirrored to external repos); see each directory's `README.md` rather than a dedicated guide. + +This root file holds only cross-cutting rules that apply everywhere (writing style, branch targeting, cross-package sync, public-API scrutiny, comment/doc conventions). ## Dependencies @@ -110,30 +75,10 @@ Key architectural rule: The CDN/relay does not know anything about media. Anythi 3. For JS/TS development, bun workspaces are used with configuration in the root `package.json` 4. Consult `doc/` for documentation and the [IETF datatracker](https://datatracker.ietf.org/doc/draft-lcurley-moq-lite/) for specification drafts when working on protocol-level code -## Version Matching Convention - -When matching on `Version` enums, default to the **newest** draft behavior so future versions default forward. Explicitly list older versions: - -```rust -// CORRECT: future versions get draft-17+ behavior -match version { - Version::Draft14 | Version::Draft15 | Version::Draft16 => { /* old behavior */ } - _ => { /* newest/draft-17 behavior */ } -} -``` - ## Writing Style - **No em dashes (—)** in code, comments, doc comments, commit messages, or any prose. Use a period and start a new sentence, or use a comma/parenthesis if the clauses are tightly bound. -## Rust Conventions - -- **Error handling**: Use `thiserror` with `#[from]` for library crates, `anyhow` for binaries. Always add `#[non_exhaustive]` to public `thiserror` enums. -- Use `anyhow::Context` (`.context("msg")`) instead of `.map_err(|_| anyhow::anyhow!("msg"))` for error conversion -- **Config flags + TOML merge**: For any `#[arg]` field on a TOML-loadable config, use `Option` (not bare `bool` / `String` / etc.). The TOML→CLI merge clobbers bare fields with their `Default` when the flag is absent, silently overwriting TOML values. See `rs/moq-relay/src/config.rs::tests` for the regression test; add one for any new flag. -- **Prefer `if let` / `let ... else` over an unwrapping `match`**: a `match` whose only job is to unwrap (`Ok(v) => v` / `Some(v) => v`) reads cleaner as `if let Some(v) = x { ... }` or `let Some(v) = x else { ... };`. Matching on an `Option`/`Result` just to bind the inner value is the tell. Keep `match` when both arms do real work or you need the `Err` / `None` payload. -- **`poll_*` plumbing**: a `Poll::Pending => Poll::Pending` arm usually means `ready!(...)` will collapse the match. And `.map_err(Into::into)` on a fallible result is usually better as `Ok(x?)` (the `?` does the `From` conversion). These compose: `let v = ready!(inner.poll_next(cx))?;` in a `fn -> Poll>` both unwraps the `Poll` and converts the error. - ## Comment Conventions - Keep things brief and avoid comments if the code is self-explanatory. Reserve comments for the non-obvious WHY: a hidden constraint, a subtle invariant, a workaround for a specific bug, behavior that would surprise a reader. This is about *implementation* comments inside function bodies and on private items. @@ -163,15 +108,12 @@ This applies whenever you add or widen a `pub` item, especially in library crate ## Tooling -- **TypeScript**: Always use `bun` for all package management and script execution (not npm, yarn, or pnpm) +Language-specific tooling (TypeScript/`bun`/Biome, JS async patterns, Web Components UI, Rust/`cargo`) lives in the per-directory guides. See [Per-Directory Guides](#per-directory-guides). + - **Common**: Use `just` for common development tasks -- **Rust**: Use `cargo` for Rust-specific operations -- **Formatting/Linting**: Biome for JS/TS formatting and linting -- **UI**: Plain Web Components in `@moq/watch/ui` and `@moq/publish/ui`, built directly on `@moq/signals` - **Builds**: Nix flake for reproducible builds (optional) - **Local-first**: When work can live in a `just` recipe (invoked via `nix develop --command`) or as logic in a GitHub Actions workflow step, prefer the recipe. The same code then runs reproducibly on a developer machine and in CI, and is debuggable locally without pushing commits. Workflow YAML should mostly delegate to `just`; reach for plugins (`dorny/paths-filter`, custom actions, etc.) only when a recipe genuinely can't express the logic. - **CI**: Prefer building release artifacts inside Nix (`nix build .#pkg`) over relying on runner-provided toolchains and `apt`/`brew` packages. Pinning the build environment in `flake.lock` makes artifacts deterministic and decouples them from drift in GitHub Actions runner images. Reach for the runner-native toolchain only when Nix doesn't fit (e.g. Windows runners). -- **JS async patterns**: Use `Effect.interval()`, `Effect.timer()`, and `Effect.event()` helpers from `@moq/signals` instead of raw `setInterval`, `setTimeout`, `addEventListener`. These handle cleanup automatically when the Effect is closed. ## Testing Approach @@ -225,7 +167,7 @@ When making changes to the codebase: ## PR Reviews -CodeRabbit reviews PRs automatically, but it has an hourly quota and runs out of org credits. If a PR shows a "Review limit reached" / "out of usage credits" message instead of an actual review, run the `/review` skill locally against the PR to get review feedback without waiting for the quota to refill. +CodeRabbit reviews PRs automatically, but it has an hourly quota and runs out of org credits. If a PR shows a "Review limit reached" / "out of usage credits" message instead of an actual review (or CodeRabbit otherwise fails to produce one), run the `/review` skill locally against the PR to get review feedback without waiting for the quota to refill. Then act on the findings the same way you would CodeRabbit's: push the high-confidence, unambiguous fixes directly, and escalate anything ambiguous, architectural, or open to interpretation by asking first rather than guessing. When reviewing a PR, always include a list of the public API changes (new/renamed/removed/signature-changed `pub` items in `rs/moq-*` and `js/*`), and call out anything that is breaking per [Branch Targeting](#branch-targeting). Distinguish genuinely public surface from `pub(crate)` / private items so the breaking-change and branch-targeting rules are applied to the right things. diff --git a/js/CLAUDE.md b/js/CLAUDE.md new file mode 100644 index 000000000..b47385532 --- /dev/null +++ b/js/CLAUDE.md @@ -0,0 +1,72 @@ +# js/CLAUDE.md + +Scopes the `/js` TypeScript/JavaScript workspace. Universal rules (writing style / no em-dashes, Branch Targeting, Cross-Package Sync, AI Attribution, Public API Scrutiny, Refactor As You Go, comment/doc conventions) live in the root `CLAUDE.md` and are not repeated here. + +## Workspace layout + +Bun workspaces; members listed in the repo-root `package.json` (not in `js/`). Deps hoist to the repo root `node_modules`, not into `js/`. Run recipes via `just js ` (see `js/justfile`). Packages, grouped by role (each mirrors its `rs/` counterpart where one exists), roughly in dependency order: + +**Foundation** +- `@moq/signals` (`signals/`): reactive core. `Signal`, `Computed`, `Effect`, plus framework adapters at subpaths `./solid`, `./react`, `./dom`. No deps on other workspace packages. Everything below uses it. + +**Transport / protocol** +- `@moq/net` (`net/`): browser networking. Connect to a relay, then publish/consume broadcasts/tracks/groups/frames over WebTransport (WebSocket fallback). Negotiates `moq-lite` (`lite/`) or IETF `moq-transport` (`ietf/`). Mirror of `rs/moq-net`. Optional `zod` peer dep for `./zod` JSON-frame helpers. + +**Container / catalog formats** +- `@moq/loc` (`loc/`): Low Overhead Container frame encoding. Thin layer on `@moq/net`. +- `@moq/json` (`json/`): snapshot/delta JSON over a track via RFC 7396 merge-patch. Exposes the base `Producer`/`Consumer` that `@moq/hang`'s catalog extends. +- `@moq/msf` (`msf/`): MOQT Streaming Format catalog types (zod schemas). + +**Media** +- `@moq/hang` (`hang/`): WebCodecs media layer. Subpaths `./catalog`, `./container`, `./util`. Mirror of `rs/hang`. Catalog is a JSON track describing other tracks; container frames are timestamp + codec bitstream (CMAF under `container/cmaf`). +- `@moq/watch` (`watch/`): subscribe + decode + render, with optional UI. Subpaths `.`, `./element`, `./ui`, `./support`. +- `@moq/publish` (`publish/`): capture + encode + publish, with optional UI. Same subpath shape as watch. + +**Apps / examples** +- `@moq/boy` (`moq-boy/`): MoQ Boy web viewer. The only package using `.tsx`/Solid. +- `@moq/clock` (`clock/`): private native example (publish/subscribe a clock). +- `@moq/token` (`token/`): JWT generation/validation (`jose`); also ships a `moq-token` bin. Mirror of `rs/moq-token`. + +Top-level entrypoints re-export their deps under namespaces (`export * as Net from "@moq/net"`, `Signals`, `Hang`) so consumers get one import. `Lite`/`Moq` aliases are `@deprecated`, use `Net`. + +## Signals + Effect (the reactivity model) + +This is the spine of the JS code; read `signals/src/index.ts` before touching reactive code. + +- `Signal`: mutable observable. `set`/`update`/`mutate` write, `peek` reads without subscribing. Writes are coalesced per microtask; subscribers fire only when the value actually changed. Equality is deep for plain objects/arrays but identity (`===`) for class instances (two `Broadcast` instances are never equal). Force a notify with `set(v, true)`; suppress with `set(v, false)`. `Signal.from(x)` wraps non-signals; cross-package-version identity uses a `Symbol.for` brand, not `instanceof`. +- `Computed`: read-only derived signal. Its `fn` reads deps with `effect.get(...)` just like an effect. Value is `undefined` until the first run completes and after `close()`; always handle the `undefined` case. A standalone `Computed` must be `close()`d; one made via `effect.computed()` is closed with its parent. +- `Effect`: runs `fn(effect)`, reruns whenever a tracked signal changes. Track deps inside `fn` with `effect.get(signal)` (returns current value and subscribes). `effect.getAll([...])` reads several and returns `undefined` if any is falsy. + +Lifecycle and cleanup (the rules that actually bite): + +- Register teardown with `effect.cleanup(fn)`. Everything registered during a run is torn down before the next run and on `close()`. `close()` is permanent; reruns are not. +- Use the Effect-scoped helpers instead of raw timers/listeners so cleanup is automatic: `effect.interval`, `effect.timer`, `effect.timeout`, `effect.animate`, `effect.event(target, type, listener)` (merges an `AbortSignal`), `effect.subscribe(sig, fn)` (runs now + on change), `effect.set(sig, value, cleanup)`, `effect.proxy(dst, src)`. Do NOT reach for raw `setInterval`/`setTimeout`/`requestAnimationFrame`/`addEventListener` inside an effect. +- Nesting: `effect.run(fn)` / `effect.computed(fn)` create child scopes closed with the parent. Prefer nested effects over one giant effect so unrelated deps do not re-trigger each other. +- Async: `effect.spawn(() => Promise)` runs a task and blocks the next rerun until it settles (warns after 5s). `effect.cancel` (promise) and `effect.abort` (`AbortSignal`) fire when the current run is torn down; `effect.closed` resolves on `close()`. +- DEV warnings catch leaks: a signal passing ~100 subscribers throws ("may be leaking"); an effect that subscribed to nothing warns ("will never rerun"); a `FinalizationRegistry` warns if an Effect is GC'd without `close()`. If you see these, you forgot a `close()` or tracked the wrong thing. + +## Producer / consumer and pub/sub shapes + +Networking objects split state from behavior: a plain `XxxState` class holds `Signal` fields, and the public `Xxx` class wraps it (see `net/src/broadcast.ts`, `track.ts`, `group.ts`). The publisher side answers `requested()` (await the next subscribed track) and writes; the consumer side `subscribe(name, priority)`s and reads. `closed` is exposed both as a `Signal` on state and as a `Promise` on the object. `@moq/json` and `@moq/hang/catalog` follow the same `Producer`/`Consumer` pair, with hang's catalog `Producer`/`Consumer` extending json's generics. + +## Web Components UI (watch/ui, publish/ui) + +Plain custom elements built directly on `@moq/signals`, no framework (except moq-boy, which uses Solid). The pattern, from `watch/src/element.ts` and `watch/src/ui/element.ts`: + +- `class Foo extends HTMLElement` with `static observedAttributes`. Attributes are the public API; mirror each into a `Signal` in `attributeChangedCallback`. +- Create the `Effect` in `connectedCallback`, call `effect.close()` in `disconnectedCallback`. A module-level `FinalizationRegistry` closes the Effect if the element is GC'd without disconnect (there is no real destructor for custom elements). +- Build DOM with `@moq/signals/dom` (`create`, reactive helpers) and drive visibility/content from `effect.get(...)` inside `effect.run(...)`. UI components are functions `(parent: Effect, host) => HTMLElement` that register their own reactivity on `parent` (see `watch/src/ui/components/*`). +- Styles are imported as `?inline` CSS strings into a `ShadowRoot`. The `./element` / `./ui` / `./support` subpaths are side-effectful (they call `customElements.define`); the package marks them in `sideEffects` and they are NOT re-exported from the main entry (import from the subpath). These web-component packages set `"jsr": false` because JSR forbids the `HTMLElementTagNameMap` augmentation custom elements need. + +## Conventions + +- ESM only (`"type": "module"`). Relative imports include the `.ts`/`.tsx` extension in the lower-level packages (`net`, `signals`, `hang`); `rewriteRelativeImportExtensions` in `tsconfig.json` rewrites them to `.js` on build. Some higher-level packages (watch/publish) still omit extensions, so match the file you are editing. +- Document every exported symbol and add a top-of-file `@module` doc block to each entrypoint (root convention; the published JSR/`.d.ts` docs render these). Use `@public` on the load-bearing classes. +- Build is per-package: `tsc -b` (or `vite build` for the bundled UI/web-component packages) then `bun ../common/package.ts`, which rewrites `package.json` exports from `./src/*.ts` to built `./*.js`/`.d.ts` and runs `publint`. Release via `bun ../common/release.ts`. + +## Tooling and testing + +- Use `bun` for everything (install, scripts, test runner). Never npm/yarn/pnpm. +- Biome handles formatting and linting; config is the repo-root `biome.jsonc` (tabs, width 4, line length 120). `just fix` runs `bun biome check --write`. +- Tests are `*.test.ts` run by `bun test`. Add tests where easy (signals, varint, path, ring buffers, sync all have them). +- `just js check` type-checks + biome-checks every package; `just js test` runs all unit tests; `just js build` builds all. From repo root these are `just check` / `just fix` / `just build`. diff --git a/py/CLAUDE.md b/py/CLAUDE.md new file mode 100644 index 000000000..d88853b8a --- /dev/null +++ b/py/CLAUDE.md @@ -0,0 +1,33 @@ +# py/CLAUDE.md + +Scopes the `/py` uv workspace. Universal rules (writing style / no em-dashes, Branch Targeting, Cross-Package Sync, AI Attribution) live in the root `CLAUDE.md`. + +## Two packages, one wheel boundary + +- `moq-ffi/` (`moq_ffi`): the generated uniffi bindings layer over `rs/moq-ffi`. A Maturin project; one wheel covers every crate exposed via moq-ffi (uniffi-linked libs cannot be split across wheels). Keep this layer thin. `moq_ffi/__init__.py` mostly re-exports generated symbols (`Container`, `MoqError`, `MoqSession`, `MoqClient`, ...). Do not hand-write ergonomics here; that belongs in `moq-rs`. +- `moq-rs/` (`moq`): the pure-python ergonomic wrapper consumers actually import (`import moq`). Depends on `moq-ffi` via a `~=0.2.x` compatible-release pin. This is where the friendly API lives. + +## Releases + +Two independently-versioned PyPI distributions: + +- `moq-ffi` (import `moq_ffi`): version tracks `rs/moq-ffi`; `release-py-ffi.yml` fires on `moq-ffi-v*` tags. +- `moq-rs` (import `moq`, since `moq` was taken on PyPI): versioned by hand. Bump `py/moq-rs/pyproject.toml`; on merge to `main`, `release-py.yml` publishes only if that version isn't already on PyPI (the registry is the gate). The `~=0.2.x` pin lets it float to the latest `moq-ffi` patch. + +## moq-rs layout + +`moq/__init__.py` is the single public surface; it re-exports everything and defines `__all__`. Keep new public symbols flowing through it. Modules map to roles: + +- `client.py` (`Client`): high-level connect with automatic origin wiring (simple mode) or a caller-provided origin (advanced mode). +- `server.py` (`Server`, `Request`, `Transport`): accept side. +- `origin.py`: `OriginProducer`/`OriginConsumer` and announce types, the pub/sub routing layer. +- `publish.py` / `subscribe.py`: the producer/consumer pairs (`Broadcast`, `Track`, `Group`, `Media`, `Audio`). +- `types.py`: plain data types (`Catalog`, `Frame`, `Video`, `Audio`, codecs, dimensions). + +The wrapper mirrors the `rs/moq-ffi` surface, so changes there (see the root Cross-Package Sync table) usually need a matching edit here. The producer/consumer and origin shapes parallel `rs/moq-net`; keep names aligned with the Rust side. + +## Conventions + +- Document public symbols (the package ships `py.typed`; types and docstrings are the API). No em dashes in docstrings or comments. +- Async API: `Client`/`Server` are async context managers; iterate announcements/tracks with `async for`. Match the existing pattern in `client.py` examples. +- Tooling: `uv` workspace. Run via `just py ` (see `py/justfile`). Tests live under each package's `tests/`. diff --git a/rs/CLAUDE.md b/rs/CLAUDE.md new file mode 100644 index 000000000..17eecb66d --- /dev/null +++ b/rs/CLAUDE.md @@ -0,0 +1,96 @@ +# rs/CLAUDE.md + +Reference for the `/rs` Cargo workspace. Universal rules (writing style, no em dashes, Branch Targeting, Cross-Package Sync, Public API Scrutiny, Refactor As You Go, AI Attribution) live in the root `/CLAUDE.md` and are not repeated here. + +Workspace members live in the root `Cargo.toml` (`[workspace]`). `rust-version = "1.85"`, edition 2024. Shared versions/paths are pinned under `[workspace.dependencies]`; new crates should add their dep there and reference it via `{ workspace = true }`. + +## Crate Map + +Layered roughly transport -> container/format -> media -> apps/bindings. + +**Transport / protocol** +- `moq-net` (lib): the core wire layer. Negotiates `moq-lite` or IETF `moq-transport`. Owns the Broadcast/Track/Group/Frame model and the Producer/Consumer split (see below). Generic over `web_transport_trait::Session` (no concrete QUIC dep). Submodules are private; the public surface is re-exported flat from the crate root. +- `moq-native` (lib): native connection helpers. `ClientConfig`/`ServerConfig` wrap QUIC backends (Quinn/Quiche/Noq/Iroh), WebTransport, WebSocket, TCP (qmux), Unix sockets, TLS, cert hot-reload, logging, jemalloc. Re-exports `moq_net`. Example: `examples/clock.rs`. +- `kio` (lib): "easy async". `Producer`/`Consumer` shared-state channels with `Waiter`-based notification, built on `std::task::Waker`, no runtime dependency. Underpins all the `poll_*` plumbing in moq-net and moq-mux. `src/producer.rs`, `src/consumer.rs`, `src/waiter.rs`. + +**Container / catalog formats** (standalone specs, mostly no moq-* deps, reused by moq-mux) +- `hang` (lib): media layer on `moq-net`. `catalog/` is the JSON manifest (`Catalog`, root.rs); `container/` is the frame format (timestamp + codec payload, `container::Frame`). +- `moq-loc` (lib): LOC (Low Overhead Container) wire frame codec. Top-level `encode`/`decode` + `Frame`. QUIC varints, property KVPs. +- `moq-msf` (lib): IETF MSF/CMSF catalog types (`Catalog`, `Track`, `Packaging`, `Role`). serde JSON. Alternative to hang's catalog. +- `moq-json` (lib): generic snapshot/delta value publishing over a track using RFC 7396 JSON Merge Patch. `Producer`/`Consumer`, `Guard` (RAII edit). Late joiners reconstruct from snapshot + deltas. + +**Media bridge / codecs** +- `moq-mux` (lib): the conversion layer. File/stream formats (`container/`: fmp4, flv, hls, mkv, ts, loc) and codec parsers (`codec/`: h264, h265, av1, vp8/9, opus, aac, ...) <-> hang broadcasts. `Container` trait + generic `Producer`/`Consumer`. Dual catalog (`catalog::hang`, `catalog::msf`). +- `moq-audio` (lib): native PCM <-> Opus (`unsafe-libopus`). `AudioProducer`/`AudioConsumer`, `Encoder`/`Decoder`, `AudioFormat`. Optional `capture` feature (cpal microphone), `resample`. +- `moq-video` (lib): native webcam capture + H.264 via `ffmpeg-next`. `capture::Config`, `encode::{Encoder, Producer, publish_capture}`. ffmpeg types kept out of the public signature (see `error/`). + +**Apps / binaries** +- `moq-relay` (lib+bin): clusterable, media-agnostic relay. axum HTTP API, JWT auth, WebSocket fallback, clustering. Config/TOML merge pattern lives here (see below). +- `moq-cli` (lib+bin, `moq`): serve/accept/publish/subscribe; stdin/stdout media piping. +- `moq-bench` (bin): relay load generator. `JoinSet`-spawned staggered connections, rand sampling. +- `moq-boy` (bin): crowd-controlled Game Boy emulator publisher (blocking emulator thread + async monitor tasks). +- `moq-token` (lib) / `moq-token-cli` (bin): JWT auth. `Claims`, `Algorithm`, `KeyType` (EC/RSA/OCT/OKP), JWKS. CLI does generate/sign/verify. + +**Bindings** +- `moq-ffi` (cdylib+staticlib): UniFFI bindings (Python/Swift/Kotlin/Go). Proc-macro based (`uniffi::setup_scaffolding!("moq")`, `#[uniffi::Object]`/`#[uniffi::export]`), no `.udl`. Exposes `Moq*Producer`/`Moq*Consumer`, `MoqError` (`#[uniffi(flat_error)]`). +- `libmoq` (staticlib): C bindings. `cbindgen` `build.rs` emits `moq.h` + pkg-config. `extern "C"` over opaque handles; dedicated tokio runtime thread (`LazyLock`). +- `moq-gst` (cdylib): GStreamer plugin. `gst::plugin_define!`, `moqsrc`/`moqsink` elements bridging to a background tokio task. + +When you change `moq-ffi`'s surface, mirror it in `libmoq` and the language wrappers (see the Cross-Package Sync table in root). + +## Producer / Consumer Model (moq-net) + +The whole stack is built on a split-handle pattern: a `Producer` writes, one or more `Consumer`s read, state is shared via `kio`. This recurs in moq-net, moq-mux, moq-json. + +- Broadcast: `BroadcastProducer` / `BroadcastConsumer` / `BroadcastDynamic` (`model/broadcast.rs:74,370,216`). +- Track: `TrackProducer` / `TrackConsumer` / `TrackWeak` (`model/track.rs:206,459,425`). +- Group: `GroupProducer` / `GroupConsumer` (`model/group.rs:140,286`). Consumers `clone()` for fanout. +- Frame: `FrameProducer` (impls `BufMut`) / `FrameConsumer` (`model/frame.rs:162,317`). +- Origin: `OriginProducer` / `OriginConsumer` (`model/origin.rs`). + +## Async / poll plumbing + +Two ways to drive things, both backed by `kio`: +- `async fn` (requires an active tokio runtime; awaiting outside one may panic, see `moq-net/src/lib.rs:42`). +- `poll_*` counterparts that take a `&kio::Waiter` and return `Poll<...>`, drivable from any executor or synchronously (`kio` is built on `std::task::Waker`). The `async` method usually just wraps the `poll_*` one via `kio::wait`. Example pair: `TrackConsumer::poll_recv_group` / `recv_group` (`moq-net/src/model/track.rs:502,518`). + +Follow the root `poll_*` conventions: collapse `Poll::Pending => Poll::Pending` with `ready!(...)`, and prefer `Ok(x?)` over `.map_err(Into::into)` so a fallible poll reads `let v = ready!(inner.poll_next(cx))?;`. Representative `ready!` sites: `moq-mux/src/container/consumer.rs:201`, `moq-net/src/model/group.rs`. + +## Version matching + +`moq_net::Version` is `#[non_exhaustive]`, splitting `Lite(lite::Version)` and `Ietf(ietf::Version)` (`version.rs:47`). When matching on a `Version` (or the inner draft enums), default to the **newest** draft so future versions fall forward; list older versions explicitly: + +```rust +match version { + Version::Draft14 | Version::Draft15 | Version::Draft16 => { /* old behavior */ } + _ => { /* newest / draft-17+ behavior */ } +} +``` + +Negotiation: `version::NEGOTIATED` lists SETUP-negotiated versions in preference order; newer drafts negotiate via dedicated ALPNs (`version::ALPNS`). The version-to-behavior dispatch lives in `setup.rs:73` (`SetupVersion::from_version`). + +## Rust conventions + +- **Prefer `kio` over tokio sync primitives**: reach for `kio::Producer`/`Consumer` (and the `poll_*` plumbing) instead of `tokio::sync` channels or `watch`. A `tokio::sync::watch` (or a channel) carrying a single value is a code smell. `kio` ties into the runtime-free `poll_*` model and avoids a hard runtime dependency. +- **Errors**: `thiserror` with `#[from]` for libraries, `anyhow` (with `.context("...")`, not `.map_err(|_| anyhow!())`) for binaries. Always `#[non_exhaustive]` on public error enums (e.g. `moq-net/src/error.rs:6`, `moq-ffi/src/error.rs:4`, `moq-loc/src/lib.rs:55`). Use `#[error(transparent)]` + `#[from]` for wrapped foreign errors (see `moq-token/src/error.rs`). +- **Config + TOML merge**: any `#[arg]` field on a TOML-loadable config must be `Option`, never a bare `bool`/`String`/etc. The TOML->CLI merge re-applies clap defaults and silently clobbers TOML values for bare fields. See `moq-relay/src/config.rs` and its regression tests (`cli_does_not_clobber_toml_*`, around line 126); add such a test for any new flag. +- **Config structs**: `#[derive(Parser, Serialize, Deserialize)]` with `#[serde(deny_unknown_fields, default)]`, clap `#[arg(long, env = "MOQ_...")]`, nested configs via `#[command(flatten)]`, and an `.init()`/`.load()` method that produces the live object. Add `#[non_exhaustive]` + `Default`/constructor to configs consumers build (per root Public API Scrutiny). +- **Unwrapping**: prefer `if let Some(v) = x { ... }` / `let Some(v) = x else { ... };` over a `match` whose only job is to bind the inner value. Keep `match` when both arms do real work. +- **Naming**: role-based module + short unprefixed type (`encode::Encoder`, `capture::Config`), not `EncoderConfig`/`CameraConfig`. Re-export flat to avoid stutter (`mod encoder` private, `pub use encoder::Encoder`). + +## Binary setup + +Binaries are `#[tokio::main] async fn main() -> anyhow::Result<()>`. Install the rustls crypto provider before anything TLS: + +```rust +rustls::crypto::aws_lc_rs::default_provider().install_default().expect("crypto provider"); +``` + +Then `Config::load()?` (initializes tracing), build clients/servers via `.init()`, and run an event loop with `tokio::select!`. See `moq-relay/src/main.rs`, `moq-bench/src/main.rs`. + +## Testing + +- `just check` runs all tests + lint; `just fix` auto-fixes formatting/lint. `cargo test -p ` for one crate. +- Rust tests are `#[cfg(test)] mod tests` inline in the source file. +- Async tests that depend on time call `tokio::time::pause()` first so timers fire instantly and deterministically (e.g. `moq-net/src/model/origin.rs:1099`). +- Config-merge regressions belong next to the config (`moq-relay/src/config.rs::tests`); they serialize env mutation with a lock since clap reads env. diff --git a/rs/moq-bench/src/config.rs b/rs/moq-bench/src/config.rs index 0f96742fc..17a862f10 100644 --- a/rs/moq-bench/src/config.rs +++ b/rs/moq-bench/src/config.rs @@ -102,7 +102,7 @@ impl Config { /// /// Every overridable field is `Option`, so an absent CLI flag leaves the /// TOML value untouched during the final `update_from` re-parse. See - /// `CLAUDE.md` for why bare fields would silently clobber the TOML. + /// `rs/CLAUDE.md` for why bare fields would silently clobber the TOML. pub(crate) fn parse_and_merge(args: I) -> anyhow::Result where I: IntoIterator, diff --git a/rs/moq-gst/Cargo.toml b/rs/moq-gst/Cargo.toml index 38a6e3e7c..919e4d888 100644 --- a/rs/moq-gst/Cargo.toml +++ b/rs/moq-gst/Cargo.toml @@ -6,7 +6,7 @@ repository = "https://github.com/kixelated/moq" license = "MIT OR Apache-2.0" version = "0.2.7" -edition = "2021" +edition = "2024" rust-version.workspace = true publish = true diff --git a/rs/moq-gst/src/source/imp.rs b/rs/moq-gst/src/source/imp.rs index fa62cc57e..de8b7edc7 100644 --- a/rs/moq-gst/src/source/imp.rs +++ b/rs/moq-gst/src/source/imp.rs @@ -3,7 +3,7 @@ use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::{LazyLock, Mutex}; use std::time::Duration; -use anyhow::{bail, Context, Result}; +use anyhow::{Context, Result, bail}; use gst::glib; use gst::prelude::*; use gst::subclass::prelude::*; diff --git a/rs/moq-relay/src/config.rs b/rs/moq-relay/src/config.rs index 087f5c8ba..3fd2c4e53 100644 --- a/rs/moq-relay/src/config.rs +++ b/rs/moq-relay/src/config.rs @@ -79,7 +79,7 @@ impl Config { /// (if `file` is set) → CLI args re-applied so explicit flags / env vars /// override TOML. /// - /// # Pitfall (see `CLAUDE.md` and `tests` below) + /// # Pitfall (see `rs/CLAUDE.md` and `tests` below) /// /// The final `update_from` re-runs the clap parser over `args`. For /// fields typed as bare `bool`, an absent CLI flag writes diff --git a/rs/moq-relay/src/stats.rs b/rs/moq-relay/src/stats.rs index 30d957bdb..6e8783ca7 100644 --- a/rs/moq-relay/src/stats.rs +++ b/rs/moq-relay/src/stats.rs @@ -32,7 +32,7 @@ pub struct StatsConfig { /// re-parse. With a bare `bool`, an absent `--stats-enabled` CLI flag /// writes the `Default::default()` value (`false`) over the TOML value. /// See `tests::cli_does_not_clobber_toml_stats_enabled` and the - /// "Config flags + TOML merge" note in `CLAUDE.md`. + /// "Config flags + TOML merge" note in `rs/CLAUDE.md`. #[arg( long = "stats-enabled", env = "MOQ_STATS_ENABLED",