Skip to content

Commit 128af68

Browse files
committed
moq-lite: compress only frame payloads, not the framing
Switch group compression from "compress the whole frame sequence" to "compress only the payloads." The payloads of a group still form one DEFLATE stream (reset per group, sliced per frame into each frame's opaque Payload), so cross-frame redundancy is retained, but the FRAME framing stays in the clear. This is for version agility and caching: with the framing uncompressed, a relay or cache can keep payloads compressed in memory, read frame metadata without inflating, and re-frame a group across transport versions (new GROUP/FRAME headers) without decompress/recompress. None of that is possible if the framing is inside the DEFLATE blob. Message Length is again the on-wire (compressed) Payload size. Also revert the moq-compression extension to its pre-PR state: its earlier enum/per-object-override design is superseded by this model and will be redone to mirror the final moq-lite design. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W8bLV6vHzucLNvDhPk3bMP
1 parent 607ed83 commit 128af68

2 files changed

Lines changed: 41 additions & 66 deletions

File tree

draft-lcurley-moq-compression.md

Lines changed: 29 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,9 @@ informative:
2525
--- abstract
2626

2727
This document defines a payload compression extension for MoQ Transport {{moqt}}.
28-
A track-level Compression property is a boolean hint by which the original publisher signals that a track's object payloads are worth compressing.
29-
The algorithm is negotiated independently on each hop, and compression is applied per hop: an object is compressed only on a hop that has negotiated the extension and a shared algorithm, and is sent verbatim otherwise.
30-
Each object is compressed independently so objects remain individually decodable, an object can opt out when compression would not help, and the decompressed bytes — the actual object — are unchanged end to end.
28+
A track-level Compression property lets the original publisher signal that a track's object payloads are worth compressing, and with which algorithm.
29+
Compression is then applied independently on each hop: a payload is compressed only on a hop that has negotiated the extension and whose receiver supports the algorithm, and is sent verbatim otherwise.
30+
Each object is compressed independently so objects remain individually decodable, and the decompressed bytes — the actual object — are unchanged end to end.
3131

3232
--- middle
3333

@@ -43,10 +43,10 @@ But MoQ also carries non-media tracks — JSON, text, telemetry, captions, uncom
4343
For these tracks there is no standard, transport-visible way to compress payloads, so each application reinvents it, and relays cannot help.
4444

4545
Like HTTP Transfer-Encoding, the on-wire compression is a hop-by-hop optimization: it does not conceptually change the object payload — the decompressed bytes *are* the object — it only changes how those bytes are carried over a single hop.
46-
What this extension adds on top is an end-to-end *signal*: a boolean track property by which the original publisher marks the content as worth compressing. The signal travels end to end; the choice of algorithm and the compression itself happen per hop.
46+
What this extension adds on top is an end-to-end *signal*: a track property by which the original publisher marks the content as worth compressing and names the algorithm. The signal travels end-to-end; the compression happens per hop.
4747

48-
- **Publisher signals, hops apply**: the COMPRESSION track property is set by the original publisher and carried end to end, but a payload is only compressed on a hop that negotiated the extension and a shared algorithm. Where the extension is not negotiated, the same payload travels verbatim.
49-
- **Per object, independently**: each object payload is an independent compressed stream with no shared dictionary or state between objects. This keeps every object individually decodable, avoids head-of-line decoding within a group, and lets an individual object opt out of compression when it would not benefit.
48+
- **Publisher signals, hops apply**: the COMPRESSION track property is set by the original publisher and carried end-to-end, but a payload is only compressed on a hop that negotiated the extension and whose receiver supports the algorithm. Where the extension is not negotiated, the same payload travels verbatim.
49+
- **Per object, independently**: each object payload is an independent compressed stream with no shared dictionary or state between objects. This keeps every object individually decodable and avoids head-of-line decoding within a group.
5050

5151

5252
# Setup Negotiation
@@ -65,69 +65,44 @@ COMPRESSION Setup Option {
6565

6666
**Algorithm**:
6767
One or more Algorithm identifiers (see [Compression Algorithms](#compression-algorithms)) that the sender can decompress, each a varint, filling the Option Value.
68-
They are listed in the sender's order of preference, most-preferred first.
6968
The identifier `none` (0) MUST NOT be listed (it requires no negotiation).
7069

7170
A sender MUST NOT compress with an algorithm the receiver did not advertise in its SETUP.
72-
The negotiated algorithm for a hop — the **hop default** — is the first algorithm in the receiver's advertised list that the sender can also produce; if the lists do not intersect, the hop has no default and every payload travels verbatim.
73-
This keeps the common case free of per-object signaling: where the COMPRESSION track property is present and the hop has a default, a receiver decompresses each object with the hop default unless that object carries a [per-object override](#per-object-override) naming a different algorithm (in particular `none`, to send it verbatim). Where the property is absent, the extension is not negotiated, or the algorithm lists do not intersect, the sender was not permitted to compress, so every payload is verbatim.
71+
This makes the on-wire state unambiguous on every hop without any per-object signaling: a receiver decompresses a track's payloads **if and only if** the COMPRESSION track property is present and the receiver advertised that algorithm in its own SETUP. In every other case — the property absent, the extension not negotiated, or the algorithm not advertised by the receiver — the sender was not permitted to compress, so the receiver treats the payloads as verbatim.
7472

7573

7674
# COMPRESSION Track Property
77-
The COMPRESSION property is the original publisher's signal that a track's object payloads are worth compressing.
75+
The COMPRESSION property is the original publisher's signal that a track's object payloads are worth compressing, and which algorithm to use.
7876
It is a track-level Key-Value-Pair carried with the track's properties (see {{moqt}} Section 2.5 and Section 12), set by the original publisher and forwarded unchanged by relays.
7977
Because the value is a single integer, COMPRESSION uses an even Type so the value is a bare varint:
8078

8179
~~~
8280
COMPRESSION Track Property {
8381
Type (vi64) = 0xC03D0
84-
Value (vi64) ; boolean hint
82+
Value (vi64) ; Algorithm identifier
8583
}
8684
~~~
8785

8886
**Value**:
89-
A boolean hint: `1` means the track's payloads are worth compressing; `0`, or absence of the property, means they are not and are always transmitted verbatim.
90-
Values greater than `1` are reserved for future use and MUST be treated as `1` by a receiver that does not understand them, so the hint stays additive.
91-
The property names no algorithm: which algorithm is used, if any, is the per-hop negotiated [hop default](#setup-negotiation), overridable [per object](#per-object-override).
87+
The Algorithm identifier the publisher recommends for this track's payloads.
88+
The absence of the property, or a value of `none` (0), means the track is not marked for compression and its payloads are always transmitted verbatim.
9289

9390
The property is fixed for the lifetime of the track and MUST NOT change.
9491
A relay MUST forward it unchanged on every hop, including a hop that has not negotiated the extension: there it is simply an ignored unknown Key-Value-Pair, but forwarding it lets a further-downstream hop that does negotiate the extension still act on the publisher's signal.
9592

96-
Compression is enabled only by the combination of a non-zero hint and the extension being negotiated with a shared algorithm on a hop.
97-
A publisher MUST NOT compress object payloads on a track that does not carry a non-zero COMPRESSION hint.
93+
Compression is enabled only by the combination of this track property and the extension being negotiated on a hop.
94+
A publisher MUST NOT compress object payloads on a track that does not carry the COMPRESSION property, and there is no way to enable compression on a per-object basis: the property governs the whole track, and on a compressing hop every non-empty payload is compressed.
9895

99-
Whether a given object is actually compressed is decided per hop and per object:
96+
Whether payloads are actually compressed is decided per hop:
10097

101-
- On a hop where the extension is negotiated and a [hop default](#setup-negotiation) exists, each non-empty object payload is compressed with the hop default and the receiver decompresses it — unless the object carries a [per-object override](#per-object-override), which names the algorithm actually used (including `none`, to send that object verbatim).
102-
- On any other hop — the extension not negotiated, or the algorithm lists do not intersect — payloads are sent verbatim. The receiver either never sees the property (an ignored unknown Key-Value-Pair) or sees it but knows the sender was not permitted to compress for it, so it treats the payloads as verbatim either way.
98+
- On a hop where the extension is negotiated and the receiver advertised the property's algorithm, every non-empty object payload MUST be compressed with that algorithm, and the receiver decompresses it.
99+
- On any other hop — the extension not negotiated, or the receiver did not advertise that algorithm — payloads are sent verbatim. The receiver either never sees the property (an ignored unknown Key-Value-Pair) or sees it but knows the sender was not permitted to compress for it, so it treats the payloads as verbatim either way.
103100

104-
A sender SHOULD send an object verbatim (via a `none` override) whenever the hop default would not make that object smaller — for example a small JSON merge-patch delta that DEFLATE would enlarge.
105101
Compression applies to the object payload only; object properties and message framing are never compressed.
106-
An empty payload (size 0) MUST NOT be compressed and remains empty on the wire; it needs no override.
102+
An empty payload (size 0) MUST NOT be compressed and remains empty on the wire.
107103

108104
A publisher SHOULD set COMPRESSION only for payload types that benefit from it.
109-
Already-compressed media SHOULD omit it (or use `0`).
110-
111-
112-
# Per-Object Override {#per-object-override}
113-
The COMPRESSION_ALGORITHM property is an optional object-level Key-Value-Pair that overrides, for a single object, the algorithm a hop would otherwise apply.
114-
Because the value is a single integer, it uses an even Type so the value is a bare varint:
115-
116-
~~~
117-
COMPRESSION_ALGORITHM Object Property {
118-
Type (vi64) = 0xC03D2
119-
Value (vi64) ; Algorithm identifier
120-
}
121-
~~~
122-
123-
**Value**:
124-
The [algorithm](#compression-algorithms) actually used for this object's payload on this hop.
125-
`none` (0) means the object is carried verbatim; any other identifier names the algorithm whose output the payload is.
126-
A sender MUST NOT name an algorithm the receiver did not advertise in its SETUP.
127-
128-
The property is meaningful only on a hop that has a [hop default](#setup-negotiation); where present it replaces the hop default for that object alone, and elsewhere objects are always verbatim and any COMPRESSION_ALGORITHM property MUST be ignored.
129-
Unlike the boolean COMPRESSION hint, it is not the publisher's end-to-end signal: because it records what a hop actually did, a relay rewrites or removes it to reflect what it did on each downstream hop, exactly as it does for the payload bytes.
130-
Its typical use is a `none` override that keeps an incompressible or tiny object verbatim while the rest of the track is compressed.
105+
Already-compressed media SHOULD omit it (or use `none`).
131106

132107

133108
# Compression Algorithms {#compression-algorithms}
@@ -144,33 +119,33 @@ There is no shared dictionary or state between objects, so each object decompres
144119

145120

146121
# Relay Behavior
147-
A relay forwards the boolean COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop.
122+
A relay forwards the COMPRESSION track property unchanged — it is the publisher's end-to-end signal — and applies compression independently on each hop.
148123

149-
On its upstream subscription, the relay receives each object compressed with that hop's default unless the object carried a per-object override; it reads the [COMPRESSION_ALGORITHM](#per-object-override) property, or the hop default in its absence, to decompress as needed.
150-
On each downstream subscription the relay serves, it compresses each object with that downstream's hop default when one exists and sends objects verbatim otherwise, rewriting or removing the per-object COMPRESSION_ALGORITHM property to reflect what it actually did on that hop.
124+
On its upstream subscription, the relay receives payloads compressed if and only if that hop compressed them (the extension negotiated and the relay advertised the algorithm); it decompresses them as needed.
125+
On each downstream subscription the relay serves, it compresses payloads with the track's algorithm when that downstream negotiated the extension and advertised the algorithm, and sends them verbatim otherwise.
151126

152-
Compression is thus driven by the publisher's hint and each hop's negotiation, not by the relay's own initiative: a relay does not compress a track the publisher did not mark.
127+
Compression is thus driven by the publisher's track property, not by the relay: a relay does not compress a track the publisher did not mark.
153128
In every case the decompressed bytes delivered to the application MUST be identical to what the origin published.
154129

155130
A relay or generic library MUST NOT inspect or modify the decompressed contents unless otherwise negotiated; only recompression that preserves the decompressed bytes exactly is permitted.
156131

157132

158133
# Security Considerations
159134
Compressing data that mixes attacker-controlled and secret content in the same object can leak the secret through compressed size, as in the CRIME and BREACH attacks.
160-
A publisher MUST NOT set a non-zero COMPRESSION hint on a track whose object payloads combine secret material with attacker-influenced material.
135+
A publisher MUST NOT set COMPRESSION on a track whose object payloads combine secret material with attacker-influenced material.
161136
Because compression here is per-object with no cross-object dictionary, the exposure is bounded to within a single object, but it is not eliminated.
162137

163138
A malicious sender could emit a small compressed payload that decompresses to a very large buffer (a "decompression bomb").
164139
A receiver MUST bound the size of a decompressed object payload. If the bound is exceeded it MUST reset the affected Subscribe/Fetch stream (rather than allocate unbounded memory) and MAY close the session with a PROTOCOL_VIOLATION if it considers the peer abusive; the reset is stream-scoped so a single bad object does not tear down unrelated subscriptions.
165140

166-
Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `0`).
141+
Compression is orthogonal to {{moqt}} end-to-end encryption: an encrypted payload is effectively incompressible, so a publisher using end-to-end encryption SHOULD omit COMPRESSION (or use `none`).
167142

168143

169144
# IANA Considerations
170145

171146
This document requests the following registrations.
172147
High, distinctive values are requested to avoid the low ranges reserved by {{moqt}} and to minimize collisions with provisional registrations by other extensions; they also avoid the greasing pattern (`0x7f * N + 0x9D`).
173-
Each Type is even so that its value is a bare varint with no length prefix (see {{moqt}} Section 2.5).
148+
The parameter Type is even so that its value is a bare varint with no length prefix (see {{moqt}} Section 2.5).
174149

175150
## MOQT Setup Options
176151

@@ -182,12 +157,11 @@ This document requests a registration in the "MOQT Setup Options" registry ({{mo
182157

183158
## MOQT Properties
184159

185-
This document requests registrations in the "MOQT Properties" registry ({{moqt}} Section 15.8), used for object and track properties.
160+
This document requests a registration in the "MOQT Properties" registry ({{moqt}} Section 15.8), used for object and track properties.
186161

187-
| Value | Name | Scope | Reference |
188-
|:--------|:----------------------|:-------|:--------------|
189-
| 0xC03D0 | COMPRESSION | Track | This Document |
190-
| 0xC03D2 | COMPRESSION_ALGORITHM | Object | This Document |
162+
| Value | Name | Scope | Reference |
163+
|:--------|:------------|:------|:--------------|
164+
| 0xC03D0 | COMPRESSION | Track | This Document |
191165

192166
## MOQT Compression Algorithms
193167

0 commit comments

Comments
 (0)