Skip to content

Commit 1e9a6e3

Browse files
lwwmanningclaude
andcommitted
[RFC 60] Address fifth /rfc-review round: Stage 1 → 1.5 compat clarification
One MINOR fix. The §13 incremental-shipping table said Stage 1.5 reads Stage 1 files "Yes (wire-format-identical)" — bit-identical at the bytes level but a Stage 1.5 reader using EDEN-`S` to dequantize codes written under `S = 1` produces incorrect values (the codes index a centroid grid scaled for S=1; dividing by a different S at decode time yields a different vector than the encoder intended). Resolution: commit explicitly to "Stage 1 is preview, no backward-compat commitment; Stage 1.5 is the first stable wire format." Pre-Stage-1.5 files must be re-encoded. Documented in §6 (Stage 1 refinement) and §13 (Incremental shipping table). The alternative — adding a per-file `s_variant` metadata field — was considered but rejected for permanent metadata cost on a short-lived experimental phase. Signed-off-by: Will Manning <will@willmanning.io> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
1 parent 49c73b0 commit 1e9a6e3

1 file changed

Lines changed: 23 additions & 9 deletions

File tree

rfcs/0060-block-turboquant.md

Lines changed: 23 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -695,6 +695,18 @@ budget. `S = 1` is wire-format-stable today (it's already in the code)
695695
and matches what the note [14] calls "TurboQuant_mse" — a known but
696696
suboptimal point on the curve.
697697

698+
**Stage 1 is a preview format with no backward-compat commitment.**
699+
TurboQuant has not been included in a Vortex release, so the Stage 1
700+
wire format (`S = 1`) is provisional. When Stage 1.5 lands and pins
701+
EDEN's `S(d, b)`, the new constant becomes the stable encode/decode
702+
behavior for **all** TurboQuant files; pre-Stage-1.5 files (if any
703+
escaped the preview phase) must be re-encoded. Stage 1.5 is therefore
704+
**the first stable wire format**, not a follow-up to a frozen Stage 1.
705+
This avoids the alternative of adding a per-file `s_variant` metadata
706+
field to distinguish files encoded under `S = 1` vs EDEN-`S` — the
707+
alternative would work but carries permanent metadata cost for a
708+
short-lived experimental phase.
709+
698710
If a future stage adds EDEN's native unbiased mode (§15), an
699711
`unbiased: bool` metadata flag is added at prost tag 7 and the cache
700712
key extends to `(padded_dim, bit_width, biased)`; biased and unbiased
@@ -1680,15 +1692,17 @@ Removal is a Stage 1 cleanup task once the example migrates.
16801692

16811693
### Incremental shipping
16821694

1683-
| Stage | Ships to users? | Reads prior stage files? | Notes |
1684-
| -------------- | --------------- | --------------------------- | ----------------------------------------------------- |
1685-
| 1 (MSE) | Yes | N/A (first preview) | Single block, biased only, `S = 1` |
1686-
| 1.5 (EDEN-`S`) | Yes | Yes (wire-format-identical) | Layers EDEN-`S` on top; no metadata or storage change |
1687-
| 2 (blocks) | Yes | Yes (k=1 is identical) | `block_size` metadata added; k>1 needs S2+ readers |
1688-
| 3 (PDX) | Yes | Yes (FSL codes still work) | PDX codes need `PDXArray` registered |
1689-
1690-
Each stage is independently shippable. Users can upgrade incrementally.
1691-
Files written by earlier stages are always readable by later decoders.
1695+
| Stage | Ships to users? | Reads prior stage files? | Notes |
1696+
| -------------- | ------------------------ | ---------------------------------------------------------- | ---------------------------------------------------------------------------------- |
1697+
| 1 (MSE) | Preview only (no compat) | N/A (first preview) | Single block, biased only, `S = 1`. Wire format is provisional. |
1698+
| 1.5 (EDEN-`S`) | **First stable release** | No — Stage 1 was preview; Stage 1 files must be re-encoded | Layers EDEN-`S` on top; no metadata or storage shape change. Pins the wire format. |
1699+
| 2 (blocks) | Yes | Yes (k=1 is identical to Stage 1.5) | `block_size` metadata added; k>1 needs Stage 2+ readers. |
1700+
| 3 (PDX) | Yes | Yes (FSL codes still work) | PDX codes need `PDXArray` registered. |
1701+
1702+
Stages 1.5 → 2 → 3 are independently shippable and each later stage reads
1703+
all earlier-stage files correctly. Stage 1 is explicitly outside this
1704+
guarantee — it is a preview phase intended to validate the design before
1705+
Stage 1.5 commits to a stable wire format.
16921706

16931707
## Drawbacks
16941708

0 commit comments

Comments
 (0)