Skip to content

Commit 49c73b0

Browse files
lwwmanningclaude
andcommitted
[RFC 60] Address fourth /rfc-review round: Stage 2 norm_factor + D.15 polish
Two MINOR fixes catching internal inconsistencies introduced by iteration 3's additions. MINORs: - D.6 Stage 2 encode pseudocode was missing the `norm_factor` parameter added to D.4/D.5 in iteration 3. Stage 2 normalizes per block dimension B (`B^(-num_rounds/2)`), not per padded_dim. Fixed and added a comment clarifying that Stage 2 inherits Stage 1 / Stage 1.5's scale-`S` decision (S = 1 today; EDEN-S keyed on B once Stage 1.5 lands). - D.15 `extract_signs` was named in the SORF pseudocode but its body was an unlabeled for-loop above. Wrapped the body in an explicit function signature so the reader doesn't have to infer the binding. Signed-off-by: Will Manning <will@willmanning.io> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: Will Manning <will@willmanning.io>
1 parent 50b6308 commit 49c73b0

1 file changed

Lines changed: 14 additions & 6 deletions

File tree

rfcs/0060-block-turboquant.md

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -2084,14 +2084,19 @@ fn tq_decode(tq: TurboQuantArray) -> Vector<F, d>:
20842084
```text
20852085
fn tq_encode_stage2(v: Vector<F, d>, cfg: &TurboQuantConfig, B: u32) -> TurboQuantArray:
20862086
k = d / B # exact division required; if not, fall back to Stage 1 padded
2087+
norm_factor = B^(-cfg.num_rounds / 2) # Stage 2 normalizes per block dimension
20872088
20882089
for i in 0..k:
20892090
v_i = v[i*B .. (i+1)*B]
20902091
n_i = ‖v_i‖₂
20912092
if n_i > 0:
20922093
u_i = v_i / n_i
2093-
r_i = SORF(u_i, block_seed(seed, i), cfg.num_rounds)
2094+
# Same SORF as D.4 but applied per-block; block_seed mixes the array's seed with
2095+
# the block index so each block has an independent rotation (see Open Questions §3).
2096+
r_i = SORF(u_i, block_seed(cfg.seed, i), cfg.num_rounds, norm_factor)
20942097
for j in 0..B:
2098+
# S = S(B, cfg.bit_width); Stage 2 inherits the Stage 1 / Stage 1.5 scale
2099+
# decision (S = 1 today; EDEN-S keyed on block dim B once Stage 1.5 lands).
20952100
codes[i*B + j] = nearest_centroid(r_i[j] * S, centroids)
20962101
else:
20972102
codes[i*B .. (i+1)*B] = 0
@@ -2346,11 +2351,14 @@ rounds, the total sign budget is `num_rounds * padded_dim` signs;
23462351
extract them in **round-major, block-major** order:
23472352

23482353
```text
2349-
for round in 0..num_rounds:
2350-
for block in 0..(padded_dim / 64): // padded_dim divisible by 64 (guaranteed by >= 128 and power-of-2)
2351-
u = next_u64(state)
2352-
for bit in 0..64:
2353-
sign[round][block * 64 + bit] = if (u >> bit) & 1 == 1 { +1.0 } else { -1.0 }
2354+
fn extract_signs(state: &mut u64, num_rounds: u8, padded_dim: usize) -> [[f32; padded_dim]; num_rounds]:
2355+
let mut signs = zero-init array [num_rounds][padded_dim]
2356+
for round in 0..num_rounds:
2357+
for block in 0..(padded_dim / 64): // padded_dim divisible by 64 (guaranteed by >= 128 and power-of-2)
2358+
u = next_u64(state)
2359+
for bit in 0..64:
2360+
signs[round][block * 64 + bit] = if (u >> bit) & 1 == 1 { +1.0 } else { -1.0 }
2361+
return signs
23542362
```
23552363

23562364
If `padded_dim < 64` the contract degrades (`padded_dim < MIN_DIMENSION

0 commit comments

Comments
 (0)