Skip to content

Commit 2af4654

Browse files
claudeclaude
committed
PR #477 review: ColumnOutOfBounds variant, unbundle_from deprecation, doc corrections
- soa_envelope: verify_layout now returns ColumnOutOfBounds (not overloaded StrideMismatch) when a column end exceeds stride; test updated. 8 tests green. - kv_bundle: deprecate unbundle_from — it is NOT the inverse of weighted-average bundle_into; call sites gated #[allow(deprecated)] + FIXME. 4 tests green. - soa-three-tier-model: layout table labeled current/transitional; entity_type marked scheduled-for-removal. - q3 probe: baton 'stone' corrected to Superseded (conflicts with arch target-state). - TECH_DEBT: TD-UNBUNDLE-FROM-1 logged. https://claude.ai/code/session_0147hSzjmWZDuy2MSQNrhEK5
1 parent 2f15278 commit 2af4654

5 files changed

Lines changed: 72 additions & 17 deletions

File tree

‎.claude/board/TECH_DEBT.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,17 @@
1515

1616
## Open Debt
1717

18+
### TD-UNBUNDLE-FROM-1 — `unbundle_from` is NOT the inverse of `bundle_into` (2026-06-07)
19+
20+
**Open.** `crates/lance-graph-planner/src/cache/kv_bundle.rs` — `unbundle_from`
21+
uses `wrapping_sub` as the "undo" of `bundle_into`. But `bundle_into` is a
22+
weighted average: `(old * w_self + new * w_new) / total`. Subtraction is not the
23+
inverse. `AttentionMatrix::set` calls both in sequence, silently corrupting the
24+
gestalt ~1 bit per epoch. Measurable after ~100 epochs. Function is marked
25+
`#[deprecated]` with a doc warning; callers use `#[allow(deprecated)]` + FIXME.
26+
**Paid by:** switch to raw-sum + count tracking so exact integer subtraction is
27+
possible. Cross-ref: `kv_bundle.rs:28-33`.
28+
1829
### TD-HELIX-OVERLAP-1 (D-HELIX-1) — `helix` re-derives existing CERTIFIED primitives (clean-room by directive)
1930

2031
**Open.** `crates/helix` ships as a zero-dep clean-room codec per the user directive "scoped only to crate." ~80% of its pipeline duplicates existing, in-places-CERTIFIED workspace code: Fisher-Z/arctanh→i8 (`bgz-tensor::projection::Base17Fz`, `bgz-tensor::fisher_z::FamilyGamma` ρ≥0.999), golden-spiral azimuth (`jc::weyl`), stride-4 coupling (`thinking-engine::reencode_safety`, `highheelbgz`), EULER_GAMMA hand-off (`jc::precond`, `bgz-tensor::euler_fold`), 256-palette/L1 (`bgz17::palette`). Genuinely new = the `√u` equal-area hemisphere placement + the PLACE/RESIDUE doctrine. **Paid by** (when it graduates from clean-room): the consolidation path in `crates/helix/KNOWLEDGE.md` § Overlap & Consolidation — re-export `FamilyGamma` behind a feature; route coupling through the canonical `(i·11)%17`/stride-4 zipper; feed `ResidueEdge` into the existing HIP/TWIG CAKES tier. **Also owed:** a fidelity-vs-ground-truth probe (the naive-u8 floor gate ≥0.9980 Pearson is currently CONJECTURE — NOT RUN) before promotion. Cross-ref: E-HELIX-OVERLAP, `encoding-ecosystem.md`.

‎crates/lance-graph-contract/src/soa_envelope.rs‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,10 @@ pub enum EnvelopeError {
118118
StrideMismatch { declared: usize, summed: usize },
119119
/// Two columns overlap, or a gap/ordering violation was found.
120120
ColumnOverlap { col_a: u16, col_b: u16 },
121+
/// A column's byte range ends past the declared row stride. Distinct from
122+
/// [`StrideMismatch`]: the widths can sum to the stride while a column is
123+
/// still positioned (via its `row_offset`) so its end exceeds the stride.
124+
ColumnOutOfBounds { col: u16, col_end: usize, stride: usize },
121125
/// `as_le_bytes().len()` is not `row_stride * n_rows` (backing store size mismatch).
122126
PacketSizeMismatch { expected: usize, found: usize },
123127
/// A requested row or column index is out of bounds.
@@ -194,9 +198,10 @@ pub trait SoaEnvelope {
194198
let (a_start, a_end) = a.row_byte_range();
195199
summed += a.col_bytes_per_row();
196200
if a_end > stride {
197-
return Err(EnvelopeError::StrideMismatch {
198-
declared: stride,
199-
summed: a_end,
201+
return Err(EnvelopeError::ColumnOutOfBounds {
202+
col: a.name_id,
203+
col_end: a_end,
204+
stride,
200205
});
201206
}
202207
for b in &cols[i + 1..] {
@@ -346,7 +351,7 @@ mod tests {
346351
let env = TestEnvelope { cols, stride: 8, rows: 1, bytes: vec![0u8; 8], cycle: 0 };
347352
assert!(matches!(
348353
env.verify_layout(),
349-
Err(EnvelopeError::StrideMismatch { .. })
354+
Err(EnvelopeError::ColumnOutOfBounds { col: 1, col_end: 12, stride: 8 })
350355
));
351356
}
352357

‎crates/lance-graph-planner/src/cache/kv_bundle.rs‎

Lines changed: 37 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,32 @@ pub fn bundle_into(source: &HeadPrint, target: &mut HeadPrint, weight_self: f32,
2626
}
2727

2828
/// Unbundle: subtract out (XOR analog for i16).
29+
///
30+
/// # Correctness warning — this is NOT the inverse of [`bundle_into`]
31+
///
32+
/// [`bundle_into`] performs a **weighted average**: `(old * w_self + new * w_new) / total`.
33+
/// Subtraction is the inverse of XOR-bundle or simple add-bundle, NOT of weighted-average
34+
/// bundle. Calling this after `bundle_into` silently corrupts the gestalt: the gestalt
35+
/// drifts every epoch because the removed value is the rounded average representation,
36+
/// not the original addend. After ~100 epochs the corruption is measurable.
37+
///
38+
/// The correct approach for an updateable gestalt is one of:
39+
/// 1. **Re-accumulate**: on update, rebuild the gestalt from scratch by iterating all heads.
40+
/// 2. **Track raw sum + count**: store `sum: [i32; BASE_DIM]` and `count: u32` separately
41+
/// so exact subtraction is possible without rounding loss.
42+
/// 3. **Accept approximate unbundle only when weight_self = weight_new = 1.0**: then
43+
/// `bundle_into` reduces to `(old + new) / 2` and there is still no exact inverse.
44+
///
45+
/// Tracked as tech-debt in `.claude/board/TECH_DEBT.md` (unbundle_from correctness).
46+
///
47+
/// # Panics
48+
/// Never panics — wrapping arithmetic.
49+
#[deprecated(
50+
since = "0.0.0",
51+
note = "NOT the inverse of bundle_into (weighted-average). \
52+
See function doc for the correct unbundle strategies. \
53+
Tracked: .claude/board/TECH_DEBT.md — unbundle_from correctness."
54+
)]
2955
pub fn unbundle_from(source: &HeadPrint, target: &mut HeadPrint) {
3056
for d in 0..BASE_DIM {
3157
target.dims[d] = target.dims[d].wrapping_sub(source.dims[d]);
@@ -72,10 +98,16 @@ impl AttentionMatrix {
7298
}
7399

74100
/// Set attention head and update gestalt.
101+
///
102+
/// Note: the gestalt update uses the deprecated `unbundle_from` which is NOT the
103+
/// exact inverse of `bundle_into` (weighted-average). The gestalt drifts slowly
104+
/// over many epochs. FIXME: rebuild gestalt from scratch or switch to raw-sum
105+
/// tracking — tracked in `.claude/board/TECH_DEBT.md`.
75106
pub fn set(&mut self, row: usize, col: usize, head: HeadPrint) {
76107
let idx = row * self.resolution + col;
77-
// Unbundle old from gestalt
108+
// Unbundle old from gestalt (approximate — see method doc).
78109
let old = self.heads[idx].clone();
110+
#[allow(deprecated)]
79111
unbundle_from(&old, &mut self.gestalt);
80112
// Bundle new into gestalt
81113
bundle_into(&head, &mut self.gestalt, self.epoch as f32, 1.0);
@@ -129,8 +161,11 @@ mod tests {
129161
assert_eq!(target.dims[d], expected, "dim {d} mismatch");
130162
}
131163

132-
// Unbundle b from target: should shift back toward a
164+
// Unbundle b from target: documents the approximate (not exact) behaviour.
165+
// This test verifies the arithmetic, NOT that round-trip fidelity holds
166+
// (it doesn't — unbundle_from is not the inverse of bundle_into).
133167
let before_unbundle = target.clone();
168+
#[allow(deprecated)]
134169
unbundle_from(&b, &mut target);
135170
// After unbundle, each dim should be before - b
136171
for d in 0..BASE_DIM {

‎docs/architecture/soa-three-tier-model.md‎

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -185,18 +185,22 @@ The `MailboxSoA<N>` columns are CPU-style registers: fixed width, fixed byte
185185
offset, little-endian, indexed by position. There is no schema in the row.
186186
The row is a register bank.
187187

188+
**Current / transitional layout** (target state removes `entity_type[N]` — see §3.2):
189+
188190
```
189-
Byte offset Width Column LE kind
190-
────────── ───── ────── ───────
191-
0 4·N energy[N] f32 × N
192-
4N N plasticity[N] u8 × N
191+
Byte offset Width Column LE kind
192+
────────── ───── ────── ───────
193+
0 4·N energy[N] f32 × N
194+
4N N plasticity[N] u8 × N
193195
5N 4·N last_active_cycle[N] u32 × N
194-
9N 8·N edges[N] u64 × N (CausalEdge64 LE)
195-
17N N qualia[N] u8 × N (QualiaI4_16D packed)
196-
18N 2·N meta[N] u16 × N (MetaWord LE)
197-
20N 2·N entity_type[N] u16 × N
198-
22N 4 mailbox_id u32
199-
22N+4 4 current_cycle u32
196+
9N 8·N edges[N] u64 × N (CausalEdge64 LE)
197+
17N N qualia[N] u8 × N (QualiaI4_16D packed)
198+
18N 2·N meta[N] u16 × N (MetaWord LE)
199+
20N 2·N entity_type[N] u16 × N ← TRANSITIONAL: scheduled
200+
for removal once O(1)
201+
HHTL lookup is the sole path
202+
22N 4 mailbox_id u32
203+
22N+4 4 current_cycle u32
200204
... ... (scalars follow)
201205
```
202206

‎docs/probes/q3-standing-wave-falsification.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -196,7 +196,7 @@ No such `f` exists in the codebase.
196196
|-------|--------|--------|
197197
| Binary `vsa_permute` is norm-preserving cyclic rotation | ✅ Correct | `ndarray/vsa.rs:326–349` |
198198
| Role-key orthogonality via disjoint slices | ✅ Correct-by-construction | `vsa/roles.rs`, `grammar/role_keys.rs` |
199-
| Baton carries `(u16 target, CausalEdge64)` across mailbox boundaries | ✅ Correct | `collapse_gate.rs:177` |
199+
| Baton carries `(u16 target, CausalEdge64)` across mailbox boundaries | ⚠️ Superseded | `collapse_gate.rs:177` — the write-side push carrier (baton) existed; but per `soa-three-tier-model.md` §target-state (2026-06-07) it is scheduled for removal. The SoA snapshot (read-side pull) replaces the inter-mailbox handoff. Marking ✅ here conflicts with the arch doc; this entry is a correction note, not a reversal of the probe finding. |
200200
| CAM-PQ codec is separate from VSA (I-VSA-IDENTITIES) | ✅ Correct | enforced architecturally |
201201
| `vsa16k_bind` = elementwise multiply (Hadamard product) | ✅ Correct | `fingerprint.rs:468` |
202202
| `vsa16k_bundle` = elementwise sum, no normalization | ✅ Correct | `fingerprint.rs:479` |

0 commit comments

Comments
 (0)