Skip to content

Commit 920671d

Browse files
committed
feat(contract): EdgeCodecFlavor selector for class-chosen edge encoding
Adds canonical_node::EdgeCodecFlavor { CoarseOnly, CoarseResidue, Pq32x4 } and a defaulted ClassView::edge_codec_flavor(class) selector, so a class/schema picks how its 16-byte EdgeBlock (+ optional value-slab residue) is interpreted. Iron invariant (tested): the flavor is INTERPRETATION, not layout — every variant leaves NODE_ROW_STRIDE = 512 untouched, so adopting one needs no ENVELOPE_LAYOUT_VERSION bump (canon "registry-resolved via classid -> ClassView"). Default CoarseOnly matches the all-zero bootstrap reading. The trait method is defaulted, so RegistryClassView inherits it (non-breaking); per-class override is follow-up wiring in lance-graph-ontology. Encode/measure kernels live in ndarray (hpc::edge_codec + hpc::reliability, commit d3b608f) per the hardware/thinking split. Board: LATEST_STATE contract inventory + AGENT_LOG updated in this commit (mandatory board hygiene). +3 tests; contract lib 609 green; clippy -D warnings clean. https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
1 parent 6d48ced commit 920671d

5 files changed

Lines changed: 145 additions & 13 deletions

File tree

.claude/board/AGENT_LOG.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,15 @@
1+
## 2026-06-14 — edge-codec flavors: all three implemented, class-selectable, measured (ICC/Pearson/Cronbach/Spearman)
2+
3+
**Main thread (Opus 4.8 1M), branch `claude/wonderful-hawking-lodtql` (ndarray + lance-graph).** Operator (relaying a second session's 32×4-bit edge proposal + the key insight *"you could literally combine deterministic↔residue"*): *"implement all and allow the class>schema inheritance mapping to choose which flavor … measure all options and validate/invalidate ICC Pearson Cronbach alpha … using the JC crate and ndarray/crates/hpc/pillars."* Resolved the second session's false dichotomy: the deterministic part (nearest-centroid palette index, recomputed via AMX `matmul_i8_to_i32`) is the EdgeBlock byte unchanged; the residue is a value-slab 4-bit plane — so all flavors are *interpretations* of the locked 16-byte block, none changes `NODE_ROW_STRIDE`.
4+
5+
**Shipped:**
6+
- **ndarray** (`d3b608f`): `hpc::reliability` (Pearson/Spearman/Cronbach α/ICC(2,1) + `FidelityReport`; the JC-consumable measurement layer — jc/pillar had only private Spearman, missing the other three) and `hpc::edge_codec` (Codebook k-means, `CoarseResidueCodec`, `ProductQuantizer`, `reconstruct_coarse`). Harness `examples/edge_codec_compare`. 16 unit + 9 doctests; lib clippy `-D warnings` clean.
7+
- **lance-graph contract**: `EdgeCodecFlavor` enum (`CoarseOnly`/`CoarseResidue`/`Pq32x4`) + defaulted `ClassView::edge_codec_flavor` selector (non-breaking). +3 tests, 609 lib green, clippy clean. LATEST_STATE contract inventory updated (same commit).
8+
9+
**Measured (AMX host):** CoarseResidue dominates agreement (ICC 0.97–0.99, ρ 0.98, α 0.99 across blob+continuous); Pq32x4 preserves rank (ρ 0.60–0.67) but not absolute distance (ICC 0.11–0.29 — the Pearson-vs-ICC contrast working as designed); CoarseOnly collapses on continuous (ICC 0.003). AMX assign 100% vs scalar, 24–28 GMAC/s.
10+
11+
**Deferred/flagged:** turbovec PQ4 *throughput* path blocked on #493 P2 (turbovec `ndarray-simd` feature removed in `7fa217c`; polyfill fns gone). Fidelity is kernel-independent, so throughput-only follow-up. Per-class flavor STORAGE (override `edge_codec_flavor` from a class config) = follow-up in `lance-graph-ontology`. Also fixed bgz17 SIMD gather OOB (P1 from #493, commit `6d48ced`).
12+
113
## 2026-06-13 — turbovec ⇄ ndarray integration: fork-wired + ndarray::simd polyfill GEMM + measured AMX-vs-LUT
214

315
**Main thread (Opus 4.8 1M) + 1 Opus general-purpose agent (bgz-tensor synergy map).** User: "create a crate in lance-graph for turbovec and check synergies; route SIMD through ndarray::simd (simd.rs→simd_amx/avx512/ops/soa); the polyfill does the work, ndarray ships AMX via byte-asm dispatch; pin rust 1.95." Cross-repo, branch `claude/wonderful-hawking-lodtql` in all three repos.

.claude/board/LATEST_STATE.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -512,3 +512,13 @@ PR sequence: #360 → #361 → post-#360 substrate-sweep (this PR).
512512
> **2026-06-01 — PR-in-flight (autoattended)** (D-EW64-3/4): `lance_graph_contract::episodic_edges` gains `EpisodicEdges64::{coldest, contains, promote_into}` + the `DemotionSink` trait. `coldest()` = the eviction victim (symmetric to `strongest()`); `contains()` = family-discriminating membership; `promote_into(e, sink)` = `promote` routing the evicted (coldest) edge to a `DemotionSink` — the hot→cold connectome exit. `DemotionSink` impls (surreal/LanceDB-LIVE "wingman", `E-SUBSTRATE-IS-THE-SCHEDULER`) are deferred + GATED on OQ-11.6. Zero-dep; contract lib 545 green; default clippy clean; `episodic_edges.rs` pedantic+nursery clean.
513513
514514
> **2026-06-01 — Shipped (autoattended, 5-agent council)** (D-ATOM-4/RawEdge): `contract::counterfactual` wired into `lib.rs` (was orphaned); `RawEdge(i8)` mantissa-only **structural** impl of `EpisodicEdge` (`size_of==1` — a u64 newtype could read plasticity 50–52); `deposit_counterfactual` v2 filled (−6 on split). Closes the counterfactual seam (NOT the prefetch loop). +3 latent scaffold fixes. 550 contract lib green, clippy clean. The council REFUTED the prior "compose `Heel.plasticity` × MRU" ① resolution (`E-BASIN-NOT-EDGE-PLASTICITY`): coarse strength = MRU slot-order (shipped); per-edge Hebbian = per-plane `PlasticityState` (gated).
515+
516+
## 2026-06-14 — Append: `EdgeCodecFlavor` selector + ndarray edge-codec/reliability layer (branch `claude/wonderful-hawking-lodtql`)
517+
518+
(Per APPEND-ONLY rule: new top-of-inventory entry. Branch work; records the contract type so a new session does not re-derive it.)
519+
520+
### Current Contract Inventory — new entry
521+
522+
- **`canonical_node::EdgeCodecFlavor`** (NEW; re-exported from `lib.rs`). Per-class selector for how a node's 16-byte `EdgeBlock` (+ optional value-slab residue) is *interpreted* — `CoarseOnly` (1 B palette index, the canon zero-fallback default), `CoarseResidue` (1 + ⌈D/2⌉ B, value-slab signed-4-bit residue), `Pq32x4` (16 B = 32×4-bit product code, the turbovec PQ model). `bytes_per_vector(dim)` + `is_layout_preserving()` (always `true`). **Iron invariant:** the flavor is interpretation, NOT layout — every variant leaves `NODE_ROW_STRIDE = 512` untouched, so adoption needs no `ENVELOPE_LAYOUT_VERSION` bump (canon "registry-resolved via `classid → ClassView`"). Default `CoarseOnly` matches the all-zero bootstrap. Selection surface: new defaulted `ClassView::edge_codec_flavor(&self, ClassId) -> EdgeCodecFlavor` (non-breaking — `RegistryClassView` inherits the default; per-class override is the follow-up wiring in `lance-graph-ontology`). +3 tests; contract lib 609 green; clippy `-D warnings` clean.
523+
- **Encode/measure kernels live in `ndarray` (the hardware layer), not the contract:** `ndarray::hpc::edge_codec` (Codebook k-means, `CoarseResidueCodec`, `ProductQuantizer`, `reconstruct_coarse`) + `ndarray::hpc::reliability` (Pearson r, Spearman ρ, Cronbach α, ICC(2,1), `FidelityReport`). Harness `examples/edge_codec_compare` measures all flavors × {blob, continuous} regimes. **Measured:** CoarseResidue dominates agreement (ICC 0.97–0.99, ρ 0.98, α 0.99); Pq32x4 keeps rank (ρ 0.60–0.67) but not absolute distance (ICC 0.11–0.29); CoarseOnly collapses on continuous (ICC 0.003); AMX `matmul_i8_to_i32` assign 100% vs scalar, 24–28 GMAC/s. ndarray commit `d3b608f`.
524+
- **Deferred (flagged):** turbovec PQ4 *throughput* path (the FastScan nibble-LUT for the Pq32x4 flavor) blocked on the **#493 P2** build break — `lance-graph-turbovec` requests the `ndarray-simd` turbovec feature that was REMOVED (turbovec commit `7fa217c`); the polyfill fns are gone. turbovec's API is end-to-end (owns encode+scan), so it is a *PQ4 flavor*, not a residue-nibble-scan primitive. Fidelity (what ICC/Pearson/α measure) is independent of the fast kernel, so this is throughput-only follow-up.

crates/lance-graph-contract/src/canonical_node.rs

Lines changed: 108 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -45,29 +45,43 @@ impl NodeGuid {
4545
/// Panics (incl. const-eval) when `family` or `identity` exceed 24 bits — the
4646
/// silent-truncation footgun: distinct u32 inputs would otherwise collapse
4747
/// to the same stored key.
48-
pub const fn new(classid: u32, heel: u16, hip: u16, twig: u16, family: u32, identity: u32) -> Self {
48+
pub const fn new(
49+
classid: u32,
50+
heel: u16,
51+
hip: u16,
52+
twig: u16,
53+
family: u32,
54+
identity: u32,
55+
) -> Self {
4956
assert!(family <= 0x00FF_FFFF, "family must fit in 24 bits");
5057
assert!(identity <= 0x00FF_FFFF, "identity must fit in 24 bits");
5158
let c = classid.to_le_bytes();
5259
let h = heel.to_le_bytes();
5360
let p = hip.to_le_bytes();
5461
let t = twig.to_le_bytes();
55-
let f = family.to_le_bytes(); // low 3 bytes
62+
let f = family.to_le_bytes(); // low 3 bytes
5663
let i = identity.to_le_bytes(); // low 3 bytes
5764
Self([
5865
c[0], c[1], c[2], c[3], // 0..4 classid
59-
h[0], h[1], // 4..6 HEEL
60-
p[0], p[1], // 6..8 HIP
61-
t[0], t[1], // 8..10 TWIG
62-
f[0], f[1], f[2], // 10..13 family
63-
i[0], i[1], i[2], // 13..16 identity
66+
h[0], h[1], // 4..6 HEEL
67+
p[0], p[1], // 6..8 HIP
68+
t[0], t[1], // 8..10 TWIG
69+
f[0], f[1], f[2], // 10..13 family
70+
i[0], i[1], i[2], // 13..16 identity
6471
])
6572
}
6673

6774
/// Default-class, default-basin node: only `identity` discriminates.
6875
/// This is the bootstrap address while classid and family are zero.
6976
pub const fn local(identity: u32) -> Self {
70-
Self::new(Self::CLASSID_DEFAULT, 0, 0, 0, Self::FAMILY_DEFAULT, identity)
77+
Self::new(
78+
Self::CLASSID_DEFAULT,
79+
0,
80+
0,
81+
0,
82+
Self::FAMILY_DEFAULT,
83+
identity,
84+
)
7185
}
7286

7387
#[inline]
@@ -171,6 +185,60 @@ pub struct EdgeBlock {
171185
pub out_family: [u8; 4],
172186
}
173187

188+
/// Which edge-codec flavor a class uses to *read* its node's edge block.
189+
///
190+
/// The flavor is an INTERPRETATION of the canonical 16-byte [`EdgeBlock`] (plus
191+
/// an optional value-slab residue), selected per class via
192+
/// [`ClassView::edge_codec_flavor`](crate::class_view::ClassView::edge_codec_flavor)
193+
/// — never a change to [`NodeRow`]'s 512-byte layout. Every variant leaves
194+
/// [`NODE_ROW_STRIDE`] untouched (the canon "registry-resolved via
195+
/// `classid → ClassView`" rule), so adopting a flavor needs NO
196+
/// `ENVELOPE_LAYOUT_VERSION` bump.
197+
///
198+
/// Encode/reconstruct kernels live in `ndarray::hpc::edge_codec`; per-flavor
199+
/// fidelity is measured by `ndarray::hpc::reliability` (see the
200+
/// `edge_codec_compare` example — CoarseResidue dominates on agreement, Pq32x4
201+
/// preserves rank but not absolute distance). Default is [`CoarseOnly`], the
202+
/// zero-fallback reading that matches the canon all-zero bootstrap default.
203+
///
204+
/// [`CoarseOnly`]: EdgeCodecFlavor::CoarseOnly
205+
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default)]
206+
#[repr(u8)]
207+
pub enum EdgeCodecFlavor {
208+
/// 1 byte/vector: each edge byte is a palette/centroid index — the
209+
/// [`EdgeBlock`] read literally. The canon zero-fallback default.
210+
#[default]
211+
CoarseOnly = 0,
212+
/// 1 + ⌈D/2⌉ bytes: coarse index + a per-dimension signed-4-bit residue
213+
/// carried in the reserved value slab. Highest fidelity / agreement.
214+
CoarseResidue = 1,
215+
/// 16 bytes: the edge block read as 32 × 4-bit product-quantizer codes (the
216+
/// turbovec PQ model). Preserves neighbour *rank* better than absolute
217+
/// distance (low ICC, decent Spearman).
218+
Pq32x4 = 2,
219+
}
220+
221+
impl EdgeCodecFlavor {
222+
/// Per-vector byte cost for dimensionality `dim` (D even for the residue's
223+
/// nibble packing; `⌈D/2⌉` is used so odd D rounds up).
224+
#[inline]
225+
pub const fn bytes_per_vector(self, dim: usize) -> usize {
226+
match self {
227+
EdgeCodecFlavor::CoarseOnly => 1,
228+
EdgeCodecFlavor::CoarseResidue => 1 + dim.div_ceil(2),
229+
EdgeCodecFlavor::Pq32x4 => 16,
230+
}
231+
}
232+
233+
/// Every flavor re-interprets the SAME 512-byte node row — none changes
234+
/// [`NODE_ROW_STRIDE`], so no flavor requires a layout-version bump. This is
235+
/// the canon invariant, encoded so a regression test can assert it.
236+
#[inline]
237+
pub const fn is_layout_preserving(self) -> bool {
238+
true
239+
}
240+
}
241+
174242
/// One node = 4096 bit = 512 byte: key(16) | edges(16) | value(480).
175243
///
176244
/// The 480-byte value is deferred — energy/meta/qualia/entity_type, materialized
@@ -353,6 +421,37 @@ mod tests {
353421
assert_eq!(core::mem::size_of_val(&e), 16);
354422
}
355423

424+
#[test]
425+
fn edge_codec_flavor_default_is_coarse_only() {
426+
// Zero-fallback default: the all-zero reading is the canon bootstrap.
427+
assert_eq!(EdgeCodecFlavor::default(), EdgeCodecFlavor::CoarseOnly);
428+
assert_eq!(EdgeCodecFlavor::CoarseOnly as u8, 0);
429+
}
430+
431+
#[test]
432+
fn edge_codec_flavor_byte_costs() {
433+
// D = 128: coarse 1 B, residue 1 + 64 = 65 B, PQ fixed 16 B.
434+
assert_eq!(EdgeCodecFlavor::CoarseOnly.bytes_per_vector(128), 1);
435+
assert_eq!(EdgeCodecFlavor::CoarseResidue.bytes_per_vector(128), 65);
436+
assert_eq!(EdgeCodecFlavor::Pq32x4.bytes_per_vector(128), 16);
437+
// Odd D rounds the residue nibble count up.
438+
assert_eq!(EdgeCodecFlavor::CoarseResidue.bytes_per_vector(7), 1 + 4);
439+
}
440+
441+
#[test]
442+
fn every_flavor_preserves_node_layout() {
443+
// The canon invariant: a flavor is an interpretation, never a stride
444+
// change — so no flavor forces an ENVELOPE_LAYOUT_VERSION bump.
445+
for f in [
446+
EdgeCodecFlavor::CoarseOnly,
447+
EdgeCodecFlavor::CoarseResidue,
448+
EdgeCodecFlavor::Pq32x4,
449+
] {
450+
assert!(f.is_layout_preserving());
451+
}
452+
assert_eq!(NODE_ROW_STRIDE, core::mem::size_of::<NodeRow>());
453+
}
454+
356455
#[test]
357456
fn uniqueness_guard_is_noop_outside_bootstrap() {
358457
// family != 0 ⇒ no longer the bootstrap address: the guard is a no-op
@@ -414,10 +513,7 @@ mod tests {
414513

415514
#[test]
416515
fn node_row_column_table_sums_to_row_stride() {
417-
let total: usize = NODE_ROW_COLUMNS
418-
.iter()
419-
.map(|c| c.col_bytes_per_row())
420-
.sum();
516+
let total: usize = NODE_ROW_COLUMNS.iter().map(|c| c.col_bytes_per_row()).sum();
421517
assert_eq!(total, NODE_ROW_STRIDE);
422518
assert_eq!(NODE_ROW_STRIDE, core::mem::size_of::<NodeRow>());
423519
}

crates/lance-graph-contract/src/class_view.rs

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -204,6 +204,20 @@ pub trait ClassView {
204204
})
205205
.collect()
206206
}
207+
208+
/// Which edge-codec flavor this class reads its node edge block with.
209+
///
210+
/// Default is
211+
/// [`EdgeCodecFlavor::CoarseOnly`](crate::canonical_node::EdgeCodecFlavor::CoarseOnly)
212+
/// — the canon zero-fallback reading (each edge byte is a palette index). An
213+
/// implementor overrides this to let a class opt into residue or PQ fidelity.
214+
/// This is *selection only*: every flavor shares the SAME byte layout, so the
215+
/// choice never changes `NODE_ROW_STRIDE` (canon "registry-resolved via
216+
/// `classid → ClassView`", never a stride change).
217+
#[inline]
218+
fn edge_codec_flavor(&self, _class: ClassId) -> crate::canonical_node::EdgeCodecFlavor {
219+
crate::canonical_node::EdgeCodecFlavor::CoarseOnly
220+
}
207221
}
208222

209223
/// One populated field to render — the late-resolved `label` + its `predicate` key.

crates/lance-graph-contract/src/lib.rs

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -106,7 +106,7 @@ pub mod world_map;
106106
pub mod world_model;
107107

108108
// Re-exports for the most commonly used collapse_gate types.
109-
pub use canonical_node::{EdgeBlock, NodeGuid, NodeRow};
109+
pub use canonical_node::{EdgeBlock, EdgeCodecFlavor, NodeGuid, NodeRow};
110110
pub use class_view::{ClassId, ClassProjection, ClassView, FieldMask, RenderRow};
111111
pub use collapse_gate::{GateDecision, MailboxId, MergeMode};
112112
pub use episodic_edges::{EdgeRef, EpisodicEdges64};

0 commit comments

Comments
 (0)