Skip to content

Commit 2f1ad2e

Browse files
committed
feat(graph+contract): CLAM containment + CAKES nearest over HHTL prefixes (Inc 0, the manifold facet)
panCAKES == radix trie == HHTL (E-PANCAKES-IS-RADIX-IS-HHTL): the CLAM cluster tree IS the radix trie of the classid·HEEL·HIP·TWIG nibble paths already in the keys, so the structural neighborhood is pure prefix arithmetic, zero value decode. - NiblePath::common_prefix_depth (contract) — the radix-trie nearest-neighbor measure; longest-common-prefix = CAKES attraction. +1 unit test. - MailboxSoaView::hhtl_path_at (contract) — per-row HHTL NiblePath, deferred- binding default None (canon NodeRow already carries key(16); the override exposes what's there). - graph::mailbox_scan::clam_contained (is_ancestor_of = the radix subtree = CLAM cluster) + cakes_nearest (common_prefix_depth ranking, k-NN). Both key-only, zero value decode. - Tests: containment = radix subtree (rows 0,1,2 under 1·2; leaf narrows to 0); CAKES ranks by shared depth [(0,3),(1,2),(2,2)]; deferred-None yields nothing (coarser-facet fallback); F2 zero-value-decode extended to CLAM/CAKES (the GuardedSoa value columns still panic-guarded). 7/7 mailbox_scan + 21/21 hhtl green, clippy clean. This is the first dispatch-table facet beyond the classid node-match: proximity/ neighborhood resolves on the key (CLAM/CAKES), free, per E-CLAM-IS-THE-MANIFOLD- ENGINE. Edge-deref (EdgeBlock) + helix exact-location (value decode) are the next tiers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
1 parent 04576d9 commit 2f1ad2e

4 files changed

Lines changed: 195 additions & 1 deletion

File tree

.claude/board/EPIPHANIES.md

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,23 @@
1+
## 2026-06-18 — E-PANCAKES-IS-RADIX-IS-HHTL — panCAKES ≡ radix trie ≡ HHTL: the CLAM cluster tree is NOT a separate structure, it IS the radix trie of the HHTL prefixes already in the keys; so CLAM/CAKES = prefix arithmetic on the GUID, zero value decode
2+
3+
**Status:** FINDING (operator-stated identity; wired this commit). The unification that makes the manifold-geometry facet (`E-CLAM-IS-THE-MANIFOLD-ENGINE`) *free*: there is no CLAM tree to build and store — the tree IS the radix trie of the `classid·HEEL·HIP·TWIG` nibble paths that already live in every GUID key.
4+
5+
**The three are one structure seen three ways:**
6+
- **HHTL** = the cascade tiers in the key (`NiblePath` over `classid·HEEL·HIP·TWIG`).
7+
- **radix trie** = prefix tree; routing = bit-shift on the nibble path, not hash.
8+
- **panCAKES** = the compressed CLAM tree — and a CLAM cluster IS a radix-trie subtree (shared prefix = same cluster); the cluster structure IS the codec.
9+
10+
**Operational consequence (the wiring):** the CLAM/CAKES operations reduce to pure prefix arithmetic on the key, **zero value decode** —
11+
- **CLAM containment** (which cluster / subtree) = `NiblePath::is_ancestor_of` — the radix subtree under the query prefix.
12+
- **CAKES nearest** (ranked similar) = `NiblePath::common_prefix_depth` (added this commit) — longest-common-prefix ranking IS the entropy-scaling NN over the cluster tree; deeper shared prefix ⇒ same deeper cluster ⇒ nearer.
13+
- **panCAKES compression** = the trie itself (shared prefixes are the dedup).
14+
15+
No separate index, no scent-vector tree materialization for the *structural* neighborhood — the keys are the tree. (CAKES over *content scent vectors* in `ndarray::hpc::clam` remains the metric-space path for non-prefix similarity; this identity covers the HHTL-prefix structural neighborhood, the free tier.)
16+
17+
**Wired:** `NiblePath::common_prefix_depth`; `MailboxSoaView::hhtl_path_at` (deferred-binding, default `None`); `graph::mailbox_scan::{clam_contained, cakes_nearest}` over the View — all key-only, F2 zero-value-decode-guarded (#544). Cross-refs: `E-CLAM-IS-THE-MANIFOLD-ENGINE`, `E-ADJACENCY-IS-KEY-AND-EDGECODEC`, `E-GUID-IS-THE-GRAPH`, `hhtl::NiblePath`, `ndarray::hpc::clam` (CAKES/panCAKES/CHAODA).
18+
19+
---
20+
121
## 2026-06-18 — E-CLAM-IS-THE-MANIFOLD-ENGINE — the CLAM facet is not a containment check; it is the CAKES+CHAODA+LFD ensemble (ndarray `clam.rs`): containment + ranked-NN + anomaly + compression, one tree, one geometric measure
222

323
**Status:** FINDING (operator-stated, grounded in `ndarray/src/hpc/clam.rs` — CAKES arXiv:2309.05491 Partition Alg 1 / ρ-NN Alg 4 / DFS-sieve Alg 6, panCAKES Alg 2, **CHAODA Phase 4 `anomaly_scores` from the LFD distribution**; + `lance-graph/crates/perturbation-sim/src/chaoda.rs` CHAODA-lite which names ndarray's `ClamTree` as the production path). Enriches `E-ADJACENCY-IS-KEY-AND-EDGECODEC`: the "HHTL/CLAM neighborhood" facet is a whole geometry engine, not `is_ancestor_of`.

crates/lance-graph-contract/src/hhtl.rs

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -238,6 +238,35 @@ impl NiblePath {
238238
})
239239
}
240240

241+
/// The depth of the **longest common prefix** with `other` — the radix-trie
242+
/// nearest-neighbor measure. Larger ⇒ the two paths share more cascade tiers
243+
/// ⇒ they sit in the same deeper CLAM cluster ⇒ they are nearer.
244+
///
245+
/// This is the operational form of `panCAKES ≡ radix trie ≡ HHTL`
246+
/// (`E-PANCAKES-IS-RADIX-IS-HHTL`): CAKES nearest-neighbor over the cluster
247+
/// tree is *longest-common-prefix ranking* over the HHTL nibble paths — no
248+
/// separate tree to build, the keys ARE the tree. Pure prefix arithmetic on
249+
/// the key; never touches the value slab.
250+
#[must_use]
251+
pub const fn common_prefix_depth(self, other: Self) -> u8 {
252+
let max = if self.depth < other.depth {
253+
self.depth
254+
} else {
255+
other.depth
256+
};
257+
let mut d = 0u8;
258+
// Walk depth-by-depth while the aligned prefixes agree. `prefix(d)` is
259+
// `Some` for every d ≤ depth, so the unwraps below cannot fail.
260+
while d < max {
261+
let next = d + 1;
262+
match (self.prefix(next), other.prefix(next)) {
263+
(Some(a), Some(b)) if a.path == b.path && a.depth == b.depth => d = next,
264+
_ => break,
265+
}
266+
}
267+
d
268+
}
269+
241270
/// Lower a [`NodeGuid`](crate::canonical_node::NodeGuid) prefix to a 16-nibble
242271
/// `NiblePath`, the routing-path counterpart of the GUID's
243272
/// `classid · HEEL · HIP · TWIG` cascade (identity-architecture v1 §3).
@@ -456,6 +485,24 @@ mod tests {
456485
);
457486
}
458487

488+
#[test]
489+
fn common_prefix_depth_is_the_radix_nn_measure() {
490+
let a = NiblePath::root(1).child(2).child(3);
491+
let b = NiblePath::root(1).child(2).child(4);
492+
let c = NiblePath::root(1).child(2);
493+
let d = NiblePath::root(9);
494+
assert_eq!(a.common_prefix_depth(a), 3, "self ⇒ full depth");
495+
assert_eq!(a.common_prefix_depth(b), 2, "1·2 shared, leaf differs");
496+
assert_eq!(a.common_prefix_depth(c), 2, "ancestor ⇒ min depth");
497+
assert_eq!(a.common_prefix_depth(d), 0, "different basin ⇒ 0");
498+
assert_eq!(
499+
a.common_prefix_depth(b),
500+
b.common_prefix_depth(a),
501+
"symmetric"
502+
);
503+
assert_eq!(NiblePath::EMPTY.common_prefix_depth(a), 0);
504+
}
505+
459506
#[test]
460507
fn is_ancestor_of_is_cheap_prefix_reachability() {
461508
let mammal = NiblePath::root(0x0).child(0x3); // Endurant → …mammal

crates/lance-graph-contract/src/soa_view.rs

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,24 @@ pub trait MailboxSoaView {
8888
None
8989
}
9090

91+
/// The HHTL routing path ([`NiblePath`](crate::hhtl::NiblePath)) of `row`'s
92+
/// GUID key — the `classid·HEEL·HIP·TWIG` cascade lowered to a nibble path.
93+
/// This is the **radix-trie / CLAM cluster address** of the node
94+
/// (`panCAKES ≡ radix trie ≡ HHTL`): containment = `is_ancestor_of`,
95+
/// CAKES nearest = `common_prefix_depth`, both pure key arithmetic, **zero
96+
/// value decode**.
97+
///
98+
/// **Default = `None` (zero-fallback, deferred binding)** — same discipline as
99+
/// [`row_for_local_key`](MailboxSoaView::row_for_local_key): a view that has
100+
/// not materialized a per-row key/HHTL column returns `None`, and a CLAM/CAKES
101+
/// scan over it yields nothing (the consumer falls back to a coarser facet).
102+
/// An owner that carries the GUID key per row overrides this (the canon
103+
/// `NodeRow` already holds `key(16)`, so the override exposes what is there).
104+
#[inline]
105+
fn hhtl_path_at(&self, _row: usize) -> Option<crate::hhtl::NiblePath> {
106+
None
107+
}
108+
91109
// NOTE (follow-up): the qualia column (`QualiaI4_16D`) accessor is intentionally omitted —
92110
// add `fn qualia(&self) -> &[crate::qualia::QualiaI4_16D]` when the first consumer
93111
// (planner strategy selection) needs it; keep the read surface minimal until then.

crates/lance-graph/src/graph/mailbox_scan.rs

Lines changed: 110 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@
3434
//! lookup (via [`MailboxSoaView::row_for_local_key`]); edge dispatch lands once
3535
//! the representation boundary is resolved.
3636
37+
use lance_graph_contract::hhtl::NiblePath;
3738
use lance_graph_contract::soa_view::MailboxSoaView;
3839

3940
use crate::graph::graph_router::Backend;
@@ -87,6 +88,55 @@ pub fn match_node_by_local_key<V: MailboxSoaView>(view: &V, local_key: u64) -> O
8788
})
8889
}
8990

91+
/// **CLAM containment** — the rows in `query`'s subtree: every row whose HHTL
92+
/// path is a descendant-or-equal of `query` (`query.is_ancestor_of(path)`).
93+
///
94+
/// This is the `panCAKES ≡ radix trie ≡ HHTL` neighborhood (`E-CLAM-IS-THE-MANIFOLD-ENGINE`
95+
/// / `E-PANCAKES-IS-RADIX-IS-HHTL`): the CLAM cluster is the radix-trie subtree
96+
/// under the query prefix. Pure key arithmetic — **zero value decode**. Rows with
97+
/// no materialized HHTL path (`hhtl_path_at == None`) are skipped.
98+
pub fn clam_contained<V: MailboxSoaView>(view: &V, query: NiblePath) -> Vec<NodeMatch> {
99+
(0..view.n_rows())
100+
.filter(|&row| view.hhtl_path_at(row).is_some_and(|p| query.is_ancestor_of(p)))
101+
.map(|row| NodeMatch {
102+
row,
103+
backend: Backend::MailboxSoa,
104+
})
105+
.collect()
106+
}
107+
108+
/// **CAKES nearest** — the `k` rows nearest `query` by longest-common-prefix
109+
/// depth (descending), the radix-trie nearest-neighbor over the HHTL paths.
110+
///
111+
/// Returns `(NodeMatch, shared_depth)`; deeper shared prefix ⇒ nearer (same deeper
112+
/// CLAM cluster). Ties keep ascending row order (stable). Pure key arithmetic —
113+
/// **zero value decode**; rows without a materialized HHTL path are skipped. This
114+
/// is CAKES "attraction" expressed as `NiblePath::common_prefix_depth`
115+
/// (`E-CLAM-IS-THE-MANIFOLD-ENGINE`).
116+
pub fn cakes_nearest<V: MailboxSoaView>(
117+
view: &V,
118+
query: NiblePath,
119+
k: usize,
120+
) -> Vec<(NodeMatch, u8)> {
121+
let mut scored: Vec<(NodeMatch, u8)> = (0..view.n_rows())
122+
.filter_map(|row| {
123+
view.hhtl_path_at(row).map(|p| {
124+
(
125+
NodeMatch {
126+
row,
127+
backend: Backend::MailboxSoa,
128+
},
129+
query.common_prefix_depth(p),
130+
)
131+
})
132+
})
133+
.collect();
134+
// Descending by shared depth; stable sort preserves ascending row order on ties.
135+
scored.sort_by(|a, b| b.1.cmp(&a.1));
136+
scored.truncate(k);
137+
scored
138+
}
139+
90140
#[cfg(test)]
91141
mod tests {
92142
use super::*;
@@ -100,6 +150,7 @@ mod tests {
100150
class_ids: Vec<u16>,
101151
edges: Vec<u64>,
102152
keyed_rows: Vec<(u64, usize)>,
153+
paths: Vec<Option<NiblePath>>,
103154
}
104155

105156
impl MailboxSoaView for GuardedSoa {
@@ -141,6 +192,9 @@ mod tests {
141192
.find(|(k, _)| *k == local_key)
142193
.map(|(_, r)| *r)
143194
}
195+
fn hhtl_path_at(&self, row: usize) -> Option<NiblePath> {
196+
self.paths.get(row).copied().flatten()
197+
}
144198
}
145199

146200
fn sample() -> GuardedSoa {
@@ -149,6 +203,15 @@ mod tests {
149203
class_ids: vec![7, 9, 7, 9, 7],
150204
edges: vec![0; 5],
151205
keyed_rows: vec![(0xABCD, 3), (0x1234, 0)],
206+
// HHTL radix-trie paths (root basin 1):
207+
// row0: 1·2·3 row1: 1·2·4 row2: 1·2 row3: 1·5 row4: 9 (other basin)
208+
paths: vec![
209+
Some(NiblePath::root(1).child(2).child(3)),
210+
Some(NiblePath::root(1).child(2).child(4)),
211+
Some(NiblePath::root(1).child(2)),
212+
Some(NiblePath::root(1).child(5)),
213+
Some(NiblePath::root(9)),
214+
],
152215
}
153216
}
154217

@@ -193,9 +256,55 @@ mod tests {
193256
#[test]
194257
fn f2_zero_value_decode_the_scan_never_panics_on_value_columns() {
195258
// The GuardedSoa panics if energy()/meta_raw() are read. If this test
196-
// completes, the classid node-match touched ONLY the class column.
259+
// completes, the classid node-match + CLAM/CAKES touched ONLY the
260+
// class/HHTL key columns, never the value slab.
197261
let soa = sample();
198262
let _ = match_nodes_by_class(&soa, 7);
199263
let _ = match_node_by_local_key(&soa, 0x1234);
264+
let _ = clam_contained(&soa, NiblePath::root(1).child(2));
265+
let _ = cakes_nearest(&soa, NiblePath::root(1).child(2).child(3), 3);
266+
}
267+
268+
#[test]
269+
fn clam_contained_is_the_radix_subtree() {
270+
// query = 1·2 ⇒ its CLAM cluster = the radix subtree under 1·2:
271+
// rows 0 (1·2·3), 1 (1·2·4), 2 (1·2 itself). NOT 3 (1·5) or 4 (other basin 9).
272+
let soa = sample();
273+
let rows: Vec<usize> = clam_contained(&soa, NiblePath::root(1).child(2))
274+
.iter()
275+
.map(|m| m.row)
276+
.collect();
277+
assert_eq!(rows, vec![0, 1, 2]);
278+
// a deeper query narrows the subtree to the exact leaf.
279+
let leaf: Vec<usize> = clam_contained(&soa, NiblePath::root(1).child(2).child(3))
280+
.iter()
281+
.map(|m| m.row)
282+
.collect();
283+
assert_eq!(leaf, vec![0]);
284+
}
285+
286+
#[test]
287+
fn cakes_nearest_ranks_by_longest_common_prefix() {
288+
// query = 1·2·3 (row 0). Shared-prefix depths:
289+
// row0 1·2·3 →3, row1 1·2·4 →2, row2 1·2 →2, row3 1·5 →1, row4 9 →0.
290+
let soa = sample();
291+
let near = cakes_nearest(&soa, NiblePath::root(1).child(2).child(3), 3);
292+
let ranked: Vec<(usize, u8)> = near.iter().map(|(m, d)| (m.row, *d)).collect();
293+
assert_eq!(ranked, vec![(0, 3), (1, 2), (2, 2)], "nearest-3 by shared depth");
294+
assert!(near.iter().all(|(m, _)| m.backend == Backend::MailboxSoa));
295+
// k larger than n returns all rows, still depth-sorted descending.
296+
let all = cakes_nearest(&soa, NiblePath::root(1).child(2).child(3), 99);
297+
let depths: Vec<u8> = all.iter().map(|(_, d)| *d).collect();
298+
assert_eq!(depths, vec![3, 2, 2, 1, 0]);
299+
}
300+
301+
#[test]
302+
fn clam_cakes_skip_rows_with_no_materialized_path() {
303+
// A view with all-None hhtl paths (the deferred-binding default) yields
304+
// nothing — the consumer falls back to a coarser facet, never a wrong row.
305+
let mut soa = sample();
306+
soa.paths = vec![None; 5];
307+
assert!(clam_contained(&soa, NiblePath::root(1)).is_empty());
308+
assert!(cakes_nearest(&soa, NiblePath::root(1), 5).is_empty());
200309
}
201310
}

0 commit comments

Comments
 (0)