diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index c4eb2e39..ee8a8421 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,20 @@ +## 2026-06-19 — E-OUTAGE-CASCADE-IS-NON-LOCAL — CORRECTION to `E-BASIN-IS-A-NODE`: the electricity-outage perturbation does NOT decay with HHTL hop; the "cascade round = hop / hop bounds reach" identity is a property of the COGNITIVE substrate (propagation rides basin-tree EDGES by construction), NOT of the DC-power-flow electrical metaphor — they must not be conflated + +**Status:** FINDING (measured correction; probe `perturbation-sim/examples/outage_over_hhtl_hops.rs`). **Corrects** the second [H] sub-claim posted in `E-BASIN-IS-A-NODE` ("cascade round k = nodes at hop-distance k; Weyl bounds magnitude per round, hop-count bounds the reach"). Operator prompt: "model the electricity outage perturbation again with the HHTL L1-L4." The honest answer: I did, and it **refutes** the literal-electrical reading. + +**What the probe measured.** A sparse hierarchically-coupled backbone (8 leaf basins, Cheeger bisection recovers the L1-L4 HHTL tiers), seeded at the max-base-flow line, `simulate_outage` run, perturbation `node_field` and `trip_round` scored against the `node_distance(PrefixDepth)` tree-hop from the seed: +- magnitude vs hop is **non-monotone** (hop0 5.13, hop2 5.22, hop4 0.22, **hop6 rebounds to 2.82**), Spearman ρ = **−0.233** (weak); +- the cascade **islands** (the grid fragments — node_field becomes a least-norm proxy, not physical); +- trip-order vs hop ρ = **−0.20** (if anything, later rounds trip *closer*, the opposite of hop-local). + +So on the DC-power-flow model the outage is **non-local**: a far basin can feel as much shape as a near one, because flow redistributes globally (LODF), and the network islands. **The identity is NOT a property of the electrical metaphor.** + +**What survives (the load-bearing distinction).** The hop = cascade-level identity holds for the **cognitive substrate**, where propagation is constrained to the basin-tree EDGES *by construction* — a delta reaches a node only through its HHTL neighbours, so reach IS hop-bounded and `node_distance(PrefixDepth)` IS the propagation metric. The electrical DC cascade is the **non-local counter-model that proves the distinction**: do not import "cascade reach = hop" from the power-grid analogy into the substrate as if measured there. The cognitive case is edge-local (trivially hop-bounded); the electrical case is globally-coupled (not hop-bounded). `E-BASIN-IS-A-NODE`'s *other* axes are untouched — basin=node, distance=hop=`PrefixDepth` (the placement probe `basin_placement_learning.rs`, green 75.8 %, stands), Morton-pyramid distribution. Only the electrical-cascade-as-propagation framing is corrected. + +**Self-correction note (process):** the probe's first auto-verdict rubber-stamped ρ=−0.233 as "SUPPORTED" (threshold ρ<−0.2). That was the confirmation-bias trap (`adk-behavior-monitor`: "feels like success"). The verdict was tightened to require **monotone decay AND no islanding AND ρ<−0.5**; under the honest gate the result reads CORRECTED, not supported. Cross-refs: `E-BASIN-IS-A-NODE` (the corrected claim), `perturbation-sim::cascade::simulate_outage`, `mailbox_scan::node_distance` (the edge-local metric that IS hop-bounded). + +--- + ## 2026-06-19 — E-GUID-SELF-ROUTES-THE-BASIN-TREE — `memberof` needs no coarse-fingerprint table: the HHTL nibbles ALREADY IN the GUID (classid·HEEL·HIP·TWIG) are the basin-tree route; the parent's address = HHTL-tier truncation of the child's own GUID, so every ancestor's route key derives from one key by truncation, zero lookups for addresses **Status:** FINDING **[G]** for the truncation-is-route identity (pure key arithmetic; `NiblePath::parent`/`from_guid_prefix` already wired). Operator sharpening ("what about using the HHTL in the guid") — resolves the open cross-shard `memberof` question raised at `E-BASIN-IS-A-NODE` and corrects this session's own "hand the parent to a separate coarse-fingerprint table" framing. Sharpens `E-COARSE-QUANTIZER-IS-SCALE-FREE-ROUTER` and `E-TENANT-ANGLE-RANK-IS-CAM-PQ-ADC`. diff --git a/crates/lance-graph/src/graph/mailbox_scan.rs b/crates/lance-graph/src/graph/mailbox_scan.rs index 1389e573..9fc40d61 100644 --- a/crates/lance-graph/src/graph/mailbox_scan.rs +++ b/crates/lance-graph/src/graph/mailbox_scan.rs @@ -299,15 +299,22 @@ pub fn members(view: &V, basin_row: usize) -> Vec .collect() } -/// The resolution of a [`memberof`] query: the parent basin is either -/// materialized in this mailbox (`Local`) or lives in another shard, addressed -/// by its HHTL prefix (`Route`). +/// The resolution of a [`memberof`] query: the parent basin is materialized in +/// this mailbox (`Local`), lives in another shard addressed by its HHTL prefix +/// (`Route`), or the node IS a top-tier basin with no parent (`Top`). /// /// The GUID self-routes (`E-GUID-SELF-ROUTES-THE-BASIN-TREE`): the parent's HHTL /// prefix **is** the shard/route key (`E-COARSE-QUANTIZER-IS-SCALE-FREE-ROUTER` /// — the prefix is simultaneously the CLAM cluster key, the IVF cell, and the /// shard key). So an unmaterialized parent is a **`Route`, not an absence** — no /// separate coarse-fingerprint table is consulted; the prefix routes directly. +/// +/// `Top` is the genuine "no parent" case (the DOLCE top facet). It is **distinct +/// from [`memberof`] returning `None`**, which means the node's own HHTL path is +/// not materialized (the deferred-binding default of +/// [`MailboxSoaView::hhtl_path_at`]) — i.e. *unresolved, fall back to a coarser +/// facet*, NOT *no parent*. Conflating the two would silently stop routing a row +/// whose path simply has not been bound yet. #[derive(Debug, Clone, PartialEq, Eq)] pub enum BasinOf { /// The parent basin-node row, materialized in this mailbox. @@ -316,6 +323,10 @@ pub enum BasinOf { /// the shard that owns the prefix (the coarse router keys on exactly this). /// Zero value decode; the route key derives from the child's own GUID. Route(NiblePath), + /// The node is a top-tier basin (`NiblePath::parent() == None`) — the DOLCE + /// top facet, genuinely no parent. Distinct from `memberof` returning `None` + /// (path not materialized; unresolved). + Top, } /// **`memberof`** (many-to-one) — the basin a node belongs to: the parent path @@ -330,11 +341,28 @@ pub enum BasinOf { /// Returns: /// - `Some(BasinOf::Local(row))` — the parent basin-node is in this mailbox; /// - `Some(BasinOf::Route(prefix))` — the parent lives in another shard, addressed -/// by its HHTL prefix (route it; **never `None` for a node that has a parent**); -/// - `None` — only at the top tier (`parent() == None`, the basin/DOLCE top facet -/// has no parent). +/// by its HHTL prefix (route it; **never an absence for a node that has a parent**); +/// - `Some(BasinOf::Top)` — the node is a genuine top-tier basin: a **depth-1** +/// root (`NiblePath::root(0..16)`) whose `parent()` is `None`; +/// - `None` — **unresolved**: either the node's HHTL path is not materialized (the +/// deferred-binding default of [`MailboxSoaView::hhtl_path_at`]) OR it is the +/// **depth-0 `NiblePath::EMPTY` "no route" sentinel**. Both mean *fall back to a +/// coarser facet*, NOT "no parent". Kept distinct from `Top` so a yet-to-be-bound +/// row — or an explicit no-route sentinel — is not mistaken for a root and +/// silently dropped from routing. pub fn memberof(view: &V, member_row: usize) -> Option { - let parent = view.hhtl_path_at(member_row)?.parent()?; + // `None` here = path not materialized (deferred-binding) → unresolved. + let path = view.hhtl_path_at(member_row)?; + // The depth-0 `EMPTY` "no route" sentinel is ALSO unresolved — its `parent()` + // is `None` like a real root, but it is not a top-tier basin (it has no basin + // at all). Distinguish by depth before classifying `Top` (codex #550 P2). + if path.depth() == 0 { + return None; + } + // A real top-tier basin: a depth-1 root, path exists but has no parent. + let Some(parent) = path.parent() else { + return Some(BasinOf::Top); + }; Some( (0..view.n_rows()) .find(|&row| view.hhtl_path_at(row) == Some(parent)) @@ -689,8 +717,9 @@ mod tests { for m in members(&soa, 2) { assert_eq!(local_row(memberof(&soa, m.row)), Some(2)); } - // row4 (9) is a top-tier basin (depth 1) → parent() is None. - assert_eq!(memberof(&soa, 4), None); + // row4 (9) is a top-tier basin (depth 1) → parent() is None → Top, + // NOT None (None is reserved for an unmaterialized path). + assert_eq!(memberof(&soa, 4), Some(BasinOf::Top)); } #[test] @@ -701,6 +730,41 @@ mod tests { assert_eq!(memberof(&soa, 2), Some(BasinOf::Route(NiblePath::root(1)))); } + #[test] + fn memberof_unmaterialized_path_is_none_not_top() { + // Codex #549 P2: a deferred-binding row (hhtl_path_at == None) must return + // None (unresolved, fall back) — DISTINCT from a real top-tier basin, + // which returns Some(Top). Conflating them silently stops routing a + // not-yet-bound row. + let mut soa = sample(); + soa.paths[0] = None; // row0's path not materialized + assert_eq!( + memberof(&soa, 0), + None, + "unmaterialized path ⇒ None (fall back)" + ); + // row4 still a genuine top-tier basin ⇒ Some(Top), not None. + assert_eq!(memberof(&soa, 4), Some(BasinOf::Top)); + } + + #[test] + fn memberof_empty_sentinel_is_none_not_top() { + // Codex #550 P2: NiblePath::EMPTY (depth 0, the "no route" sentinel) has + // parent() == None like a real root, but it is NOT a top-tier basin — it + // has no basin at all. It must read as None (unresolved), not Some(Top), + // so the no-route fallback is preserved. + let mut soa = sample(); + soa.paths[4] = Some(NiblePath::EMPTY); // depth-0 sentinel + assert_eq!( + memberof(&soa, 4), + None, + "EMPTY (depth 0) ⇒ None (unresolved), never Top" + ); + // root(>=16) also yields EMPTY → same no-route classification. + soa.paths[4] = Some(NiblePath::root(16)); + assert_eq!(memberof(&soa, 4), None); + } + #[test] fn members_memberof_are_key_only_no_value_decode() { // F2: navigating the virtual basin tree must never touch the value slab. diff --git a/crates/perturbation-sim/Cargo.toml b/crates/perturbation-sim/Cargo.toml index e2d72471..2077847a 100644 --- a/crates/perturbation-sim/Cargo.toml +++ b/crates/perturbation-sim/Cargo.toml @@ -116,3 +116,7 @@ path = "examples/inertia_ingest.rs" [[example]] name = "basin_placement_learning" path = "examples/basin_placement_learning.rs" + +[[example]] +name = "outage_over_hhtl_hops" +path = "examples/outage_over_hhtl_hops.rs" diff --git a/crates/perturbation-sim/examples/outage_over_hhtl_hops.rs b/crates/perturbation-sim/examples/outage_over_hhtl_hops.rs new file mode 100644 index 00000000..2d228ed7 --- /dev/null +++ b/crates/perturbation-sim/examples/outage_over_hhtl_hops.rs @@ -0,0 +1,258 @@ +//! PROBE — the electricity-outage perturbation over the HHTL L1-L4 hops +//! (E-BASIN-IS-A-NODE, the second unmeasured [H] sub-claim). +//! +//! The claim under test: *"cascade round k = nodes at hop-distance k from the +//! seed; Weyl bounds the magnitude per round, hop-count bounds the reach."* — +//! i.e. the outage perturbation propagates **outward over the HHTL basin tree**, +//! so a node's perturbation magnitude / trip timing tracks its HHTL tree-hop +//! distance from the seed. +//! +//! Honest design. We build a **sparse hierarchically-coupled backbone**: 8 leaf +//! groups arranged as a ring whose inter-group bridges weaken with dendrogram +//! depth (siblings strong, the HEEL cut weakest). Recursive Cheeger bisection +//! (`hhtl_keys`) then recovers that hierarchy as the `(HEEL, HIP, TWIG)` tiers — +//! the L1-L4 path. A sparse backbone (not all-pairs) means a single trip causes +//! a *real* flow redistribution, not the near-zero noise an over-connected grid +//! produces. We seed the **actual maximum-base-flow line** (the most-loaded line, +//! whose loss perturbs the most) and measure, against the HHTL tree-hop from the +//! seed: +//! +//! (1) MAGNITUDE DECAY — does the per-node perturbation `node_field` (angle +//! deviation) decay as HHTL hop from the seed grows? (The defensible half: +//! perturbation attenuates with distance.) +//! (2) TRIP ORDER — for cascaded trips, does `trip_round` (the L1-L4 cascade +//! level) grow with HHTL hop? (The electrically-suspect half: DC power flow +//! redistributes *non-locally*, so this may NOT hold — and that is itself +//! the finding.) +//! +//! We report both with real numbers and let the data grade the claim. No assert +//! on (2): the point is to learn whether the electrical cascade is hop-local. +//! +//! Run: cargo run --manifest-path crates/perturbation-sim/Cargo.toml \ +//! --example outage_over_hhtl_hops + +use perturbation_sim::{ + dc_flows, hhtl_keys, simulate_outage, symmetric_eigen, CascadeConfig, Edge, Grid, HhtlKey, +}; + +const GROUPS: usize = 8; // 2^3 leaf basins → a clean depth-3 HHTL tree +const SIZE: usize = 6; // nodes per leaf group + +/// Sparse hierarchically-coupled backbone: dense intra-group rings + a *ring* of +/// inter-group bridges whose susceptance weakens with dendrogram depth. Cheeger +/// bisection recovers the dendrogram as the HHTL tiers; the sparsity makes a +/// single trip a real (non-degenerate) perturbation. +fn hierarchical_grid() -> Grid { + let n = GROUPS * SIZE; + let mut e = Vec::new(); + // Strong intra-group ring + chords. + for g in 0..GROUPS { + let base = g * SIZE; + for i in 0..SIZE { + for j in (i + 1)..SIZE { + if j == i + 1 || (i + j) % 3 == 0 { + e.push(Edge::new(base + i, base + j, 1.0, 1.5)); + } + } + } + } + // Dendrogram backbone (a ring 0-1-…-7-0), bridge node-0 ↔ node-0: + // TWIG (siblings) strong : 0-1, 2-3, 4-5, 6-7 + // HIP (pair-merge) medium : 1-2, 5-6 + // HEEL (half-merge) weak : 3-4 and the closure 7-0 + let bridge = |e: &mut Vec, g1: usize, g2: usize, b: f64| { + e.push(Edge::new(g1 * SIZE, g2 * SIZE, b, 0.6)); + }; + for s in [(0, 1), (2, 3), (4, 5), (6, 7)] { + bridge(&mut e, s.0, s.1, 1.0); // TWIG siblings + } + bridge(&mut e, 1, 2, 0.30); // HIP + bridge(&mut e, 5, 6, 0.30); // HIP + bridge(&mut e, 3, 4, 0.08); // HEEL cut + bridge(&mut e, 7, 0, 0.05); // HEEL closure (avoids islanding, weak) + Grid::new(n, e) +} + +fn common_prefix_depth(a: HhtlKey, b: HhtlKey) -> u32 { + if a.heel != b.heel { + 0 + } else if a.hip != b.hip { + 1 + } else if a.twig != b.twig { + 2 + } else { + 3 + } +} + +/// HHTL tree-hop (the mailbox_scan `node_distance(PrefixDepth)` metric, depth 3). +fn tree_hop(a: HhtlKey, b: HhtlKey) -> u32 { + let cpd = common_prefix_depth(a, b); + (3 - cpd) + (3 - cpd) +} + +/// Spearman rank correlation (small N; ties broken by position — a +/// direction-and-strength read). +fn spearman(xs: &[f64], ys: &[f64]) -> f64 { + let rank = |v: &[f64]| -> Vec { + let mut idx: Vec = (0..v.len()).collect(); + idx.sort_by(|&a, &b| v[a].partial_cmp(&v[b]).unwrap_or(std::cmp::Ordering::Equal)); + let mut r = vec![0.0; v.len()]; + for (rank, &i) in idx.iter().enumerate() { + r[i] = rank as f64; + } + r + }; + let (rx, ry) = (rank(xs), rank(ys)); + let n = xs.len() as f64; + let mean = (n - 1.0) / 2.0; + let (mut num, mut dx, mut dy) = (0.0, 0.0, 0.0); + for i in 0..xs.len() { + let (ax, ay) = (rx[i] - mean, ry[i] - mean); + num += ax * ay; + dx += ax * ax; + dy += ay * ay; + } + if dx == 0.0 || dy == 0.0 { + 0.0 + } else { + num / (dx.sqrt() * dy.sqrt()) + } +} + +fn main() { + let grid = hierarchical_grid(); + let keys = hhtl_keys(&grid); + let (n, m) = (grid.n, grid.edges.len()); + + // Balanced injection: source in group 0, sink in group 4 (across the HEEL + // cut — forces real flow over the weak backbone bottleneck). + let mut p = vec![0.0; n]; + p[0] = 1.0; + p[4 * SIZE] = -1.0; + + // Seed = the actual maximum-base-flow line (its loss perturbs the most). + let alive0 = vec![true; m]; + let eig0 = symmetric_eigen(&grid.laplacian_of(&alive0), n); + let theta0 = eig0.pseudo_apply(&p, 1e-9); + let flow0 = dc_flows(&grid, &alive0, &theta0); + let seed_line = (0..m) + .max_by(|&a, &b| { + flow0[a] + .abs() + .partial_cmp(&flow0[b].abs()) + .unwrap_or(std::cmp::Ordering::Equal) + }) + .unwrap(); + let seed_node = grid.edges[seed_line].from; + let seed_key = keys[seed_node]; + + let cfg = CascadeConfig { + overload_factor: 1.0, + max_rounds: 32, + rel_tol: 1e-9, + }; + let r = simulate_outage(&grid, &p, seed_line, cfg); + + let hop: Vec = (0..n) + .map(|i| f64::from(tree_hop(keys[i], seed_key))) + .collect(); + + println!("PROBE — outage perturbation over HHTL L1-L4 hops (E-BASIN-IS-A-NODE)"); + println!( + " grid: {GROUPS} leaf basins × {SIZE} nodes = {n} nodes, {m} lines; seed line {seed_line} \ + (node {seed_node}, |base flow|={:.3})", + flow0[seed_line].abs() + ); + println!( + " cascade: {} rounds, {} lines tripped ({:.1}%), islanded={}, Weyl satisfied={}", + r.rounds, + r.shape.n_tripped(), + r.fraction_tripped * 100.0, + r.islanded, + r.spectral.weyl_satisfied + ); + + // ── (1) MAGNITUDE DECAY: node_field vs HHTL hop ─────────────────────────── + println!("\n (1) perturbation magnitude vs HHTL hop from seed:"); + let mut buckets: std::collections::BTreeMap = + std::collections::BTreeMap::new(); + for (h, field) in hop.iter().zip(r.shape.node_field.iter()) { + let entry = buckets.entry(*h as u32).or_insert((0.0, 0)); + entry.0 += *field; + entry.1 += 1; + } + for (h, (sum, cnt)) in &buckets { + println!( + " hop {h}: mean |Δθ| = {:.3e} (n={cnt})", + sum / *cnt as f64 + ); + } + let rho_mag = spearman(&hop, &r.shape.node_field); + println!( + " Spearman ρ(hop, |Δθ|) = {rho_mag:+.3} (negative ⇒ magnitude decays with hop)" + ); + + // ── (2) TRIP ORDER: trip_round vs HHTL hop ──────────────────────────────── + println!("\n (2) cascade trip-round vs HHTL hop (tripped non-seed lines):"); + let mut trip_hops: Vec = Vec::new(); + let mut trip_rounds: Vec = Vec::new(); + for (e, edge) in grid.edges.iter().enumerate() { + if r.shape.trip_round[e] >= 1 { + trip_hops.push(hop[edge.from].min(hop[edge.to])); + trip_rounds.push(f64::from(r.shape.trip_round[e])); + } + } + if trip_hops.len() >= 3 { + let rho_trip = spearman(&trip_rounds, &trip_hops); + println!( + " {} cascaded trips; Spearman ρ(trip_round, hop) = {rho_trip:+.3} \ + (positive ⇒ later rounds trip farther out = hop-local cascade)", + trip_hops.len() + ); + } else { + println!( + " only {} cascaded trip(s) — too few for a trip-order correlation; \ + magnitude decay (1) carries the reach signal.", + trip_hops.len() + ); + } + + // ── Verdict — honest, not rubber-stamped ────────────────────────────────── + // SUPPORTED requires ALL of: a real (non-degenerate) field, a STRICTLY + // monotone decay across hop buckets, no islanding (else node_field is a + // least-norm proxy, not physical), and a clear ρ. A weak/non-monotone/ + // islanded ρ is NOT support — it is the non-locality finding. + let max_field = r.shape.node_field.iter().cloned().fold(0.0_f64, f64::max); + let bucket_means: Vec = buckets.values().map(|(s, c)| s / *c as f64).collect(); + let monotone_decay = bucket_means.windows(2).all(|w| w[1] <= w[0]); + println!("\n VERDICT:"); + if max_field < 1e-6 { + println!( + " [DEGENERATE] perturbation ~0 everywhere (max |Δθ|={max_field:.2e}) — seed trip \ + barely moved flow; ρ is noise. Needs a more loaded seed." + ); + } else if monotone_decay && !r.islanded && rho_mag < -0.5 { + println!( + " [SUPPORTED] magnitude decays MONOTONICALLY with HHTL hop (ρ={rho_mag:+.3}, no \ + islanding) — hop bounds reach on the electrical model too." + ); + } else { + println!( + " [CLAIM CORRECTED — electrical cascade is NON-LOCAL] magnitude does NOT decay \ + cleanly with HHTL hop (ρ={rho_mag:+.3}; monotone={monotone_decay}; islanded={}; \ + trip-order anti-correlated). DC power flow redistributes globally — a far basin can \ + feel as much shape as a near one, and the network islands. So the \ + 'cascade round = hop / hop bounds reach' identity is NOT a property of the electrical \ + DC-flow metaphor.", + r.islanded + ); + } + println!( + " WHAT SURVIVES: the hop=cascade-level identity is a property of the COGNITIVE substrate, \ + where propagation rides the basin-tree EDGES by construction (a delta reaches a node only \ + through its HHTL neighbours, so reach IS hop-bounded) — NOT of the unconstrained electrical \ + cascade. The two must not be conflated: `node_distance(PrefixDepth)` is the edge-local \ + metric; the DC outage is the non-local counter-model that proves the distinction." + ); +}