Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .claude/board/AGENT_LOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
## 2026-06-14 — edge-codec flavors: all three implemented, class-selectable, measured (ICC/Pearson/Cronbach/Spearman)

**Main thread (Opus 4.8 1M), branch `claude/wonderful-hawking-lodtql` (ndarray + lance-graph).** Operator (relaying a second session's 32×4-bit edge proposal + the key insight *"you could literally combine deterministic↔residue"*): *"implement all and allow the class>schema inheritance mapping to choose which flavor … measure all options and validate/invalidate ICC Pearson Cronbach alpha … using the JC crate and ndarray/crates/hpc/pillars."* Resolved the second session's false dichotomy: the deterministic part (nearest-centroid palette index, recomputed via AMX `matmul_i8_to_i32`) is the EdgeBlock byte unchanged; the residue is a value-slab 4-bit plane — so all flavors are *interpretations* of the locked 16-byte block, none changes `NODE_ROW_STRIDE`.

**Shipped:**
- **ndarray** (`d3b608f`): `hpc::reliability` (Pearson/Spearman/Cronbach α/ICC(2,1) + `FidelityReport`; the JC-consumable measurement layer — jc/pillar had only private Spearman, missing the other three) and `hpc::edge_codec` (Codebook k-means, `CoarseResidueCodec`, `ProductQuantizer`, `reconstruct_coarse`). Harness `examples/edge_codec_compare`. 16 unit + 9 doctests; lib clippy `-D warnings` clean.
- **lance-graph contract**: `EdgeCodecFlavor` enum (`CoarseOnly`/`CoarseResidue`/`Pq32x4`) + defaulted `ClassView::edge_codec_flavor` selector (non-breaking). +3 tests, 609 lib green, clippy clean. LATEST_STATE contract inventory updated (same commit).

**Measured (AMX host):** CoarseResidue dominates agreement (ICC 0.97–0.99, ρ 0.98, α 0.99 across blob+continuous); Pq32x4 preserves rank (ρ 0.60–0.67) but not absolute distance (ICC 0.11–0.29 — the Pearson-vs-ICC contrast working as designed); CoarseOnly collapses on continuous (ICC 0.003). AMX assign 100% vs scalar, 24–28 GMAC/s.

**Deferred/flagged:** turbovec PQ4 *throughput* path blocked on #493 P2 (turbovec `ndarray-simd` feature removed in `7fa217c`; polyfill fns gone). Fidelity is kernel-independent, so throughput-only follow-up. Per-class flavor STORAGE (override `edge_codec_flavor` from a class config) = follow-up in `lance-graph-ontology`. Also fixed bgz17 SIMD gather OOB (P1 from #493, commit `6d48ced`).

## 2026-06-13 — turbovec ⇄ ndarray integration: fork-wired + ndarray::simd polyfill GEMM + measured AMX-vs-LUT

**Main thread (Opus 4.8 1M) + 1 Opus general-purpose agent (bgz-tensor synergy map).** User: "create a crate in lance-graph for turbovec and check synergies; route SIMD through ndarray::simd (simd.rs→simd_amx/avx512/ops/soa); the polyfill does the work, ndarray ships AMX via byte-asm dispatch; pin rust 1.95." Cross-repo, branch `claude/wonderful-hawking-lodtql` in all three repos.
Expand Down
16 changes: 16 additions & 0 deletions .claude/board/INTEGRATION_PLANS.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,19 @@
## 2026-06-14 — entropy-ladder-spo-rung-v1 (Staunen↔Wisdom entropy coordinate unifies SPO rungs + NARS reliability + edge-codec; R1 shipped, R2–R6 roadmap)

**Status:** PLATEAU — R1 (foundation) SHIPPED; R2–R6 planned (probe-gated where marked). **Plan file:** `.claude/plans/entropy-ladder-spo-rung-v1.md`.
**Owns:** 7 deliverables D-EL-1..6 + D-EL-COCA.
- D-EL-1: entropy-ladder foundation — `ndarray::hpc::{reliability, edge_codec, entropy_ladder}` + `EdgeCodecFlavor` selector + bgz17 OOB fix (SHIPPED; ρ=−0.78 reliability proxy; ICC/Pearson/Cronbach/Spearman; edge-codec fidelity table)
- D-EL-2: `entropy_class` → CausalEdge64 spare bits [63:61] (version-gated, field-isolation tests) — next
- D-EL-3: CAM-PQ AMX centroid assignment (assignment-as-GEMM + 2×2/4×4 tiled grid) — queued
- D-EL-4: HHTL+helix basin attraction (probe-gated, +15% recall vs HHTL-alone)
- D-EL-5: Markov SPO rung-ladder → episodic context/basins/supporting edges (probe-gated)
- D-EL-6: energy axis / particle↔wave (gated on Mailbox-SoA map)
- D-EL-COCA: superposition 2/3 pruning (needs cluster-identity layer; I-VSA-IDENTITIES design-gate)
**Key decisions:** read the 3×palette256 SPO already in CausalEdge64 (NO re-quant); Pearl 2³ mask = the 8 SPO iterations; flavors/classes are interpretation not layout (no NODE_ROW_STRIDE / no ENVELOPE_LAYOUT_VERSION bump); COCA pruning must bundle identities not PQ codes (I-VSA-IDENTITIES).
**Repos:** ndarray (`d3b608f`, `83be7c3`) + lance-graph (`6d48ced`, `920671d`), branch `claude/wonderful-hawking-lodtql`. PRs: this plateau pair.

---

## 2026-06-09 — polyglot-container-query-membrane-v1 (Node Container answers Cypher + SurrealQL AST + DataFusion UDF over one HHTL address space; mailbox = a normal cold path)

**Status:** RESEARCH MAP + PLAN. Grounded by two parallel sweeps (lance-graph + surrealdb fork) with main-thread spot-verification; one agent claim caught false (SoaEnvelope has ZERO real impls — identity N3 stands live). **Plan file:** `.claude/plans/polyglot-container-query-membrane-v1.md`.
Expand Down
10 changes: 10 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -512,3 +512,13 @@ PR sequence: #360 → #361 → post-#360 substrate-sweep (this PR).
> **2026-06-01 — PR-in-flight (autoattended)** (D-EW64-3/4): `lance_graph_contract::episodic_edges` gains `EpisodicEdges64::{coldest, contains, promote_into}` + the `DemotionSink` trait. `coldest()` = the eviction victim (symmetric to `strongest()`); `contains()` = family-discriminating membership; `promote_into(e, sink)` = `promote` routing the evicted (coldest) edge to a `DemotionSink` — the hot→cold connectome exit. `DemotionSink` impls (surreal/LanceDB-LIVE "wingman", `E-SUBSTRATE-IS-THE-SCHEDULER`) are deferred + GATED on OQ-11.6. Zero-dep; contract lib 545 green; default clippy clean; `episodic_edges.rs` pedantic+nursery clean.

> **2026-06-01 — Shipped (autoattended, 5-agent council)** (D-ATOM-4/RawEdge): `contract::counterfactual` wired into `lib.rs` (was orphaned); `RawEdge(i8)` mantissa-only **structural** impl of `EpisodicEdge` (`size_of==1` — a u64 newtype could read plasticity 50–52); `deposit_counterfactual` v2 filled (−6 on split). Closes the counterfactual seam (NOT the prefetch loop). +3 latent scaffold fixes. 550 contract lib green, clippy clean. The council REFUTED the prior "compose `Heel.plasticity` × MRU" ① resolution (`E-BASIN-NOT-EDGE-PLASTICITY`): coarse strength = MRU slot-order (shipped); per-edge Hebbian = per-plane `PlasticityState` (gated).

## 2026-06-14 — Append: `EdgeCodecFlavor` selector + ndarray edge-codec/reliability layer (branch `claude/wonderful-hawking-lodtql`)

(Per APPEND-ONLY rule: new top-of-inventory entry. Branch work; records the contract type so a new session does not re-derive it.)

### Current Contract Inventory — new entry

- **`canonical_node::EdgeCodecFlavor`** (NEW; re-exported from `lib.rs`). Per-class selector for how a node's 16-byte `EdgeBlock` (+ optional value-slab residue) is *interpreted* — `CoarseOnly` (1 B palette index, the canon zero-fallback default), `CoarseResidue` (1 + ⌈D/2⌉ B, value-slab signed-4-bit residue), `Pq32x4` (16 B = 32×4-bit product code, the turbovec PQ model). `bytes_per_vector(dim)` + `is_layout_preserving()` (always `true`). **Iron invariant:** the flavor is interpretation, NOT layout — every variant leaves `NODE_ROW_STRIDE = 512` untouched, so adoption needs no `ENVELOPE_LAYOUT_VERSION` bump (canon "registry-resolved via `classid → ClassView`"). Default `CoarseOnly` matches the all-zero bootstrap. Selection surface: new defaulted `ClassView::edge_codec_flavor(&self, ClassId) -> EdgeCodecFlavor` (non-breaking — `RegistryClassView` inherits the default; per-class override is the follow-up wiring in `lance-graph-ontology`). +3 tests; contract lib 609 green; clippy `-D warnings` clean.
- **Encode/measure kernels live in `ndarray` (the hardware layer), not the contract:** `ndarray::hpc::edge_codec` (Codebook k-means, `CoarseResidueCodec`, `ProductQuantizer`, `reconstruct_coarse`) + `ndarray::hpc::reliability` (Pearson r, Spearman ρ, Cronbach α, ICC(2,1), `FidelityReport`). Harness `examples/edge_codec_compare` measures all flavors × {blob, continuous} regimes. **Measured:** CoarseResidue dominates agreement (ICC 0.97–0.99, ρ 0.98, α 0.99); Pq32x4 keeps rank (ρ 0.60–0.67) but not absolute distance (ICC 0.11–0.29); CoarseOnly collapses on continuous (ICC 0.003); AMX `matmul_i8_to_i32` assign 100% vs scalar, 24–28 GMAC/s. ndarray commit `d3b608f`.
- **Deferred (flagged):** turbovec PQ4 *throughput* path (the FastScan nibble-LUT for the Pq32x4 flavor) blocked on the **#493 P2** build break — `lance-graph-turbovec` requests the `ndarray-simd` turbovec feature that was REMOVED (turbovec commit `7fa217c`); the polyfill fns are gone. turbovec's API is end-to-end (owns encode+scan), so it is a *PQ4 flavor*, not a residue-nibble-scan primitive. Fidelity (what ICC/Pearson/α measure) is independent of the fast kernel, so this is throughput-only follow-up.
16 changes: 16 additions & 0 deletions .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,19 @@
## entropy-ladder-spo-rung-v1 — Staunen↔Wisdom entropy coordinate unifies SPO rungs + NARS reliability (R1 shipped; R2–R6 roadmap)

Plan path: `.claude/plans/entropy-ladder-spo-rung-v1.md`. Foundation: `ndarray::hpc::{reliability, edge_codec, entropy_ladder}`. Selector: `lance-graph-contract::EdgeCodecFlavor`.

| D-id | Title | Crate(s) / repo | Risk | Status | PR / Evidence |
|---|---|---|---|---|---|
| D-EL-1 | Entropy-ladder foundation (reliability + edge_codec + entropy_ladder + EdgeCodecFlavor + bgz17 fix) | ndarray + lance-graph-contract + bgz17 | LOW | **In PR** | `d3b608f`,`83be7c3`,`920671d`,`6d48ced`; ρ=−0.78; ICC 0.97–0.99 |
| D-EL-2 | `entropy_class` → CausalEdge64 spare bits [63:61] | `causal-edge` | MED | **Queued** | version-gated + field-isolation (I-LEGACY-API-FEATURE-GATED) |
| D-EL-3 | CAM-PQ AMX centroid assignment (GEMM + 2×2/4×4 grid) | `ndarray` | MED | **Queued** | bit-exact + GMAC/s probe |
| D-EL-4 | HHTL+helix basin attraction | `lance-graph` + `helix` | MED | **Probe queued** | +15% recall vs HHTL-alone gate |
| D-EL-5 | Markov SPO rung-ladder → episodic context | `deepnsm` / `lance-graph` | MED | **Probe queued** | prune-without-recall-loss gate |
| D-EL-6 | Energy axis / particle↔wave | `lance-graph` MailboxSoA | MED | **Blocked** | gated on Mailbox-SoA map |
| D-EL-COCA | Superposition 2/3 pruning (cluster-identity layer) | `deepnsm` | HIGH | **Design** | I-VSA-IDENTITIES design-gate |

---

## singleton-to-snapshot-nudge-v1 — every shared-mutable singleton → per-owner MailboxSoA + Arc-swap snapshot (7 deliverables; codebooks left as-is)

Plan path: `.claude/plans/singleton-to-snapshot-nudge-v1.md`. Companions: `bindspace-singleton-to-mailbox-soa-v1` (BindSpace dissolution), `cycle-coherent-soa-snapshot-v1` (snapshot mechanism). Debt: TD-UNBUNDLE-FROM-1.
Expand Down
105 changes: 105 additions & 0 deletions .claude/plans/entropy-ladder-spo-rung-v1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# Entropy-Ladder SPO Rung Decomposition — v1

> **Status:** PLATEAU — R1 (foundation) SHIPPED; R2–R6 planned (probe-gated where marked).
> **Confidence:** R1 measured (ρ=−0.78 reliability-proxy); R2/R3 unblocked; R4/R5 probe-first; R6 gated on the Mailbox-SoA map.
> **Date:** 2026-06-14
> **Branch:** `claude/wonderful-hawking-lodtql` (ndarray + lance-graph)
> **Owner:** main thread (Opus)

## The Click — one coordinate unifies the stack

**A fact's position on the cognitive ladder IS an entropy level.** The same
scalar orders SPO triples, NARS truth, and the linguistic ladder:

```
Staunen ── high entropy ── raw stimulus, not yet crystallized ── SYNTAX
│ │
(semantics sits between) (meaning)
│ │
Wisdom ── low entropy ── crystalline knowledge / settled fact ─ PRAGMATICS
```

`entropy = 1 − c·|2f − 1|` over a `CausalEdge64`'s NARS `(f, c)` — one minus the
decisiveness of the NARS expectation. Validated as a **reliability proxy**:
ρ(entropy, empirical prediction accuracy) = **−0.78** over a synthetic NARS
population (the more crystalline the edge, the more reliably its belief matches
fresh reality), grounded via `ndarray::hpc::reliability`.

## Architecture decisions (locked this session)

1. **No re-quantization.** The SPO rungs read the **3×palette-256 indices already
inside `CausalEdge64`** (`s_idx`/`p_idx`/`o_idx`) — "exact enough" (operator).
`cam_pq` is NOT used to re-encode SPO for the ladder.
2. **Pearl 2³ mask = the 8 SPO iterations.** The existing causal mask at
`CausalEdge64` [42:40] enumerates the 8 `(S,P,O)` subsets; `decompose_spo`
zeroes inactive components and emits an HHTL-routable `basin_key`.
3. **Flavors / classes are INTERPRETATION, not layout.** No `NODE_ROW_STRIDE`
change, no `ENVELOPE_LAYOUT_VERSION` bump (canon "registry-resolved via
`classid → ClassView`").
4. **deterministic↔residue.** Coarse = nearest-centroid palette index
(recomputed via AMX `matmul_i8_to_i32`); residue = signed-4-bit correction.
5. **I-VSA-IDENTITIES correction.** COCA "codebook superposition for 2/3 pruning"
must bundle cluster **identity fingerprints**, NOT the CAM-PQ codes
(superposing PQ codes is register-loss). Pruning needs a cluster-identity
layer that does not exist yet → design before build.
6. **Placement.** `ndarray` = hardware/math (codecs, reliability, entropy);
`lance-graph-contract` = the selector + the edge type; `causal-edge` = the
bit-storage; `lance-graph` = thinking (orchestration, basins, context).

## The Plateau (delivered) — R1

| Artifact | Repo / path | Commit |
|---|---|---|
| `hpc::reliability` (Pearson, Spearman, Cronbach α, ICC(2,1), `FidelityReport`) | ndarray `src/hpc/reliability.rs` | `d3b608f` |
| `hpc::edge_codec` (Codebook k-means, `CoarseResidueCodec`, `ProductQuantizer`, `reconstruct_coarse`) | ndarray `src/hpc/edge_codec.rs` | `d3b608f` |
| `examples/edge_codec_compare` (measure all flavors × regimes) | ndarray | `d3b608f` |
| `hpc::entropy_ladder` (`nars_entropy`, `EntropyRung`, `Quadrant`, `PEARL_SUBSETS`, `decompose_spo`, `entropy_class`) | ndarray `src/hpc/entropy_ladder.rs` | `83be7c3` |
| `examples/entropy_ladder_probe` (rung/quadrant partition + SPO decomposition) | ndarray | `83be7c3` |
| `EdgeCodecFlavor` + `ClassView::edge_codec_flavor` selector | lance-graph `lance-graph-contract` | `920671d` |
| bgz17 SIMD gather OOB guard (P1) | lance-graph `crates/bgz17/src/simd.rs` | `6d48ced` |

**Measured:**
- Edge-codec fidelity: CoarseResidue dominates agreement (ICC 0.97–0.99, ρ 0.98,
α 0.99); Pq32x4 preserves rank (ρ 0.60–0.67) but not absolute distance
(ICC 0.11–0.29); CoarseOnly collapses on continuous data (ICC 0.003). AMX
assign 100% vs scalar, 24–28 GMAC/s.
- Entropy ladder: ρ(entropy, accuracy) = −0.78; balanced rung/quadrant partitions;
crystalline SPO → H=0.107 (Pragmatics/Wisdom), ambiguous S-only → H=1.000
(Syntax/Staunen).

Tests: ndarray 28 new unit + 14 doctests; contract +3 (609 lib green); bgz17 +1.
All clippy `-D warnings` clean.

## Roadmap — R2..R6 + COCA

| Rung | What | Plugs into | Dependency / gate |
|---|---|---|---|
| **R1** | NARS f/c → validity/reliability + entropy coordinate | `nars_entropy` + `reliability` | ✅ **SHIPPED** (ρ=−0.78) |
| **R2** | Store the entropy/reliability class where it belongs | `entropy_class` → `CausalEdge64` spare bits [63:61] | `causal-edge` (zero-dep bit ops), **version-gated** + field-isolation tests (I-LEGACY-API-FEATURE-GATED) |
| **R3** | CAM-PQ AMX centroid assignment (the "scanning" win) | `matmul_i8_to_i32`; 2×2/4×4 tiled centroid grid | independent of the ladder; `cam_pq.rs:precompute_distances` → AMX variant + bit-exact/GMAC-s probe |
| **R4** | HHTL + helix basin attraction | `decompose_spo.basin_key` + `helix` residue | 🔬 **probe**: residue-Δ from basin centroid predicts re-access > HHTL-tier alone (+15% recall gate) |
| **R5** | Markov SPO rung-ladder → episodic context/basins/supporting edges | `EntropyRung` over a `CausalEdge64` stream | 🔬 **probe**: ladder prunes without recall loss; builds on R1+R2 |
| **R6** | Energy axis / particle↔wave | real `MailboxSoA.energy` → `Quadrant`'s energy input | ⏳ gated on the Mailbox-SoA map (in flight) |
| **COCA** | superposition 2/3 pruning between 2⁸ SPO | cluster-identity FP layer (NEW) | 🛑 **I-VSA-IDENTITIES**: bundle identities not PQ codes — design first |

## Probe specs (the gates)

- **R4 helix-basin** — synthetic/real episodic workload; for nodes in the same
HHTL basin, does a small Fisher-2z helix-residue Δ from the basin centroid
predict re-access better than HHTL-tier membership alone? Promote to FINDING
and wire only if ≥ +15% recall@k; else archive ("HHTL alone sufficient").
- **R5 Markov rung-ladder** — build word→clause→discourse centroid-SPO levels;
measure whether checking the low-entropy (Pragmatics) rung first prunes ≥ X%
of candidates with < Y% recall loss vs flat scan. Entropy band = the prune key.

## D-ids

- **D-EL-1** (R1) — entropy-ladder foundation + reliability + edge-codec — **Shipped**
- **D-EL-2** (R2) — entropy class in CausalEdge64 spare bits — **Queued** (next)
- **D-EL-3** (R3) — CAM-PQ AMX assignment — **Queued**
- **D-EL-4** (R4) — helix-basin attraction — **Probe queued**
- **D-EL-5** (R5) — Markov SPO rung-ladder context — **Probe queued**
- **D-EL-6** (R6) — energy axis / particle↔wave — **Blocked** (SoA map)
- **D-EL-COCA** — superposition pruning (cluster-identity layer) — **Design**

https://claude.ai/code/session_01D2WSmezQBNC3bUdHuGfGmo
Loading
Loading