|
| 1 | +# Neurocomputing Integration: Session A (Cross-Repo) |
| 2 | +## ladybug-rs × CLAM × SPO × NARS × Gestalt |
| 3 | + |
| 4 | +**Jan Hübener — Ada Architecture — March 2026** |
| 5 | + |
| 6 | +--- |
| 7 | + |
| 8 | +## SCOPE |
| 9 | + |
| 10 | +You have write access to: **ladybug-rs, crewai-rust, n8n-rs, neo4j-rs** |
| 11 | +You have read access to: **rustynum** (do NOT modify rustynum — a separate session owns that) |
| 12 | + |
| 13 | +Your job: wire the SPO distance harvest, stripe shift detector, CLAM path encoding, causal trajectory recorder, and gestalt integration into ladybug-rs — consuming rustynum types via Cargo dependencies. |
| 14 | + |
| 15 | +--- |
| 16 | + |
| 17 | +## WHAT'S ALREADY COMMITTED |
| 18 | + |
| 19 | +Branch `claude/review-rustynum-pr-80-2zNy5` in ladybug-rs — 2 commits, +1,131 / -421 lines: |
| 20 | + |
| 21 | +**Commit 1** (SIMD cleanup): |
| 22 | +- `avx_engine.rs`: Rewrote 713→597 lines, all SIMD delegated to rustynum runtime dispatch |
| 23 | +- `hdr_cascade.rs`: Removed dead `mod simd` block (76 lines) |
| 24 | +- `scent.rs`: Removed dead AVX2 TODO stub (20 lines) |
| 25 | + |
| 26 | +**Commit 2** (gestalt + σ-calibration): |
| 27 | +- **Cargo.toml**: `rustynum-bnn` dependency added (CrossPlaneVote, CausalSaliency, HaloType, CausalTrajectory, ResonatorSnapshot, RifDiff now accessible) |
| 28 | +- **hdr_cascade.rs**: `MexicanHat::from_sigma_gate()` (excite=Discovery, inhibit=Hint), `AlienSearch::with_sigma_gate()`, `calibrate_from_sigma()`, `classify_sigma()`, `significance_to_signal()` |
| 29 | +- **gestalt.rs** (NEW, 965 lines): |
| 30 | + - `GestaltState` — GREEN/AMBER/RED/BLUE from CausalSaliency |
| 31 | + - `BundlingProposal` — Hold→Flow/Block lifecycle with NARS truth, evidence count, reviewer audit trail |
| 32 | + - `detect_bundling()` — per-plane Hamming → SO/PO/SP halo, 2-close+1-far = candidate |
| 33 | + - `TiltReport` — per-plane σ deviation detection |
| 34 | + - `PlaneCalibration` — per-axis SigmaGate (S-gate, P-gate, O-gate) with `from_plane_stats()` |
| 35 | + - `TruthTrajectory` — append-only event log (MatchesAdded, CounterEvidence, ReviewApproved/Rejected, MoreEvidenceRequested, StripeMigration) |
| 36 | + - `CollapseMode` — Research (auto>0.95), Production (human>0.80), Regulated (always human) |
| 37 | + - `AntialiasedSigma` — continuous σ with soft band boundaries, direct NARS confidence mapping |
| 38 | + |
| 39 | +**Architecture decisions already locked in:** |
| 40 | + |
| 41 | +| Decision | Implementation | |
| 42 | +|----------|---------------| |
| 43 | +| SigmaGate replaces hardcoded thresholds | `MexicanHat::from_sigma_gate()` — excite=Discovery, inhibit=Hint | |
| 44 | +| Tentative = `CollapseGate::Hold` | `BundlingProposal` lifecycle: Hold→Flow (approve) or Block (reject) | |
| 45 | +| Per-plane σ calibration | `PlaneCalibration` with independent S/P/O `SigmaGate` instances | |
| 46 | +| 3 collapse modes | Research / Production / Regulated — same audit trail, different threshold | |
| 47 | +| Continuous σ scoring | `AntialiasedSigma` — soft band boundaries, no information loss at edges | |
| 48 | +| Bundling = 2-of-3 plane agreement | `detect_bundling()` → PredicateInversion / AgentConvergence / TargetDivergence | |
| 49 | +| Three-query convergence | semantic + explicit + relational in same σ-bands = Eineindeutigkeit | |
| 50 | +| Variable-width CLAM paths | nibble width = confidence (short = dense = high confidence) | |
| 51 | + |
| 52 | +**Other repos**: `rust-toolchain.toml` (stable) added to crewai-rust, n8n-rs, neo4j-rs |
| 53 | + |
| 54 | +**Do NOT re-implement or modify any of the above.** Build on top of it. |
| 55 | + |
| 56 | +--- |
| 57 | + |
| 58 | +## SIMD DISPATCH RULE |
| 59 | + |
| 60 | +The dispatch chain across the codebase: |
| 61 | +1. **AVX-512 VPOPCNTDQ** — production target (this infra, Railway) |
| 62 | +2. **AVX2** — silent fallback (Jan's local U9-185H Arrow Lake, well-known popcount algorithm) |
| 63 | +3. **Scalar** — last-resort backup for exotic targets only |
| 64 | + |
| 65 | +All hot paths route through `rustynum_accel::slice_hamming()` or equivalent which handles CPUID dispatch internally. Do NOT replace one scalar implementation with another and call it done — that was a prior session's mistake. |
| 66 | + |
| 67 | +--- |
| 68 | + |
| 69 | +## SOURCE DOCUMENTS |
| 70 | + |
| 71 | +You will receive these detailed prompts alongside this meta prompt. Read them in order: |
| 72 | + |
| 73 | +| Document | What It Specifies | |
| 74 | +|----------|-------------------| |
| 75 | +| `spo_distance_harvest_cosine_replacement_prompt.md` | SPO distance as cosine replacement at 238× less cost. The core invention. | |
| 76 | +| `spo_distance_granularity_investigation.md` | 5 granularity options (A-E). Implement A+B (free), benchmark E. | |
| 77 | +| `sigma_stripe_shift_detector_addendum.md` | 0.5σ stripe migration as distributional shift detector | |
| 78 | +| `btree_clam_path_lineage_addendum.md` | B-tree channel = CLAM path = address + lineage + causality in one u16 | |
| 79 | +| `nars_causal_trajectory_hydration_prompt.md` | RIF-BNN resonator instrumentation, CausalTrajectory, EWM/BPReLU/RIF | |
| 80 | +| `3d_wave_awareness_substrate.md` | *Reference only* — theoretical framing (Berge, Piaget, DN, signed quinary) | |
| 81 | + |
| 82 | +Each document is self-contained with implementation-grade code examples. This meta prompt tells you what order to build them in and how they connect. |
| 83 | + |
| 84 | +--- |
| 85 | + |
| 86 | +## PHASE 2: SPO Distance Harvest |
| 87 | + |
| 88 | +**File: `ladybug-rs/src/spo_harvest.rs` (NEW)** |
| 89 | + |
| 90 | +This is the cosine replacement. 238× fewer cycles, 7.3× more information per computation. The detailed spec is in `spo_distance_harvest_cosine_replacement_prompt.md`. Key deliverables: |
| 91 | + |
| 92 | +### 2.1 Core Types |
| 93 | + |
| 94 | +```rust |
| 95 | +pub struct SpoDistanceResult { |
| 96 | + pub similarity: f32, // aggregate ∈ [-1, 1] — cosine-compatible |
| 97 | + pub s_p_similarity: f32, // X-axis (Subject⊕Predicate) |
| 98 | + pub p_o_similarity: f32, // Y-axis (Predicate⊕Object) |
| 99 | + pub s_o_similarity: f32, // Z-axis (Subject⊕Object) |
| 100 | + pub halo: TypedHalo, // cross-plane vote (7 disjoint types) |
| 101 | + pub x_dist: u32, // raw popcount preserved (Phase 3 Option A) |
| 102 | + pub y_dist: u32, |
| 103 | + pub z_dist: u32, |
| 104 | +} |
| 105 | + |
| 106 | +impl SpoDistanceResult { |
| 107 | + pub fn as_cosine(&self) -> f32 { self.similarity } // drop-in |
| 108 | +} |
| 109 | +``` |
| 110 | + |
| 111 | +### 2.2 Functions to Build |
| 112 | + |
| 113 | +``` |
| 114 | +spo_distance(a, b) → SpoDistanceResult — the core 13-cycle computation |
| 115 | +harvest_to_nars(result) → NarsTruth — frequency from core ratio, confidence from entropy |
| 116 | +harvest_to_inference(result) → TypedInference — dominant halo type → typed query action |
| 117 | +AccumulatedHarvest::accumulate(result) — EMA + NARS revision across searches |
| 118 | +feed_sigma_graph(result) → Vec<SigmaEdge> — emit typed edges from harvest |
| 119 | +``` |
| 120 | + |
| 121 | +### 2.3 Key Constraint |
| 122 | + |
| 123 | +The XOR bitmasks (`x_xor`, `y_xor`, `z_xor`) computed for distance are the SAME bitmasks used for `cross_plane_vote()`. The halo extraction is FREE — no extra compute. Do not compute XOR twice. |
| 124 | + |
| 125 | +--- |
| 126 | + |
| 127 | +## PHASE 3: Distance Granularity + Stripe Shift Detector |
| 128 | + |
| 129 | +**Extends: `spo_harvest.rs` + new `shift_detector.rs`** |
| 130 | + |
| 131 | +Detailed specs in `spo_distance_granularity_investigation.md` and `sigma_stripe_shift_detector_addendum.md`. |
| 132 | + |
| 133 | +### 3.1 Option A: Raw Popcount (FREE) |
| 134 | + |
| 135 | +Check if `hdr_cascade.rs` currently discards the raw popcount after σ-binning. If yes, stop discarding it. The `SpoDistanceResult` already has `x_dist`, `y_dist`, `z_dist` fields — just make sure the cascade path also preserves them. |
| 136 | + |
| 137 | +### 3.2 Option B: Per-Word Histogram (NEAR-FREE) |
| 138 | + |
| 139 | +Check if per-word popcounts are computed individually and summed. If yes, keep the 32 individual counts: |
| 140 | + |
| 141 | +```rust |
| 142 | +pub struct PerWordHistogram { |
| 143 | + pub words: [[u16; 32]; 3], // 32 per-word popcounts × 3 planes |
| 144 | +} |
| 145 | +``` |
| 146 | + |
| 147 | +### 3.3 Stripe Shift Detector |
| 148 | + |
| 149 | +5 σ-thresholds (1.0σ, 1.5σ, 2.0σ, 2.5σ, 3.0σ). Cost: 5 CMP instructions. |
| 150 | + |
| 151 | +```rust |
| 152 | +pub struct StripeHistogram { |
| 153 | + pub below_1s: u32, |
| 154 | + pub s1_to_s15: u32, |
| 155 | + pub s15_to_s2: u32, |
| 156 | + pub s2_to_s25: u32, |
| 157 | + pub s25_to_s3: u32, |
| 158 | + pub above_3s: u32, |
| 159 | +} |
| 160 | + |
| 161 | +pub struct ShiftDetector { |
| 162 | + pub current: [StripeHistogram; 3], // S, P, O planes |
| 163 | + pub previous: [StripeHistogram; 3], |
| 164 | +} |
| 165 | + |
| 166 | +impl ShiftDetector { |
| 167 | + pub fn detect_shift(&self) -> ShiftSignal { /* KL divergence on 6 bins per plane */ } |
| 168 | +} |
| 169 | +``` |
| 170 | + |
| 171 | +**Wire into CollapseGate** (already exists in gestalt.rs): |
| 172 | +- Shift toward noise → bias HOLD |
| 173 | +- Shift toward foveal → bias FLOW |
| 174 | +- Bimodal → speciation event |
| 175 | + |
| 176 | +--- |
| 177 | + |
| 178 | +## PHASE 5: B-tree Channel = CLAM Path |
| 179 | + |
| 180 | +**File: `ladybug-rs/src/clam_path.rs` (NEW)** |
| 181 | + |
| 182 | +Detailed spec in `btree_clam_path_lineage_addendum.md`. |
| 183 | + |
| 184 | +### 5.1 ClamPath Type |
| 185 | + |
| 186 | +```rust |
| 187 | +#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] |
| 188 | +pub struct ClamPath { |
| 189 | + pub bits: u16, |
| 190 | + pub depth: u8, |
| 191 | +} |
| 192 | + |
| 193 | +impl ClamPath { |
| 194 | + pub fn from_tree_traversal(cluster_path: &[bool]) -> Self; |
| 195 | + pub fn subtree_range(&self) -> (u16, u16); // range query = lineage traversal |
| 196 | + pub fn common_ancestor_depth(&self, other: &ClamPath) -> u8; |
| 197 | + pub fn lineage_distance(&self, other: &ClamPath) -> u8; |
| 198 | + pub fn siblings(&self) -> ClamPath; // counterfactual query |
| 199 | +} |
| 200 | +``` |
| 201 | + |
| 202 | +### 5.2 Wire Into CogRecord |
| 203 | + |
| 204 | +```rust |
| 205 | +impl CogRecord { |
| 206 | + pub fn set_clam_path(&mut self, path: ClamPath) { |
| 207 | + self.btree_channel[0..2].copy_from_slice(&path.bits.to_be_bytes()); |
| 208 | + self.btree_channel[2] = path.depth; |
| 209 | + } |
| 210 | + pub fn clam_path(&self) -> ClamPath { /* inverse */ } |
| 211 | +} |
| 212 | +``` |
| 213 | + |
| 214 | +### 5.3 Triple Semantics |
| 215 | + |
| 216 | +One u16. Three query types. O(log n + k): |
| 217 | +- **Address**: Domain.Node.branch.twig.leaf |
| 218 | +- **Lineage**: ancestor → parent → self → child (range scan = phylogeny) |
| 219 | +- **Causality**: cause → mediator → effect (range scan = causal chain) |
| 220 | + |
| 221 | +--- |
| 222 | + |
| 223 | +## PHASE 6: Causal Trajectory Recorder |
| 224 | + |
| 225 | +**File: `ladybug-rs/src/causal_trajectory.rs` (NEW)** |
| 226 | + |
| 227 | +Detailed spec in `nars_causal_trajectory_hydration_prompt.md`. This is the biggest new module. |
| 228 | + |
| 229 | +### 6.1 Core Structures |
| 230 | + |
| 231 | +```rust |
| 232 | +pub struct CausalTrajectory { |
| 233 | + pub input: CogRecord3D, |
| 234 | + pub snapshots: Vec<ResonatorSnapshot>, |
| 235 | + pub rif_diffs: Vec<RifDiff>, |
| 236 | + pub ewm_corrections: Vec<EwmCorrection>, |
| 237 | + pub causal_arrows: Vec<CausalArrow>, |
| 238 | + pub nars_statements: Vec<NarsCausalStatement>, |
| 239 | + pub sigma_edges: Vec<SigmaEdge>, |
| 240 | +} |
| 241 | + |
| 242 | +pub struct CausalSaliency { |
| 243 | + pub crystallizing: BitVec, // dimensions gaining certainty |
| 244 | + pub dissolving: BitVec, // dimensions losing certainty |
| 245 | + pub contested: BitVec, // stuck transitional >3 iters |
| 246 | +} |
| 247 | +``` |
| 248 | + |
| 249 | +### 6.2 Per-Iteration Instrumentation |
| 250 | + |
| 251 | +After each resonator update (unbind → project → rebind): |
| 252 | + |
| 253 | +1. **RIF diff** — XOR(iter_t, iter_{t-2}) → what changed in 2 steps (causal chain) |
| 254 | +2. **EWM correction** — per-dimension tier classification → WHERE causation acts (saliency) |
| 255 | +3. **BPReLU arrow** — forward/backward asymmetry → WHICH DIRECTION it flows |
| 256 | +4. **Halo transition** — promotions/demotions → NARS statements |
| 257 | + |
| 258 | +### 6.3 Growth Paths (6 types) |
| 259 | + |
| 260 | +``` |
| 261 | +SubjectFirst — S crystallizes first → strong entity |
| 262 | +PredicateFirst — P crystallizes first → strong relation |
| 263 | +ObjectFirst — O crystallizes first → strong target |
| 264 | +SPFirst — S+P simultaneous → "who does what" clear |
| 265 | +SOFirst — S+O simultaneous → "who and whom" clear |
| 266 | +FullSimultaneous — all three at once → Gestalt snap (rare) |
| 267 | +``` |
| 268 | + |
| 269 | +### 6.4 Cost Budget |
| 270 | + |
| 271 | +~15μs overhead per resonator iteration (~30% on existing ~50μs). Acceptable for: full causal trajectory, NARS truth grounded in convergence dynamics, Sigma Graph edges, DN mutation guidance, warm-start capability. |
| 272 | + |
| 273 | +--- |
| 274 | + |
| 275 | +## PHASE 7: Gestalt Integration |
| 276 | + |
| 277 | +**Extends existing: `ladybug-rs/src/gestalt.rs` (DO NOT rewrite — add to it)** |
| 278 | + |
| 279 | +### 7.1 Wire detect_bundling() Into CLAM Harvest Loop |
| 280 | + |
| 281 | +When `AccumulatedHarvest` SO/SP/PO evidence crosses `CollapseMode` threshold → auto-create `BundlingProposal`. |
| 282 | + |
| 283 | +### 7.2 Expose TruthTrajectory via Semantic Redis Protocol |
| 284 | + |
| 285 | +``` |
| 286 | +GET ada:clam:bundled_with:* → tentative proposals, "status": "tentative" |
| 287 | +GET ada:clam:truth:* → current NARS truth + trajectory summary |
| 288 | +SCAN ada:clam:shift:* → stripe migration history |
| 289 | +``` |
| 290 | + |
| 291 | +### 7.3 Connect to neo4j-rs |
| 292 | + |
| 293 | +Edge types: `PROPOSED_BUNDLE` (tentative), `BUNDLED` (committed) |
| 294 | +Properties: NARS truth values, σ-significance, reviewer audit trail, growth path |
| 295 | + |
| 296 | +### 7.4 Cross-Validate CHAODA with Stripe Shift |
| 297 | + |
| 298 | +```rust |
| 299 | +fn chaoda_confirms_shift(anomaly: &AnomalyResult, shift: &ShiftSignal) -> ConfirmedShift { |
| 300 | + match (anomaly.calibration_type, shift.direction) { |
| 301 | + (Schaltminute, TowardNoise) => ConfirmedShift::GlobalDrift { confidence: 0.95 }, |
| 302 | + (Schaltsekunde, TowardFoveal) => ConfirmedShift::LocalRefinement { confidence: 0.85 }, |
| 303 | + (Schaltminute, Bimodal) => ConfirmedShift::Speciation { confidence: 0.90 }, |
| 304 | + _ => ConfirmedShift::Monitoring { confidence: 0.5 }, |
| 305 | + } |
| 306 | +} |
| 307 | +``` |
| 308 | + |
| 309 | +--- |
| 310 | + |
| 311 | +## EINEINDEUTIG MAPPING TABLE |
| 312 | + |
| 313 | +Every concept has ONE canonical address across ALL systems. Bijective. |
| 314 | + |
| 315 | +| Concept | Redis Key | Neo4j Property | CLAM Path | Lance Column | Semantic Redis | |
| 316 | +|---------|-----------|---------------|-----------|--------------|----------------| |
| 317 | +| SPO triple | `ada:spo:{hash}` | `n.spo_hash` | `ClamPath.bits` | `btree_key` | `GET spo:{hash}` | |
| 318 | +| NARS truth | `ada:truth:{hash}` | `e.frequency, e.confidence` | — | `meta` field | `GET truth:{hash}` | |
| 319 | +| Halo type | `ada:halo:{hash}` | `e.halo_type` | — | derived | `GET halo:{hash}` | |
| 320 | +| Trajectory | `ada:trajectory:{hash}` | — | — | `meta` field | `GET trajectory:{hash}` | |
| 321 | +| Bundle proposal | `ada:bundle:{id}` | `PROPOSED_BUNDLE` edge | shared prefix | `meta` field | `GET bundle:{id}` | |
| 322 | +| Shift signal | `ada:shift:{window}` | — | — | — | `GET shift:{window}` | |
| 323 | +| Anomaly score | `ada:anomaly:{hash}` | `n.chaoda_score` | leaf depth | — | `ANOMALY {hash}` | |
| 324 | +| σ-significance | `ada:sigma:{hash}` | `n.sigma_level` | — | `meta` field | `NARS {hash}` | |
| 325 | + |
| 326 | +### CogRecord 4-Channel Layout |
| 327 | + |
| 328 | +``` |
| 329 | +META (2KB): timestamps, σ-significance, convergence metadata, trajectory summary |
| 330 | +CAM (2KB): content-addressable fingerprint — Hamming search via VPOPCNTDQ |
| 331 | +B-tree(2KB): CLAM path — structural/lineage/causal range queries |
| 332 | +Embed (2KB): SPO 3-axis XOR encoding — cosine-compatible + typed halo harvest |
| 333 | +``` |
| 334 | + |
| 335 | +--- |
| 336 | + |
| 337 | +## EXECUTION ORDER |
| 338 | + |
| 339 | +``` |
| 340 | +Phase 2 → Core value (SPO distance harvest — the cosine replacement) |
| 341 | +Phase 3 → Near-free upgrades on Phase 2 (raw popcount, histogram, stripe shift) |
| 342 | +Phase 5 → ClamPath encoding (type definitions, CogRecord wire) |
| 343 | +Phase 6 → Causal trajectory recorder (biggest module, uses Phase 2 types) |
| 344 | +Phase 7 → Wires Phase 2-6 into existing gestalt.rs + neo4j-rs + Redis |
| 345 | +``` |
| 346 | + |
| 347 | +### Minimum Viable |
| 348 | + |
| 349 | +If time is limited: **Phase 2 + Phase 3**. That gives: |
| 350 | +- `spo_distance()` as cosine replacement at 238× less cost |
| 351 | +- `harvest_to_nars()` + `harvest_to_inference()` from every search |
| 352 | +- `AccumulatedHarvest` (search learns from searching) |
| 353 | +- Raw popcount + stripe shift detector at zero overhead |
| 354 | +- Wire into existing `CollapseGate` via `ShiftSignal` |
| 355 | + |
| 356 | +Everything else builds on that foundation. |
| 357 | + |
| 358 | +--- |
| 359 | + |
| 360 | +## WHAT NOT TO CHANGE |
| 361 | + |
| 362 | +``` |
| 363 | +DO NOT modify rustynum (separate session owns it) |
| 364 | +DO NOT rewrite gestalt.rs (add to it, don't replace it) |
| 365 | +DO NOT touch the SIMD dispatch in avx_engine.rs (already cleaned up) |
| 366 | +
|
| 367 | +KEEP: AVX-512 VPOPCNTDQ path (more specialized than CLAM's distances crate) |
| 368 | +KEEP: BindSpace O(1) content-addressable lookup |
| 369 | +KEEP: XOR retrieval (A⊗verb⊗B=A — VSA-specific, unique contribution) |
| 370 | +KEEP: Arrow/Lance integration (columnar, not Vec<(Id, I)>) |
| 371 | +KEEP: COW immutability (freeze after build) |
| 372 | +KEEP: HDR cascade + σ-significance (CLAM has no multi-resolution cascade) |
| 373 | +KEEP: SPO typed halo harvest (novel contribution, no equivalent anywhere) |
| 374 | +``` |
| 375 | + |
| 376 | +--- |
| 377 | + |
| 378 | +## TEST EXPECTATIONS |
| 379 | + |
| 380 | +```bash |
| 381 | +cargo test --package ladybug-rs -- spo_distance # harvest works |
| 382 | +cargo test --package ladybug-rs -- shift_detector # stripe migration works |
| 383 | +cargo test --package ladybug-rs -- clam_path # encoding round-trips |
| 384 | +cargo test --package ladybug-rs -- causal_trajectory # instrumentation records correctly |
| 385 | +cargo test --package ladybug-rs -- gestalt # bundling detection works |
| 386 | +cargo bench -- spo_distance # <15 cycles per comparison |
| 387 | +``` |
| 388 | + |
| 389 | +--- |
| 390 | + |
| 391 | +## THE PUNCHLINE |
| 392 | + |
| 393 | +Cosine consumes. SPO harvests. Every search grows the knowledge graph, refines truth values, detects distributional shifts, identifies causal structure, and prepares warm-starts. The resonator's subconscious IS the causal structure of the world it models. And it runs on popcount at 238× less cost than the operation it replaces. |
| 394 | + |
| 395 | +The architecture is confirmed. The types exist. Make it compile. |
0 commit comments