Skip to content

Commit 98d68f8

Browse files
committed
docs(mailbox): THE LE contract is one SoA end-to-end; ontology lazylock-as-is; DTO/p64 vertical audit
Sharpen the migration spec per the ruling that there is ONE little-endian contract, the same SoA layout from cognitive-shader-driver down to lance-graph storage with no boundary re-encode: - plan §2.5: THE LE contract is uniform shader SoA → contract LE types (CausalEdge64/QualiaI4_16D/MetaWord/SoaColumns/entity_type) → lance storage; persisted_row is a pointer to the same SoA row, not a re-encode - plan §4: Ontology is NOT in the SoA — stays AS IS, lazylock (registry.rs:39 LazyLock) + ontology_dictionary Lance cache (lance_cache.rs, TTL-sourced, drop-and-rebuild) - plan §2.6: DTO vertical audit — p64-bridge already conforms (LE types → palette, no re-encode = the template); engine_bridge bind/unbind_busdto is the legacy re-encode seam to collapse (S2); thinking-engine StreamDto/ ResonanceDto/BusDto/ThoughtStruct survive only as ingress adapter + read projections; ResonanceDto.energy IS MailboxSoA.energy - EPIPHANIES: E-MAILBOX-IS-BINDSPACE refinement (end-to-end + ontology + DTO) - TECH_DEBT: TD-RESONANCEDTO-DUP-1 (two ResonanceDto structs, same name) Design/spec only. https://claude.ai/code/session_017GFLBnDy23AWBqvkbHHC41
1 parent bdd103c commit 98d68f8

3 files changed

Lines changed: 119 additions & 15 deletions

File tree

.claude/board/EPIPHANIES.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,9 @@
1212

1313
**Gated staging (plan §6):** S1 add columns behind `mailbox-thoughtspace` feature → S2 move `engine_bridge` surface onto mailbox rows → S3 driver holds a sea-star of mailboxes (kill the 4096 singleton in `serve.rs`) → S4 death→SPO+Lance tombstone-witness → S5 delete `BindSpace`+`cycle` plane. Gated on `D-CE64-MB-1-impl` (par-tile) + `PR-NDARRAY-MIRI-COMPLETE`; S5 blocked on the CLAUDE.md "The Click" / `Vsa16kF32` doctrinal update (OQ-4, already flagged in `surreal/RECONCILIATION`).
1414

15-
**Cross-ref:** `E-BATON-1`, `E-CE64-MB-4`, `E-LADDER-SERVES-MAILBOX` (§6 tombstone-witness), `I-VSA-IDENTITIES`, `I-LEGACY-API-FEATURE-GATED` (feature-gate the v1 accessors during S1–S4), `causaledge64-mailbox-rename-soa-v1` (§5 MailboxSoA), `cognitive-substrate-convergence-v1` (D-CSV-7 shipped accumulator).
15+
**Refinement (same session, 2026-05-27):** the per-mailbox SoA *is* **THE little-endian contract** — singular, and the **same SoA layout runs the whole vertical with no boundary re-encode**: cognitive-shader-driver `MailboxSoA` → `lance-graph-contract` LE types (`CausalEdge64`/`QualiaI4_16D`/`MetaWord`/`SoaColumns`/`entity_type`) → lance-graph storage (Lance columns / tombstone-witness); `ShaderCrystal.persisted_row` is a pointer to the same SoA row, not a serialized copy (plan §2.5). **The Ontology is NOT in the SoA and stays AS IS** — lazylock (`registry.rs:39 LazyLock<NamespaceRegistry>`) + the `ontology_dictionary` Lance **cache** (`lance_cache.rs`, TTL-sourced, drop-and-rebuild; its own header already says "BindSpace … never lands here") (plan §4). **DTO audit (plan §2.6):** `p64-bridge` already conforms (maps `CausalEdge64`/`ThinkingStyle` straight to palette, no re-encode — the template); the legacy re-encode seam is `engine_bridge.rs` `bind_busdto`/`unbind_busdto`/`busdto_to_binary16k` (collapse in S2); `thinking-engine` DTOs survive only as the `StreamDto` ingress adapter + thin read-projections (`BusDto`/`ThoughtStruct`) over the mailbox SoA; `ResonanceDto.energy` *is* `MailboxSoA.energy` (the two `ResonanceDto` defs are `TD-RESONANCEDTO-DUP-1`).
16+
17+
**Cross-ref:** `E-BATON-1`, `E-CE64-MB-4`, `E-LADDER-SERVES-MAILBOX` (§6 tombstone-witness), `I-VSA-IDENTITIES`, `I-LEGACY-API-FEATURE-GATED` (feature-gate the v1 accessors during S1–S4), `E-CONTRACT-NO-SERIALIZE` (compile-time handshake, no membrane serialize — same byte layout to disk), `causaledge64-mailbox-rename-soa-v1` (§5 MailboxSoA), `cognitive-substrate-convergence-v1` (D-CSV-7 shipped accumulator), `TD-RESONANCEDTO-DUP-1`.
1618

1719
---
1820

.claude/board/TECH_DEBT.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,14 @@
1313
---
1414

1515

16+
### TD-RESONANCEDTO-DUP-1 (bindspace-singleton-to-mailbox-soa-v1)
17+
18+
- **Severity:** P3 (name collision; two distinct `ResonanceDto` structs under the same name)
19+
- **Surfaced in:** DTO vertical audit, 2026-05-27, branch `claude/splat3d-cpu-simd-renderer-MAOO0`
20+
- **What:** `crates/thinking-engine/src/dto.rs:59` defines `ResonanceDto { energy: Vec<f32>, top_k, cycle_count, converged }` (the Ψ ripple field); `crates/thinking-engine/src/awareness_dto.rs:21` defines a *different* `ResonanceDto { hdr: HdrResonance, dominant_perspective, gate, dissonance, total_energy, … }` (multi-perspective S/P/O). Same name, different shape, same crate.
21+
- **Owed:** dedup under `bindspace-singleton-to-mailbox-soa-v1` — the `dto.rs` energy field unifies into `MailboxSoA.energy: [f32; N]`; the `awareness_dto.rs` scalars map to SoA `meta`/`edge` columns and `HdrResonance` becomes the S/P/O read over the SoA. Rename/merge so one canonical resonance read remains.
22+
- **Status:** Open
23+
1624
### TD-GHOST-ECHO-DUP-1 (D-PERSONA-1)
1725

1826
- **Severity:** P3 (cosmetic type-dup; no runtime correctness risk — the two enums are not exchanged across a crate boundary today)

.claude/plans/bindspace-singleton-to-mailbox-soa-v1.md

Lines changed: 108 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,84 @@ working set per mailbox; many small mailboxes replace one giant space).
118118

119119
---
120120

121+
## 2.5 — THE little-endian contract: ONE SoA, end-to-end, no boundary re-encode
122+
123+
**Ruling (2026-05-27):** the per-mailbox `MailboxSoA<N>` layout *is* **THE little-endian
124+
contract** — singular, canonical. The **same SoA byte layout runs the whole vertical**, from
125+
the cognitive-shader-driver hot path down to lance-graph storage, with **no re-encode /
126+
translation at any boundary**:
127+
128+
```
129+
cognitive-shader-driver lance-graph-contract lance-graph storage
130+
MailboxSoA<N> ───────────► LE byte contract types ────────► Lance columns /
131+
(hot, owned, ephemeral) CausalEdge64 · QualiaI4_16D · SPO-G + tombstone-witness
132+
MetaWord · SoaColumns<N> · (cold, durable)
133+
entity_type u16
134+
└──────────────── ONE little-endian SoA the whole way ──────────────┘
135+
persisted_row: Option<u32> is the link, not a re-encode
136+
```
137+
138+
Consequences:
139+
140+
- **No Arrow/JSON translation membrane for the thoughtspace.** The columns the mailbox owns
141+
in RAM are the columns Lance stores. A persisted row (`ShaderCrystal.persisted_row`,
142+
`lance-graph-contract/src/cognitive_shader.rs:382`) is a **pointer to the same SoA row**
143+
laid down in Lance, not a serialized copy in a different shape. (Contrast the *ontology*
144+
path, which legitimately translates `MappingRow ↔ RecordBatch` in `lance_cache.rs` — see §4
145+
for why the ontology is a different animal.)
146+
- **The LE byte contract types are the shared vocabulary** (`lance-graph-contract`):
147+
`CausalEdge64`, `QualiaI4_16D`, `MetaWord`, the `SoaColumns<N>` floor (ndarray), `entity_type`
148+
u16. Every layer compiles against these exact types — the contract is a *compile-time*
149+
handshake (cf. `E-CONTRACT-NO-SERIALIZE`), and the *bytes* are identical from compute to disk.
150+
- **Lance is append-only/versioned**, so persisting the SoA row IS the tombstone-witness +
151+
GoBD audit trail by construction (`E-LADDER-SERVES-MAILBOX` §6) — no separate logging layer.
152+
- This is the "restore the contract, never port the carrier" rule made vertical: one i4/u8/
153+
u64 SoA from shader to storage; never re-inflate to `Vsa16kF32` at any tier.
154+
155+
This widens the migration scope from "cognitive-shader-driver only" to **the full vertical**:
156+
shader-driver SoA → contract LE types → lance-graph storage must all be THAT one SoA.
157+
158+
---
159+
160+
## 2.6 — DTO vertical audit: StreamDto / ResonanceDto / BusDto / p64 (what re-encodes vs what already conforms)
161+
162+
The flow DTOs live in `crates/thinking-engine/src/dto.rs` (a workspace member) and stage the
163+
cycle Φ→Ψ→B→Γ. Audited against the "one SoA, no re-encode" ruling (§2.5), **two patterns
164+
coexist** and the migration must collapse the legacy one onto the SoA:
165+
166+
### The GOOD pattern (already conforms — the reference) — `p64-bridge`
167+
168+
`crates/p64-bridge/src/lib.rs` maps the **canonical LE-contract types directly** to p64
169+
palette storage with **no re-encode and no `p64` dependency** (compile-time bridge, joined at
170+
the call site): `CausalEdge64 → edge_to_block` (palette row/col), `ThinkingStyle → layer_mask
171+
+ combine + contra`, `HdrSemiring → combine/contra mode`. This **is** THE SoA reaching
172+
storage — the i4/u8/u64 contract types address the palette directly. *Everything from the
173+
shader SoA to lance-graph storage should look like p64-bridge.* Keep as-is; it is the
174+
storage-ward end of the vertical.
175+
176+
### The LEGACY pattern (re-encodes — collapse it) — `thinking-engine` DTOs ↔ `engine_bridge`
177+
178+
These are heap `Vec`-based representations bridged into `BindSpace` rows by a **translation
179+
membrane** in `crates/cognitive-shader-driver/src/engine_bridge.rs`
180+
(`bind_busdto` / `unbind_busdto:310` / `busdto_to_binary16k:199`). That bind/unbind seam is
181+
exactly the re-encode the ruling forbids for the thoughtspace. Fates:
182+
183+
| DTO (`thinking-engine`) | Shape today | Fate under THE SoA |
184+
|---|---|---|
185+
| `StreamDto` (Φ ingress) | `source`, `codebook_indices: Vec<u16>`, `timestamp` | **Thin ingress adapter only** — a sensor-membrane boundary (like the ontology); codebook indices (identity refs, `I-VSA-IDENTITIES`-clean) hand straight into the mailbox SoA. Does NOT carry a parallel `Vec` through the hot path. |
186+
| `ResonanceDto` (Ψ, `dto.rs`) | `energy: Vec<f32>` (the 4096 ripple field) + `top_k`/`cycle_count`/`converged` | **Already IS `MailboxSoA.energy: [f32; N]`.** The ripple field is the mailbox's energy column — unify, don't keep a separate heap `Vec` DTO. `top_k`/`converged` are derived reads. |
187+
| `ResonanceDto` (`awareness_dto.rs`) | richer multi-perspective: `hdr: HdrResonance`, `dominant_perspective`, `gate`, `dissonance`, `total_energy`, inferred user state | **NAME-COLLISION (two `ResonanceDto`) — dedup.** Scalars (`dissonance`/`total_energy`/`gate`) → SoA `meta`/`edge` columns; `HdrResonance` = the S/P/O 3-perspective read over the SoA. Rename or merge (tech-debt entry filed). |
188+
| `BusDto` (B consequence) | `codebook_index: u16`, `energy: f32`, `top_k`, `cycle_count`, `converged` | **Becomes a view/projection over the SoA row** (read `edges`/`qualia`/`meta` + `persisted_row`), not a bound/unbound separate struct. `unbind_busdto` collapses to a column read; `bind_busdto`/`busdto_to_binary16k` collapse (the SoA row's content-ref IS the binary16k identity). |
189+
| `ThoughtStruct` (Γ collapse) | `bus`, lazy `text`, `sensor_contributions`, `tension_history: Vec<Vec<f32>>`, `style_trajectory` | **Γ projection of the persisted SoA row** (text stays lazy). `tension_history`/`style_trajectory` become witness columns / tombstone fields (`E-LADDER-SERVES-MAILBOX` §6), not parallel `Vec`s. |
190+
191+
**Ruling:** `engine_bridge`'s `bind_busdto`/`unbind_busdto`/`busdto_to_binary16k` are the
192+
re-encode boundary to dissolve (migration step S2). The thinking-engine DTOs survive only as
193+
(a) the `StreamDto` ingress adapter at the sensor membrane and (b) thin *read projections*
194+
(`BusDto`/`ThoughtStruct`) over the mailbox SoA — never as a parallel owned representation the
195+
hot path translates to/from. p64-bridge is the conformance template for the storage-ward half.
196+
197+
---
198+
121199
## 3. Column-by-column migration map
122200

123201
| `BindSpace` column | → Destination | How |
@@ -139,20 +217,36 @@ content_ref ≤ 6). That is the whole point: the mailbox thoughtspace is L1/L2-r
139217

140218
---
141219

142-
## 4. What STAYS shared (does not migrate)
143-
144-
- **`Arc<OntologyRegistry>`** — the OGIT/DOLCE/FIBO calcified ontology. Immutable at read
145-
time, hydrated/mutated on its own owner (`OntologyRegistry::append_mapping`), shared by
146-
`Arc` to every mailbox. This is cold Zone-2, not ephemeral thinking. Mailboxes hold
147-
`&OntologyRegistry` (or a cloned `Arc`), never a copy of its tables.
148-
- **Codebooks / CAM-PQ centroids** — the cleanup codebook the content references resolve
149-
against (`I-VSA-IDENTITIES` Test 3). Shared, immutable, cold.
150-
- **AriGraph SPO-G cold store + Lance dataset** — where the mailbox's witness calcifies on
151-
death (`E-LADDER-SERVES-MAILBOX` §6). Shared persistence, not per-mailbox.
152-
153-
Rule of thumb: **ephemeral per-think state → into the mailbox; calcified/immutable shared
154-
knowledge → stays a shared `Arc`.** The singleton's sin was conflating the two in one
155-
`BindSpace`.
220+
## 4. What STAYS shared (does not migrate) — the Ontology is lazylock-via-cache, AS IS
221+
222+
The **Ontology is explicitly NOT part of THE SoA contract** and does **not** migrate. It stays
223+
exactly as it is today — a lazily-initialised, cache-backed, read-only shared resource:
224+
225+
- **`LazyLock<NamespaceRegistry>`** — the seed registry is `static SEED_NAMESPACE_REGISTRY:
226+
LazyLock<NamespaceRegistry>` (`crates/lance-graph-ontology/src/registry.rs:39`), built once,
227+
read-only thereafter. Mailboxes consult it by reference; never own or copy it.
228+
- **The `ontology_dictionary` Lance cache** (`lance-graph-ontology/src/lance_cache.rs`,
229+
feature `lance-cache`) — a **CACHE of hydrated TTL** keyed by `ttl_root_checksum`; the TTL
230+
files on disk are source-of-truth; on version mismatch it **drop-and-rebuilds** (no
231+
migration ladder). Its own header already states the boundary: *"BindSpace
232+
(FingerprintColumns / QualiaColumn / MetaColumn / EdgeColumn) is the live runtime SoA and is
233+
unrelated — it never lands here."* That invariant holds unchanged under this migration.
234+
- **Why it's a different animal (and why it legitimately re-encodes):** the ontology cache
235+
*does* translate `MappingRow ↔ RecordBatch` — but it is calcified, immutable-at-read,
236+
shared cold knowledge, not ephemeral per-think thoughtspace. The "one SoA, no re-encode"
237+
rule (§2.5) governs the **thoughtspace** vertical; the ontology cache is an orthogonal
238+
TTL-projection cache and is left **as is**.
239+
240+
So: a mailbox holds the ontology as `Arc<OntologyRegistry>` / `&OntologyRegistry` (lazylock +
241+
cache, read-only); the per-row `entity_type: u16` (1-based index into the shared ontology)
242+
travels in THE SoA, but the ontology tables it indexes stay shared and cached. **No ontology
243+
work in this migration — it is `as is`.**
244+
245+
Other shared-immutable state that likewise stays out of the SoA: CAM-PQ codebooks /
246+
centroids (the cleanup codebook content references resolve against, `I-VSA-IDENTITIES`
247+
Test 3); the AriGraph SPO-G cold store. Rule of thumb: **ephemeral per-think state → into the
248+
mailbox SoA; calcified/immutable shared knowledge → stays shared (lazylock/cache), out of the
249+
SoA.**
156250

157251
---
158252

0 commit comments

Comments
 (0)