Skip to content

Commit edf0a9d

Browse files
committed
audit+contract: purge baton/emission; zero-copy SoA model correction
Every SoA envelope is zero-copy from creation to Lance tombstone. There is no baton, no CollapseGateEmission, no inter-mailbox handoff type. audit doc: - Phase 7 rewritten: LE contract is an in-place backing-store descriptor, not a transmitted packet. Baton/CollapseGateEmission references removed. The {to,from}_le_bytes on CE64/EpisodicEdges64 are Lance I/O seams, not cross-mailbox serialization. - Phase 1: emit()/CollapseGateEmission flagged as code artifact to remove. - Correction 9 added: remove MailboxSoA::emit() and CollapseGateEmission. - Geometry verdict: last_emission_cycle → rename to last_active_cycle. - Bottom line updated: zero-copy lifecycle replaces "execution split". - Phase 7 follow-up: crewai/n8n replaced with OGAR classes/ractor actors as the contract's non-HPC consumers. "packet" language removed. soa_envelope.rs: - Module doc: "in-place backing store" replaces "packet/snapshot". "zero-copy from creation to Lance tombstone" stated explicitly. - SoaEnvelope trait doc: "in-place backing store" not "packet". - Layering table: "row stride" not "row packet". - crewai/n8n removed from consumer description. - PacketSizeMismatch clarified: backing store size mismatch. https://claude.ai/code/session_0147hSzjmWZDuy2MSQNrhEK5
1 parent a676121 commit edf0a9d

2 files changed

Lines changed: 68 additions & 49 deletions

File tree

crates/lance-graph-contract/src/soa_envelope.rs

Lines changed: 21 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -10,28 +10,29 @@
1010
//! decodes — has no contract describing how those columns *assemble* into one
1111
//! row-strided packet. The parts know the LE contract; the envelope did not.
1212
//!
13-
//! [`SoaEnvelope`] is that missing contract. It makes one SoA snapshot a
14-
//! **self-describing little-endian packet per cycle**: a stable column
15-
//! ordering, a fixed row byte stride, a `cycle` version stamp, and a
13+
//! [`SoaEnvelope`] is that missing contract. It makes the in-place SoA
14+
//! backing store **self-describing at each cycle**: a stable column ordering,
15+
//! a fixed row byte stride, a `cycle` version stamp, and a
1616
//! [`ENVELOPE_LAYOUT_VERSION`]. With it, a Lance version IS a coherent LE
17-
//! packet at cycle N — not a loose collection of independently-correct
18-
//! columns.
17+
//! in-place layout at cycle N — not a loose collection of independently-
18+
//! correct columns. Nothing is serialized or transmitted; the backing bytes
19+
//! are resident in-place, zero-copy from creation to Lance tombstone.
1920
//!
2021
//! # Layering (read before adding an ndarray dependency here)
2122
//!
2223
//! This module is **zero-dep, byte-geometry only**. It describes *where*
23-
//! columns sit in a row packet and *what* LE element each holds — as data
24-
//! ([`ColumnDescriptor`]), never as ndarray generic bounds. That keeps
25-
//! `lance-graph-contract` featherweight for its non-HPC consumers
26-
//! (`crewai-rust`, `n8n-rs`), and it keeps ndarray usable standalone by any
27-
//! pure-SIMD consumer.
24+
//! columns sit in the backing store's row stride and *what* LE element each
25+
//! holds — as data ([`ColumnDescriptor`]), never as ndarray generic bounds.
26+
//! That keeps `lance-graph-contract` featherweight for its non-HPC consumers
27+
//! (OGAR classes, ractor actors), and it keeps ndarray usable standalone by
28+
//! any pure-SIMD consumer.
2829
//!
2930
//! The split is deliberate and complementary, not duplicated:
3031
//!
3132
//! | Level | Home | Answers |
3233
//! |-------|------|---------|
3334
//! | Column LE contract | `ndarray::simd::MultiLaneColumn` | "how do I sweep one typed column" |
34-
//! | Envelope LE contract | this module | "where do columns sit in the row packet, what cycle is this" |
35+
//! | Envelope LE contract | this module | "where do columns sit in the row stride, what cycle is this" |
3536
//! | Composition | `lance-graph` (always has both deps) | carve envelope columns → wrap each in `MultiLaneColumn` |
3637
//!
3738
//! ndarray never learns the envelope exists; this crate never learns ndarray
@@ -75,7 +76,7 @@ impl ColumnKind {
7576
}
7677
}
7778

78-
/// One column's placement within a single row packet.
79+
/// One column's placement within a single row of the backing store.
7980
///
8081
/// `Copy` and `repr(C)` so a descriptor table is itself a stable LE artifact.
8182
/// `name_id` is a stable column ordinal (an enum discriminant on the consumer
@@ -117,17 +118,20 @@ pub enum EnvelopeError {
117118
StrideMismatch { declared: usize, summed: usize },
118119
/// Two columns overlap, or a gap/ordering violation was found.
119120
ColumnOverlap { col_a: u16, col_b: u16 },
120-
/// `as_le_bytes().len()` is not `row_stride * n_rows`.
121+
/// `as_le_bytes().len()` is not `row_stride * n_rows` (backing store size mismatch).
121122
PacketSizeMismatch { expected: usize, found: usize },
122123
/// A requested row or column index is out of bounds.
123124
OutOfBounds,
124125
}
125126

126-
/// A self-describing little-endian SoA packet for one cycle.
127+
/// The little-endian geometry contract for one SoA envelope cycle.
127128
///
128-
/// Implemented by the owner of the backing store (e.g. the mailbox SoA). The
129-
/// envelope is read-only here; mutation lives on the owner type, never on this
130-
/// view (mirrors `MailboxSoaView` vs `MailboxSoaOwner`).
129+
/// Implemented by the owner of the in-place backing store (e.g. the mailbox
130+
/// SoA). The envelope is zero-copy from creation to Lance tombstone — nothing
131+
/// is serialized or transmitted; this trait describes *where columns sit* in
132+
/// the already-resident backing bytes and *what cycle stamp* the store carries.
133+
/// The read-only view here mirrors `MailboxSoaView` vs `MailboxSoaOwner`:
134+
/// mutation lives on the owner type, never on this trait.
131135
pub trait SoaEnvelope {
132136
/// Layout version this implementor's geometry conforms to.
133137
const LAYOUT_VERSION: u8 = ENVELOPE_LAYOUT_VERSION;

docs/probes/particle-soa-envelope-audit.md

Lines changed: 47 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -40,8 +40,10 @@ biggest structural finding.**
4040
`plasticity_counter [u8;N]`, `last_emission_cycle [u32;N]`,
4141
`edges [CausalEdge64;N]`, `qualia [QualiaI4_16D;N]`, `meta [MetaWord;N]`,
4242
`entity_type [u16;N]`; scalars `mailbox_id`, `current_cycle`, `w_slot`,
43-
`threshold`. It emits a baton via `emit(MailboxId) -> CollapseGateEmission`.
44-
**This is the particle envelope the intended model describes.**
43+
`threshold`. **This is the particle envelope the intended model describes.**
44+
⚠ The source contains `emit(MailboxId) -> CollapseGateEmission` — this method
45+
contradicts the zero-copy model (creation to Lance tombstone, no emission,
46+
no baton). It is a code artifact to be removed, not the intended design.
4547
- `BindSpace` columns (`bindspace.rs`): `FingerprintColumns { content [u64×256],
4648
cycle [f32×16_384], topic, angle, sigma }`, `EdgeColumn`, `QualiaI4Column`
4749
(+ deprecated `QualiaColumn` f32×18), `MetaColumn`, `temporal`, `expert`,
@@ -294,20 +296,22 @@ SPO decomposition is explicit as three palette indices + the
294296

295297
### Phase 7 — Little-endian contract audit
296298

297-
**There IS a single canonical LE baton contract — but it is fragmented by the
298-
two `CausalEdge64` types and by a `&[u64]` reinterpret seam in the SoA view.**
299+
**The SoA envelope is zero-copy in-place (creation → Lance tombstone). There is
300+
no baton, no emission, no serialization. The LE contract describes where columns
301+
sit in the in-place backing store, not a transmitted packet. Two fragmentation
302+
risks remain.**
299303

300-
- Canonical baton: `CollapseGateEmission` = `(u16 target, CausalEdge64)`, wire
301-
cost `13 + 10·baton_count` bytes (per CLAUDE.md E-BATON-1). `CausalEdge64` and
302-
`EpisodicEdges64` both expose `to_le_bytes / from_le_bytes / write_le / read_le`
303-
— a shared LE convention. (Confirmed — one envelope contract intent.)
304+
- The SoA is owned in-place by the mailbox. A Lance version IS the backing store
305+
at cycle N — not a serialized snapshot, not a transmitted packet.
306+
`CausalEdge64` and `EpisodicEdges64` both expose `to_le_bytes / from_le_bytes`
307+
for Lance's columnar write path (Lance reads/writes LE bytes from/into the store).
308+
These are Lance I/O seams, not cross-mailbox serialization. (Confirmed — correct design.)
304309
- p64-bridge (`p64-bridge/src/lib.rs`): `edges_to_layered_rows(&[CausalEdge64])
305310
-> [[u64;64];8]`, `edge_to_block(&CausalEdge64) -> (usize,usize)`. **This is a
306311
projection / derivation, not a layout-preserving transport.** It reads SPO +
307312
mask bits and *computes* palette addresses; it does not round-trip the edge.
308-
So p64 does **not** "preserve layout" — by design it derives a different
309-
geometry (palette planes) from the edge. (Inferred — acceptable, but it is a
310-
one-way lens, not a serializer.)
313+
p64 derives a different geometry (palette planes) from the edge — one-way lens,
314+
not a serializer. (Inferred — acceptable by design.)
311315
- SoA view reinterpret seam: `MailboxSoaView::edges_raw() -> &[u64]` (NOT
312316
`&[CausalEdge64]`) — the contract crate stays zero-dep on `causal-edge` by
313317
handing back raw `u64` that callers reconstruct via `CausalEdge64(raw)`. This
@@ -319,23 +323,25 @@ two `CausalEdge64` types and by a `&[u64]` reinterpret seam in the SoA view.**
319323
**Contract map:**
320324

321325
```
322-
ENVELOPE LE CONTRACT (canonical)
323-
├─ Baton: CollapseGateEmission (u16 target, CausalEdge64) wire = 13 + 10·n
324-
├─ CausalEdge64::{to,from}_le_bytes (causal-edge crate, v1/v2 feature-gated)
326+
ENVELOPE LE CONTRACT (in-place backing store)
327+
├─ CausalEdge64::{to,from}_le_bytes (causal-edge crate, v1/v2 feature-gated — Lance I/O seam)
325328
├─ EpisodicEdges64::{to,from}_le_bytes (matches CE64 convention)
326329
327330
FRAGMENTATION RISKS
328331
├─ ⚠ TWO CausalEdge64 layouts (causal-edge SPO-palette vs thinking-engine 8-channel)
329332
│ bridged only by layered.rs::to_spo()/from_spo() — name collision, lossy
330-
├─ ⚠ soa_view::edges_raw() -> &[u64] (reinterpret seam; layout agreement is implicit)
331-
└─ ⚠ p64-bridge = one-way projection CE64 → [[u64;64];8] (derives, does NOT preserve/round-trip)
333+
└─ ⚠ soa_view::edges_raw() -> &[u64] (reinterpret seam; v1/v2 layout agreement is implicit)
334+
335+
NOTE: CollapseGateEmission / "baton" DO NOT exist in the correct design.
336+
MailboxSoA::emit() in source is a code artifact to be removed. Every SoA is
337+
zero-copy from creation to tombstone; there is no cross-mailbox handoff type.
332338
```
333339

334340
| Question | Answer |
335341
|---|---|
336-
| Single canonical LE contract? | **Yes** for the baton + CE64/EpisodicEdges64 byte methods (Confirmed). |
342+
| Single canonical LE contract? | **Yes** CE64/EpisodicEdges64 byte methods are the Lance I/O seam. (Confirmed.) |
337343
| Hidden conversion? | **Yes**`edges_raw() -> &[u64]` reinterpret; v1/v2 layout agreement is implicit (Contradiction risk). |
338-
| Serialization tax? | Low on the hot path (baton is 8-byte words). p64 projection recomputes per dispatch — derive cost, not serialize cost. |
344+
| Serialization tax? | **None** the backing store is in-place. Lance writes LE columns directly. p64 derives palette geometry, does not serialize. |
339345
| Contract fragmentation? | **Yes** — two `CausalEdge64` types is the principal fragmentation. |
340346

341347
#### Phase 7 follow-up — the envelope must know the LE contract, not just the columns (RESOLVED 2026-06-06)
@@ -359,22 +365,23 @@ with a cycle stamp. The parts knew the LE contract; the envelope did not.
359365
(`ColumnDescriptor[]` — ordering + offset + LE `ColumnKind` + elems/row),
360366
`row_stride()`, `cycle(): u32`, `LAYOUT_VERSION`, `as_le_bytes()`, plus
361367
`row_le` / `column_le` zero-copy views and a `verify_layout()` gate (catches
362-
stride mismatch, column overlap, packet-size tear, and version skew at the
363-
Lance read boundary — closing the `edges_raw() -> &[u64]` implicit-agreement
364-
hazard).
368+
stride mismatch, column overlap, backing-store size mismatch, and version
369+
skew at the Lance read boundary — closing the `edges_raw() -> &[u64]`
370+
implicit-agreement hazard). The envelope describes the in-place backing
371+
store; nothing is packaged or transmitted.
365372
- **Composition = `lance-graph`** (the one crate that always has both deps):
366373
carve each envelope column per its descriptor, wrap in `MultiLaneColumn`.
367374

368375
**Why NOT a shared `simd-soa-contract` crate, and why NOT pull ndarray into
369-
the contract:** `lance-graph-contract` is consumed by `crewai-rust` and
370-
`n8n-rs` precisely because it is zero-dep. ndarray is the heavy HPC foundation
376+
the contract:** `lance-graph-contract` is consumed by OGAR classes and ractor
377+
actors precisely because it is zero-dep. ndarray is the heavy HPC foundation
371378
(BLAS L1/L2/L3, MKL/OpenBLAS FFI). Pulling ndarray into the contract would
372379
force that build onto every contract consumer AND force a pure-SIMD ndarray
373380
consumer to transitively pull a graph contract crate. The two-level split
374381
above keeps **both** crates clean: ndarray standalone for SIMD-only consumers,
375-
contract featherweight for crewai/n8n. The levels are complementary
376-
(column = "how to sweep one typed column"; envelope = "where columns sit, what
377-
cycle"), never restated, and neither crate depends on the other.
382+
contract featherweight for class/actor consumers. The levels are complementary
383+
(column = "how to sweep one typed column"; envelope = "where columns sit in the
384+
backing store, what cycle"), never restated, and neither crate depends on the other.
378385

379386
**Iron rule that falls out:** *ndarray owns the column contract; lance-graph
380387
owns the envelope contract; neither restates the other; lance-graph binds them.*
@@ -481,7 +488,14 @@ execution geometry explicit.
481488
7. **Retire the deprecated qualia/cycle columns** once (1) lands —
482489
`QualiaColumn` f32×18 and the `Vsa16kF32` cycle plane are pure footprint
483490
(65.5 KB/row) on the legacy envelope. (Resolves redundancy 3, 4.)
484-
8. **Demote the `ndarray-hpc` fallback wording in CLAUDE.md.** In practice
491+
9. **Remove `MailboxSoA::emit()` and `CollapseGateEmission`.** The zero-copy
492+
model (creation → Lance tombstone, no emission, no baton) means `emit()` and
493+
the `CollapseGateEmission(u16 target, CausalEdge64)` type are code artifacts
494+
from a superseded design. The KanbanColumn/`VersionScheduler`/ractor
495+
orchestration is the only secondary path; there is no inter-mailbox handoff
496+
type. Remove `emit()` from `MailboxSoA`, remove or gate `CollapseGateEmission`
497+
behind `#[deprecated]`, and update `ShaderDriver` accordingly.
498+
10. **Demote the `ndarray-hpc` fallback wording in CLAUDE.md.** In practice
485499
**no shipped consumer uses lance-graph without ndarray** — every consumer
486500
uses both. The `ndarray-hpc` feature / `blasgraph/ndarray_bridge.rs`
487501
fallback is a **CI-compile-check only** path, not a real deployment mode.
@@ -502,7 +516,7 @@ execution geometry explicit.
502516
|---|---|---|
503517
| `MailboxSoA.mailbox_id` | **KEEP** | Envelope identity, correct. |
504518
| `MailboxSoA.entity_type` | **KEEP (fix resolver)** | Correct as class pointer; resolver should match its O(1) doc. |
505-
| `MailboxSoA.energy / plasticity / last_emission_cycle` | **KEEP** | Local pragmatics, correctly owned. |
519+
| `MailboxSoA.energy / plasticity / last_emission_cycle` | **KEEP (rename)** | Local pragmatics, correctly owned. `last_emission_cycle` is a same-cycle idempotency guard; rename to `last_active_cycle` to remove the emission framing. |
506520
| `MailboxSoA.edges (CausalEdge64)` | **KEEP** | Payload, correctly owned by the mailbox. |
507521
| `MailboxSoA.qualia (QualiaI4_16D)` | **KEEP** | Canonical local pragmatic vector. |
508522
| `MailboxSoA.meta (MetaWord)` | **KEEP** | Thinking-style/awareness bits belong here, not on the edge. |
@@ -528,11 +542,12 @@ execution geometry explicit.
528542

529543
**Does the struct geometry measure what it claims?**
530544

531-
- **Payload, references, versioning, execution split: YES.** `CausalEdge64` is
532-
a clean payload, references are explicit `Copy` handles with no ownership
545+
- **Payload, references, versioning, zero-copy lifecycle: YES.** `CausalEdge64`
546+
is a clean payload, references are explicit `Copy` handles with no ownership
533547
cycles, Lance versioning gives self-through-time without row-level history
534-
duplication, and the read-only-scheduler / single-owner-mutator split is
535-
correct.
548+
duplication, the read-only-scheduler / single-owner-mutator split is correct,
549+
and the SoA envelope is zero-copy in-place from creation to Lance tombstone
550+
(no emission, no baton, no serialization).
536551
- **Identity, inheritance, single-envelope: NOT YET.** The intended
537552
`OGAR::classes::from(address)` does not exist (forward-only in OGAR; a linear
538553
scan in lance-graph); two SoA envelopes co-exist with a live deprecated f32

0 commit comments

Comments
 (0)