feat: SPO Merkle hardening — Modules 1-4#170
Merged
Merged
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 67a6b39840
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+1994
to
+1998
| // Parent's root should reflect its children's content. | ||
| // For a simple check: parent and child should both be non-zero. | ||
| // Full XOR-of-children verification requires knowing all children, | ||
| // but for lineage walk, we verify the chain is consistent. | ||
| current = parent_addr; |
There was a problem hiding this comment.
verify_lineage is documented as an integrity check, but this loop never compares parent_root against the current node’s root, so IntegrityResult::Diverged is unreachable and any non-missing lineage is reported as Consistent. In practice, if a node’s clam_merkle is corrupted (or manually mutated) to a conflicting non-zero root, this check will silently pass and downstream code cannot detect tampering.
Useful? React with 👍 / 👎.
Comment on lines
+576
to
+579
| debug_assert!(index < 128); // CONTAINER_WORDS | ||
| let half = index / 64; | ||
| let bit = index % 64; | ||
| if bitmap[half] & (1u64 << bit) == 0 { |
There was a problem hiding this comment.
read_packed_word relies on debug_assert!(index < 128), but in release builds that check is removed and index >= 128 makes half at least 2, causing an out-of-bounds panic at bitmap[half]. Because this is a pub API with a plain usize argument, any unchecked caller input can crash query paths in production instead of returning a safe miss/error.
Useful? React with 👍 / 👎.
AdaWorldAPI
force-pushed
the
claude/spo-merkle-hardening-9rw2C
branch
2 times, most recently
from
7e78244 to
400e0e2
Compare
Module 1: Wire ClamPath+MerkleRoot stamp into write_dn_path and write_dn_node. Each BindNode now carries clam_merkle (u64) packing ClamPath(24 bits) + MerkleRoot(40 bits) via blake3 of fingerprint. Module 2: Add verify_lineage() for integrity verification walking the parent chain. Add clam_merkle(), set_clam_path() methods. Module 3: Add Epoch struct (XOR dirty bitset snapshot) with changed_between() and change_count() for O(128-cycle) diff. Add snapshot_dirty() to BindSpace. Module 4: Add TruthGate (NARS truth filter, ~2 cycles) and SpoHit to graph::spo::store. Gated queries apply truth filtering BEFORE distance computation. Add read_packed_word() for zero-alloc reads. All changes live on BindNode at Addr (Gate 1), use zero-copy borrows (Gate 3), stay within RISC cycle budget (Gate 4). https://claude.ai/code/session_018L7tAcJ9ppReFdcjhYjTcb
P0-1: Replace all HashMaps/HashSets with flat arrays indexed by DN. - leaves/children_xor: Vec<MerkleHash> [65536] — 2MB each, L2-resident - parents: Vec<u16> [65536] — 128KB flat, NO_PARENT sentinel - children: Vec<Vec<u16>> [65536] — O(1) indexed by DN - dirty: DnBitSet (65536-bit, 8KB) replaces HashSet<u64> - occupied: DnBitSet tracks which slots have leaves Total: ~4.3MB flat, O(1) access at 3-5 cycles. Zero HashMap. P0-2: Fix flip_up to propagate XOR delta to root. - Old code stopped at immediate parent — depth>1 trees had stale roots. - New code walks parent chain: start → parent → grandparent → ... → root. - Added test_depth3_grandchild_changes_root verifying root changes on grandchild insert and restores on removal. P0-3: Replace SHA256 with blake3 for leaf_hash. - clam_path.rs already uses blake3; one hash function for integrity. - sha2 dep kept in Cargo.toml for other consumers (membrane, flight, etc). 25/25 merkle tests pass. https://claude.ai/code/session_018L7tAcJ9ppReFdcjhYjTcb
P0-5: Belichtung prefilter now samples from BITMAP POSITIONS (union of both containers' bitmaps) instead of fixed indices [0,17,37,59,79,101,123]. For a SparseContainer with 30 stored words, fixed-index sampling hit mostly zeros — useless estimate. Bitmap-adaptive sampling ensures every sample word contains actual data. P0-6: walk_chain_semiring reuses a single Container buffer instead of calling z.to_dense() (2KB Container::zero() + copy) per inner-loop iteration. Writes sparse data directly into the reusable buffer. 8/8 store tests pass. https://claude.ai/code/session_018L7tAcJ9ppReFdcjhYjTcb
…walk" This reverts commit 9c99562.
This reverts commit ea06761.
Lines 643, 714, 865, 900, 935, 983: all f32 sort/max_by sites now survive NaN inputs instead of panicking. https://claude.ai/code/session_018L7tAcJ9ppReFdcjhYjTcb
From<GraphError>: maps ParseError→ParseError, PlanError→PlanError,
ConfigError→PlanError, ExecutionError→ExecutionError,
UnsupportedFeature→UnsupportedFeature, InvalidPattern→InvalidPattern.
From<SpoError>: maps all variants via Display → SpoError { message }.
Verified: .parse().unwrap() in cypher.rs already uses .map_err().
https://claude.ai/code/session_018L7tAcJ9ppReFdcjhYjTcb
…collision Item 4: Remove #[macro_export] from lance_plan_err!, lance_config_err!, lance_exec_err! — these are bouncer-internal macros, not crate-level. Only used in lance_parser/error.rs tests. Item 5: Replace `pub use ast::*` glob re-export with explicit re-exports excluding CypherQuery. lance_parser::ast::CypherQuery and cypher.rs::CypherQuery are different types serving different roles — the glob made them collide at the query module level. Now the bouncer's CypherQuery stays behind lance_parser::ast:: and the old cypher.rs CypherQuery owns the unqualified name. 105/105 lance_parser tests pass. 6/6 error tests pass. https://claude.ai/code/session_018L7tAcJ9ppReFdcjhYjTcb
AdaWorldAPI
force-pushed
the
claude/spo-merkle-hardening-9rw2C
branch
from
0da809a to
fcaf4cd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Module 1: Wire ClamPath+MerkleRoot stamp into write_dn_path and write_dn_node. Each BindNode now carries clam_merkle (u64) packing ClamPath(24 bits) + MerkleRoot(40 bits) via blake3 of fingerprint.
Module 2: Add verify_lineage() for integrity verification walking the parent chain. Add clam_merkle(), set_clam_path() methods.
Module 3: Add Epoch struct (XOR dirty bitset snapshot) with changed_between() and change_count() for O(128-cycle) diff. Add snapshot_dirty() to BindSpace.
Module 4: Add TruthGate (NARS truth filter, ~2 cycles) and SpoHit to graph::spo::store. Gated queries apply truth filtering BEFORE distance computation. Add read_packed_word() for zero-alloc reads.
All changes live on BindNode at Addr (Gate 1), use zero-copy borrows (Gate 3), stay within RISC cycle budget (Gate 4).
https://claude.ai/code/session_018L7tAcJ9ppReFdcjhYjTcb