Skip to content

Commit e2ad431

Browse files
authored
Merge pull request #643 from AdaWorldAPI/claude/happy-hamilton-0azlw4
contract: Tesseract recoder + recognizer-leaf boards + network→V3-SoA sink (byte-parity)
2 parents 0ac5eb9 + 4e14db0 commit e2ad431

8 files changed

Lines changed: 1370 additions & 1 deletion

File tree

.claude/board/EPIPHANIES.md

Lines changed: 64 additions & 0 deletions
Large diffs are not rendered by default.

.claude/board/LATEST_STATE.md

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,16 @@
1010
1111
---
1212

13+
## 2026-07-04 — branch `claude/happy-hamilton-0azlw4``contract::network` — the Tesseract `Network` layer graph sunk onto V3 SoA via ruff→OGAR (byte-parity vs libtesseract)
14+
15+
**NEW** `lance_graph_contract::network`: `NetworkType` (27 layer types, ordinal == on-wire `kTypeNames` discriminant) + `NetworkHeader` (`from_le_bytes` = the base header `Network::CreateFromFile` reads before subclass dispatch: `i8 tag | u32+str type_name | i8 training | i8 needs_backprop | i32 flags | i32 ni | i32 no | i32 num_weights | u32+str name`) + `to_facet()` (the V3 SoA sink) + `NetworkType::classid()` (the `invoke_network` dispatch seed). Executes the operator directive *"6x8:8, 16 B tenant = classid + 12 B, ruff>OGAR sink-in"*: (1) the `ruff_cpp_spo` `harvest_network` example (committed to ruff) walks the 11 network headers via libclang → the `has_function`/`virtually_overrides` SPO manifest (62 classes, 5060 triples) = the `classid → ClassView` method-resolution table, NOT a hand-rolled enum; (2) each node sinks onto `crate::facet::FacetCascade` (16 B = `classid(4) | 6×(8:8)`, read `CascadeShape::G6D2`): tier0=ni, tier1=no, tier2=flags, tiers3-4=num_weights u32, tier5=lifecycle; `facet_classid = compose_classid(network_layer=0x0804, ntype)` canon-high. Byte-parity **GREEN** on real `/tmp/eng.lstm`: Rust parse == libtesseract `Network::CreateFromFile` — `Series ni=36 no=111 num_weights=385807 name=Series` — oracle `spec()` == the model spec string (known-answer self-check, 5.5.0-hdr/5.3.4-lib ABI skew guarded). Example `network_dump.rs`; +5 contract tests; clippy `-D warnings` + fmt clean (scoped `-p lance-graph-contract`). ONE `network_layer`=0x0804 OCR-domain mint added (subclasses in classid custom-low, not 27 slots). Deferred: per-subclass payload + tree recursion, the `invoke_network` keystone, the recognizer COMPUTE leaves. Refs: EPIPHANIES `E-OCR-NETWORK-SINK-1`; plan `tesseract-rs/.claude/plans/network-ruff-ogar-sink-v1.md`. Not yet a PR.
16+
17+
## 2026-07-04 — branch `claude/happy-hamilton-0azlw4``contract::unicharcompress` — the Tesseract recoder load side (byte-parity vs libtesseract)
18+
19+
**NEW** `lance_graph_contract::unicharcompress`: `UnicharCompress` (the LSTM recoder's code↔id table) + `RecodedCharId` + `RecoderError`, load side only (`from_le_bytes` / `load_from_file` = C++ `DeSerialize`; `encode` / `decode` / `code_range`; `dump_encode` / `dump_decode` parity surfaces). The FIRST binary-format leaf (`TFile` little-endian: `u32 count` + per-entry `[i8 self_normalized][i32 length][i32×length code]`). Byte-parity **GREEN** on real `/tmp/eng.lstm-recoder` — encode 112/112 + decode 112/112 + code_range=111 — via the committed `examples/recoder_dump.rs`, diffed vs a libtesseract 5.3.4 oracle (the 5.5.0-header ABI skew self-validated by the `Encode∘Decode` round-trip + `enc_size=112`). +10 contract tests; `-p lance-graph-contract` clippy `-D warnings` + fmt clean. Consumed by `tesseract-core::{Recoder, recoded_to_text}` (codes→decode→ids→`ids_to_text`; +1 boundary test, 8/8). Resolves the `recoder`=0x0802 concept (OGAR #148 mint, mirrored in the "0x08XX OCR rows" line below) to its content-store module. The recoder keystone (`invoke_recoder`) is UNBLOCKED but deferred (dispatch already proven generically by E-CPP-KEYSTONE-1). Refs: EPIPHANIES `E-CPP-PARITY-7`. Not yet a PR.
20+
21+
---
22+
1323
## 2026-06-23 — IN PR (`claude/medcare-bridge-lance-graph-wmx76z`) — ActionHandler⟷RBAC⟷orchestration spine
1424

1525
`contract::rbac`: `ScopeSpec` (axis-3 Copy token) + `ClassRbac` §4 default methods (`roles_reaching`/`row_scope`/`field_mask`; backward-compat, probe green). `contract::class_view::FieldMask::union`. `contract::action::ActionInvocation::commit_via<R: ClassRbac>` (no-admin-bypass convergence of the inline gate). `lance-graph-rbac::{authorize_scoped, ScopedDecision}` (§5 two-stage). `lance-graph-ogar::{OgarRbac<S: GrantSource>, GrantSource}` (Q5 local newtype, §6 evaporation seam). rs-graph-llm: `graph-flow-kanban::{run_cycle, CycleOutcome}` + `graph-flow-action::dispatch_via`. Plan: integration-actionhandler-rbac-orchestration-v1.
@@ -693,5 +703,5 @@ PR sequence: #360 → #361 → post-#360 substrate-sweep (this PR).
693703

694704
- **`codegen_spine::RouteBucketTyped`** (NEW; C6 merged verbatim from op-nexgen's vendored diff, codex-reviewed on nexgen PR #8). Kind-generic sibling of `RouteBucket` (`type Kind: Copy + Eq`) + `?Sized` blanket bridge (`impl<T: RouteBucket + ?Sized> RouteBucketTyped for T { type Kind = OdooMethodKind; }`) so non-Odoo codegen targets bring their own kind enum additively. Coherence rule: a type needing a different Kind skips the legacy trait. 12/12 module tests incl. dyn-object coverage.
695705
- **`emission_scan`** (NEW; op-nexgen L2). Zero-dep typed-DDL adoption counter, `classid_scan`'s design-language sibling: `TypedForm {Typed, AnyTyped, RecordLink, Stub}` (#[non_exhaustive]) + tokenizer `classify_ddl_type` (precedence Stub > RecordLink > AnyTyped > Typed; word-boundary tokens so `many`/`recording` never false-match) + `EmissionCounts` fold with `typed_ratio()` (f64, mirrors `adoption_pct`). 15 tests. Module doc NAMES the contract scan-family pattern (Form enum + classify_* + fold-to-counts): the next governance counter mirrors it.
696-
- **`ogar_codebook` 0x08XX OCR rows**`unicharset` (0x0801) / `recoder` (0x0802) / `charset` (0x0803) mirroring OGAR #148's mint (container kinds only; content never becomes concepts — Osint zero-rows precedent). Drift-guard test extended. CODEBOOK now 68 entries.
706+
- **`ogar_codebook` 0x08XX OCR rows**`unicharset` (0x0801) / `recoder` (0x0802) / `charset` (0x0803) / `network_layer` (0x0804) mirroring OGAR #148's mint (container kinds only; content never becomes concepts — Osint zero-rows precedent). `network_layer` = the KIND "a Tesseract recognizer network layer"; the 27 subclasses live in the classid custom-low half (`NetworkType` ordinal), NOT 27 slots. Drift-guard test extended. CODEBOOK now 69 entries.
697707
- **Rulings + intake record:** EPIPHANIES E-V3-XSESSION-INTAKE-1(+RULINGS), E-V3-GRAPHRAG-INV-1; handover `.claude/handovers/2026-07-02-cross-session-wishlist-intake.md`; plan Addendum-10/11 (per-consumer classid ownership + tripwires ratified; R-1 naming phantom closed — `domain:appid:classview`; R-2 closed — 512-byte row frozen, edges via strided view; L3 new-Arrow-schema design killed; five post-fuse workstreams enumerated). Knowledge: `graphrag-rs-inventory.md`.
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
//! Dump the base `Network` header at the front of a serialized recognizer
2+
//! component (`eng.lstm`) — the Rust side of the network base-header byte-parity
3+
//! leaf, sibling to `recoder_dump`. Also prints the [`FacetCascade`] the node
4+
//! sinks onto (the ruff→OGAR harvest → V3 SoA target).
5+
//!
6+
//! ```sh
7+
//! # Extract the lstm component (starts with the network, lstmrecognizer.cpp:135):
8+
//! combine_tessdata -u $(dpkg -L tesseract-ocr-eng | grep eng.traineddata) /tmp/eng.
9+
//! # C++ oracle (network_spec_oracle.cpp): links libtesseract, calls the REAL
10+
//! # Network::CreateFromFile on the same bytes and dumps the loaded top node's
11+
//! # type / ni / no / num_weights / name + spec() (the known-answer self-check).
12+
//! # ./network_spec_oracle /tmp/eng.lstm > /tmp/oracle_network.txt
13+
//! # Rust side (parses only the base header — the shared prefix of every layer):
14+
//! cargo run -p lance-graph-contract --example network_dump -- /tmp/eng.lstm > /tmp/rust_network.txt
15+
//! # The "header:" line is byte-identical between the two => the base header
16+
//! # parse is byte-parity green.
17+
//! ```
18+
19+
#![allow(
20+
clippy::print_stdout,
21+
reason = "a dump CLI example writes to stdout by design"
22+
)]
23+
24+
use std::process::ExitCode;
25+
26+
use lance_graph_contract::network::NetworkHeader;
27+
28+
fn main() -> ExitCode {
29+
let Some(path) = std::env::args().nth(1) else {
30+
eprintln!("usage: network_dump <path/to/eng.lstm>");
31+
return ExitCode::FAILURE;
32+
};
33+
let bytes = match std::fs::read(&path) {
34+
Ok(b) => b,
35+
Err(err) => {
36+
eprintln!("error reading {path}: {err}");
37+
return ExitCode::FAILURE;
38+
}
39+
};
40+
match NetworkHeader::from_le_bytes(&bytes) {
41+
Ok((header, consumed)) => {
42+
// The byte-parity line (diffed against the oracle's loaded top node).
43+
println!("header: {}", header.dump());
44+
// The V3 SoA sink: the 16-byte FacetCascade (classid + 6×8:8), hex.
45+
let f = header.to_facet();
46+
let hex: String = f.to_bytes().iter().map(|b| format!("{b:02x}")).collect();
47+
println!("facet: classid={:#010x} bytes={hex}", f.facet_classid);
48+
println!("consumed: {consumed} bytes (base header; subclass payload follows)");
49+
ExitCode::SUCCESS
50+
}
51+
Err(err) => {
52+
eprintln!("error parsing header: {err:?}");
53+
ExitCode::FAILURE
54+
}
55+
}
56+
}
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
//! Dump a `.lstm-recoder`'s encoder table (`encode`) or decode round-trip
2+
//! (`decode`) — the Rust side of the recoder byte-parity leaf, sibling to
3+
//! `unicharset_dump`.
4+
//!
5+
//! ```sh
6+
//! # on a box with libtesseract + libleptonica installed:
7+
//! combine_tessdata -u $(dpkg -L tesseract-ocr-eng | grep eng.traineddata) /tmp/eng.
8+
//! # C++ oracle (recoder_oracle.cpp): loads the SAME component via TFile and dumps
9+
//! # EncodeUnichar / DecodeUnichar / code_range. It also prints, per id, the
10+
//! # UNICHARSET bijection + an Encode.Decode round-trip so the NEW UnicharCompress
11+
//! # object layout self-validates against the 5.5.0-header / 5.3.4-lib ABI skew.
12+
//! # ./recoder_oracle /tmp/eng.lstm-unicharset /tmp/eng.lstm-recoder encode > /tmp/oracle_recoder_encode.tsv
13+
//! # ./recoder_oracle /tmp/eng.lstm-unicharset /tmp/eng.lstm-recoder decode > /tmp/oracle_recoder_decode.tsv
14+
//! # Rust side:
15+
//! cargo run -p lance-graph-contract --example recoder_dump -- /tmp/eng.lstm-recoder encode > /tmp/rust_recoder_encode.tsv
16+
//! cargo run -p lance-graph-contract --example recoder_dump -- /tmp/eng.lstm-recoder decode > /tmp/rust_recoder_decode.tsv
17+
//! diff /tmp/oracle_recoder_encode.tsv /tmp/rust_recoder_encode.tsv \
18+
//! && diff /tmp/oracle_recoder_decode.tsv /tmp/rust_recoder_decode.tsv
19+
//! # both byte-identical => the recoder load-side is byte-parity green
20+
//! ```
21+
22+
#![allow(
23+
clippy::print_stdout,
24+
reason = "a dump CLI example writes to stdout by design"
25+
)]
26+
27+
use std::path::Path;
28+
use std::process::ExitCode;
29+
30+
use lance_graph_contract::unicharcompress::UnicharCompress;
31+
32+
fn main() -> ExitCode {
33+
let Some(path) = std::env::args().nth(1) else {
34+
eprintln!("usage: recoder_dump <path/to/eng.lstm-recoder> [encode|decode]");
35+
return ExitCode::FAILURE;
36+
};
37+
let mode = std::env::args().nth(2).unwrap_or_default();
38+
match UnicharCompress::load_from_file(Path::new(&path)) {
39+
Ok(recoder) => {
40+
match mode.as_str() {
41+
"decode" => print!("{}", recoder.dump_decode()),
42+
_ => print!("{}", recoder.dump_encode()),
43+
}
44+
ExitCode::SUCCESS
45+
}
46+
Err(err) => {
47+
eprintln!("error: {err}");
48+
ExitCode::FAILURE
49+
}
50+
}
51+
}

crates/lance-graph-contract/src/lib.rs

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,8 @@ pub mod manifest;
9696
pub mod mul;
9797
pub mod nan_projection;
9898
pub mod nars;
99+
/// LSTM `Network` layer-graph structure — base-header parse + `FacetCascade` sink.
100+
pub mod network;
99101
pub mod ocr;
100102
/// D-OVC-1 — OGAR concept codebook (`0xDDCC` domain layout), wire-compat mirror.
101103
pub mod ogar_codebook;
@@ -133,6 +135,7 @@ pub mod tax;
133135
pub mod tenant_counter;
134136
pub mod thinking;
135137
pub mod unichar;
138+
pub mod unicharcompress;
136139
pub mod unicharset;
137140
pub mod unicharset_adapter;
138141
pub mod view_angle;

0 commit comments

Comments
 (0)