omc-kernel: content-addressed code DAG keyed by canonical hash

A standalone CLI that maintains a file-system content-addressed
store at ~/.omc/kernel/store/<canonical_hash_hex>.omc. Every OMC
fn has a 64-bit canonical-hash identity (whitespace-stripped,
comments removed, parameter binding normalized), and the store is
keyed on that.

Subcommands:
  ingest DIR    walk DIR, extract every fn, store one entry per
                canonical hash; sidecar JSON carries substrate
                metadata (attractor, dist, fn name, origin file)
  fetch HASH    retrieve stored source by canonical hash (hex)
  stat HASH     pretty-print substrate metadata
  ls            list stored entries + fn names
  sign FILE     emit substrate-signed wire message for OMC source
                (codec format: sampled_tokens + content_hash)
  verify        read wire msg from stdin, recover content via
                store lookup; defends against store tampering by
                recomputing canonical hash and comparing
  demo          end-to-end alpha-rename invariant recovery

End-to-end proof (in this commit):
  $ omc-kernel ingest examples/lib/      → 206 fns, 184 unique
  $ cat > /tmp/x.omc <<<"fn commit(handle) { ... handle ... }"
  $ omc-kernel sign /tmp/x.omc | omc-kernel verify
  → STORE HIT — recovered "fn commit(conn) { ... conn ... }"

Sender used `handle`, store has `conn` — same canonical address.
Alpha-rename + whitespace + comment invariance is intrinsic to
the addressing; no shared key, no certificate authority.

Substrate metadata sidecar (~/.omc/kernel/store/<hash>.json) gives
O(1) `stat` without re-canonicalizing. OMC_KERNEL_ROOT env
overrides the store location.

Honest limits documented in docs/omc_kernel.md:
- No daemon yet (CLI-level only)
- No peer-to-peer discovery (single-node)
- No outgoing-edge graph (Merkle DAG edges are a one-pass
  extension)
- No GC, no on-disk compression

Each is a separable extension that doesn't require redesigning
the address scheme.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: RandomCoder-lab

Cargo.lock

@@ -36,6 +36,8 @@ These are concrete, present-in-the-code features, not aspirations:
 
 - **`omc-grep`: alpha-rename-invariant duplicate finder.** A standalone CLI ([`docs/omc_grep.md`](docs/omc_grep.md)) that walks a tree, extracts every top-level fn, canonicalizes, and clusters by canonical hash. `--body-only` mode strips the fn signature so duplicates with *different names* surface — something text-grep, ast-grep, and tree-sitter queries can't do. On OMC's own examples tree (151 files / 2388 fns): **31.7%** redundancy with name-sensitive hashing, **33.0%** with body-only — surfacing renamed-but-identical fns like `_bucket_discrete` ≡ `endpoint_bucket` ≡ `status_bucket` that share no token in their names.
 
+- **`omc-kernel`: content-addressed code DAG keyed by canonical hash.** A persistence layer for distributed-agent code exchange ([`docs/omc_kernel.md`](docs/omc_kernel.md)). Every fn gets stored at `~/.omc/kernel/store/<hex_hash>.omc`. `omc-kernel sign FILE` emits a substrate-signed wire message; `omc-kernel verify` (stdin) recovers the original canonical form from the store — proven end-to-end with alpha-rename: sender's `fn commit(handle)` recovers as store's `fn commit(conn)`. Code becomes a content-addressed Merkle DAG over substrate addresses; version it the way IPFS versions files, except the addressing is semantic.
+
 - **Substrate-keyed code codec + compressed substrate-signed messaging.** `omc_codec_encode` produces a sampled-token payload addressed by the canonical AST hash (invariant under whitespace, comments, alpha-rename). `omc_codec_decode_lookup` returns the exact library entry on hash match. `omc_msg_sign_compressed` / `omc_msg_recover_compressed` carry the codec payload inside the substrate-signed wire format with lossless library recovery and full signature integrity. **Wire-byte sizing is honest**: token-count compression is ~N×, but wire-byte savings only appear at payloads ≳500 B with N≥8 (single-message). The always-on value is **library-lookup recovery** — alpha-rename invariant content addressing on the receiver, no shared key. 13 tests pass ([`test_codec.omc`](examples/tests/test_codec.omc), [`test_compressed_messaging.omc`](examples/tests/test_compressed_messaging.omc)). See [`experiments/seed_expansion/FINDINGS.md`](experiments/seed_expansion/FINDINGS.md).
 
 ---
@@ -264,7 +266,7 @@ fn coherent_loop(n) {
 |---|---|
 | `omnimcode-core/` | Parser, AST, interpreter, bytecode VM, substrate (`phi_pi_fib`), HBit, harmonic types, 50+ substrate builtins, substrate-routed heal pass |
 | `omnimcode-codegen/` | LLVM-backed JIT, dual-band lowerer, L1.6 array bridges, 22 harmonic-primitive intrinsics (table-driven) |
-| `omnimcode-cli/` | Standalone binary (`omnimcode-standalone`) + `omc-bench` + `omc-grep` |
+| `omnimcode-cli/` | Standalone binary (`omnimcode-standalone`) + `omc-bench` + `omc-grep` + `omc-kernel` |
 | `omnimcode-wasm/` | WebAssembly target (no LLVM, no Python) |
 | `omnimcode-lsp/` | LSP server for editor integration |
 | `omnimcode-gdextension/` | Godot 4 GDExtension binding |
@@ -354,6 +356,7 @@ Submit a package: PR an entry to [`registry/index.json`](registry/index.json).
 | **Substrate-keyed code codec + compressed messaging** | **shipped**, `omc_codec_encode/decode_lookup` + `omc_msg_sign_compressed/recover`, alpha-rename invariant, token-count ~N× (wire-byte breaks even at ≥500 B + N≥8); always-on win is library-lookup recovery; 13 tests, lossless on in-library content |
 | **Inline error-fix hints** | **shipped**, `Undefined function` errors now carry the suggested fn's signature inline (eliminates a separate `omc_help` round-trip after a typo) |
 | **`omc-grep`: alpha-rename-invariant code archaeology** | **shipped** ([docs/omc_grep.md](docs/omc_grep.md)) — standalone CLI; on OMC's examples: 31.7% redundancy (name-sensitive), 33.0% (body-only); surfaces renamed-but-identical fns that text-grep and ast-grep can't catch |
+| **`omc-kernel`: content-addressed code DAG** | **shipped** ([docs/omc_kernel.md](docs/omc_kernel.md)) — store at ~/.omc/kernel/store/<hash>.omc; alpha-rename invariant sign/verify proven end-to-end; the persistence layer for the codec wire format |
 | Two-engine parity (tree-walk + VM) | shipped, 44/45 byte-identical |
 | Embedded CPython + callbacks | shipped, 6 wrapper libs |
 | WASM + LSP + GDExtension targets | shipped |

README.md

@@ -0,0 +1,121 @@
+# omc-kernel — content-addressed code DAG over canonical hashes
+
+> **Code as a content-addressed Merkle DAG over substrate-canonical
+> addresses.** Version code the way IPFS versions files, except the
+> addressing is semantic instead of byte-level — alpha-rename,
+> whitespace, and comment edits all collapse to the same address.
+
+## What it does
+
+A standalone CLI that maintains a file-system content-addressed
+store at `~/.omc/kernel/store/<canonical_hash_hex>.omc`. Every OMC
+fn has a 64-bit canonical-hash identity (whitespace-stripped,
+comments removed, parameter binding normalized), and the store is
+keyed on that.
+
+Subcommands:
+
+| Command | Purpose |
+|---|---|
+| `omc-kernel ingest DIR` | walk DIR, extract every top-level fn from `.omc` files, store one entry per canonical hash |
+| `omc-kernel fetch HASH` | retrieve stored source by canonical hash (hex) |
+| `omc-kernel stat HASH` | substrate metadata: attractor, distance, fn name, origin file |
+| `omc-kernel ls` | list stored hashes + fn names |
+| `omc-kernel sign FILE` | emit a substrate-signed wire message for the OMC source in FILE |
+| `omc-kernel verify` | read a wire message from stdin, recover content via store lookup |
+| `omc-kernel demo` | end-to-end alpha-rename invariant recovery |
+
+## End-to-end proof (the actual run)
+
+```bash
+$ omc-kernel ingest examples/lib/
+ingested 206 fns: 184 new, 22 already present in store
+
+$ cat > /tmp/renamed.omc <<'EOF'
+fn commit(handle) { return py_call(handle, "commit", []); }
+EOF
+
+$ omc-kernel sign /tmp/renamed.omc | tee wire.json > /dev/null
+# wire is a JSON dict with content_hash + sampled_tokens
+
+$ omc-kernel verify < wire.json
+verify: content_hash = 02158af4e9935df8
+verify: store hash matches; recovered 59 bytes
+fn commit(conn) {
+    return py_call(conn, "commit", []);
+}
+```
+
+Sender wrote `fn commit(handle)`; receiver recovered `fn commit(conn)`
+— the canonical form already stored. Same canonical-hash address, no
+shared key, no certificate authority. Alpha-rename + whitespace
+invariance is intrinsic to the addressing.
+
+## Why this is a kernel primitive
+
+The store is a "kernel" in the OS sense: a single shared substrate
+that holds canonical-form content and serves it to any agent that
+asks for it by hash. The codec we shipped earlier (`omc_msg_*`)
+produces wire messages keyed on the same canonical hash; the kernel
+is the backing store that makes recovery work.
+
+Combined, you get the building blocks for a distributed code DAG:
+- Each fn has a 64-bit stable identity (canonical hash)
+- Each fn's dependencies (callees) form an outgoing-edge set — also
+  hashes
+- The DAG is content-addressed end-to-end; no naming conflicts,
+  no version-string negotiation
+- Substrate signature verifies content integrity without a key
+  (recompute and compare)
+
+## What's NOT shipped (honest limits)
+
+- **No daemon yet.** All operations are CLI-level on the store
+  directly. Multi-process/multi-host access requires a wrapper
+  (Unix domain socket / HTTP / gRPC — pick later).
+- **No peer-to-peer discovery.** Single-node only. Cross-host
+  replication is a follow-on layer: each peer holds its own store,
+  fetches can fall back to peers on miss.
+- **No outgoing-edge graph.** Each store entry has a sidecar with
+  substrate metadata but no callee list. Building the Merkle DAG
+  edges requires parsing each fn's calls and recording their
+  canonical hashes. One-pass extension.
+- **Garbage collection.** No reference counting; entries persist
+  until manually deleted. Reasonable for the prototype.
+- **Compression on disk.** Each entry is stored as raw source for
+  human inspection. Could swap to the codec payload for ~5–7× disk
+  savings on larger libraries (with the store itself as the
+  recovery library — circular but the recovery path is unchanged).
+
+Each limit is a separable extension that doesn't require redesign
+of the address scheme.
+
+## Building
+
+```bash
+PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 cargo build --release --bin omc-kernel
+./target/release/omc-kernel demo
+```
+
+No JIT or Python deps required.
+
+## Environment
+
+| Var | Purpose | Default |
+|---|---|---|
+| `OMC_KERNEL_ROOT` | override store location | `~/.omc/kernel` |
+
+## Relationship to the existing pieces
+
+| Layer | Provides | Used by |
+|---|---|---|
+| `canonical::canonicalize` | the canonical-form normalizer | omc-kernel, omc-grep, codec |
+| `tokenizer::fnv1a_64` | 64-bit canonical hash | all three |
+| `phi_pi_fib::nearest_attractor_with_dist` | Fibonacci-attractor metadata | omc-kernel `stat`, codec messages |
+| `omc_msg_sign_compressed` / `_recover_*` | OMC-builtin wire codec | sender side of the kernel |
+| **`omc-kernel`** | **persistent content-addressed store** | **receiver side** |
+
+The four-layer stack: substrate primitives → tokenizer → codec →
+kernel-store. Everything below the kernel exists already; the
+kernel is the persistence + retrieval layer that makes them
+compose into a real distributed-agent system.

docs/omc_kernel.md

@@ -28,6 +28,13 @@ required-features = ["llvm-jit"]
 name = "omc-grep"
 path = "src/bin/omc_grep.rs"
 
+# Content-addressed store keyed by canonical hash. The distributed-agent
+# kernel primitive: code as a content-addressed Merkle DAG over
+# substrate-canonical addresses. Uses serde_json for wire format.
+[[bin]]
+name = "omc-kernel"
+path = "src/bin/omc_kernel.rs"
+
 [features]
 default = ["python-embed"]
 # CPython embedding for `py_*` builtins. Forwards to core.
@@ -40,6 +47,7 @@ llvm-jit = ["dep:omnimcode-codegen", "dep:inkwell"]
 [dependencies]
 omnimcode-core = { path = "../omnimcode-core", default-features = false }
 omnimcode-codegen = { path = "../omnimcode-codegen", optional = true, features = ["llvm-jit"] }
+serde_json = "1.0"  # used by omc-kernel for wire-format messages
 # inkwell is needed at the CLI level only because we leak the LLVM
 # Context for process-lifetime; the dispatch closure lives on the
 # Interpreter and references the engine memory.