Skip to content

Commit e25af0d

Browse files
v0.10.0 memory-plus axes 1-4: Merkle manifests, dedup pool, aged zlib, substrate tokenizer
Pushing compression ceiling beyond v1.0's 297× context ratio along four orthogonal axes. All four ship with round-trip verification. Axis 1 — Merkle manifest hashes (omc_memory_create_manifest + omc_memory_recall_manifest). One manifest hash references N leaf hashes. Tested on 19 stored chapters: 1 manifest hash (5 tokens) replaces 19 flat hash refs (95 tokens). Context cost reduction: 19× over v1.0, 5,356× vs raw chapter content in context. Axis 2 — Cross-namespace dedup pool. Bodies live in shared ~/.omc/memory/_pool/<shard>/<hash>.txt; namespace dirs hold only index. Same content stored in K namespaces = 1 body file. Tested 5-way dup: naive 29,505 bytes → pool 5,901 bytes (5× disk). Axis 3 — fibtier-aware progressive compression (omc_memory_compact). Aged pool bodies rewritten as zlib-deflated with `OMCZ` magic prefix. Tested on real chapter: 6,008 → 2,743 bytes (2.19× disk). Axis 4 — substrate-aware tokenizer compaction (omc_memory_compact_substrate). Aged bodies re-encoded via tokenizer::encode + varint + deflate with `OMCT` magic. Tested on OMC code: 8,018 → 3,383 bytes (2.37× disk). Honest comparison: roughly ties OMCZ zlib (2.48× on similar content). The substrate-native path is a substrate-tokenizer integration win even though raw compression is on par with zlib. Architecture: recall path transparently inflates OMCZ + OMCT bodies legacy per-namespace storage still works as fallback 6 new MCP tools (compact, compact_substrate, create_manifest, recall_manifest, plus the existing 5) flate2 added to omnimcode-core deps (rust_backend, no system zlib) Compounding ceiling so far: Axis 1: 5,356× context compression (manifest hash for 19 chapters) Axis 2: 5× disk on duplicated content (linear with namespace count) Axis 3: 2.19× disk on aged content via zlib Axis 4: 2.37× disk on aged content via substrate tokenizer (ties Axis 3) Tests: 1111/1111 OMC tests still pass; round-trip verified for OMCZ and OMCT bodies via MCP integration test. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 19bbe1f commit e25af0d

4 files changed

Lines changed: 551 additions & 9 deletions

File tree

Cargo.lock

Lines changed: 42 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

omnimcode-core/Cargo.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,10 @@ path = "src/lib.rs"
1515
[dependencies]
1616
regex = "1.10"
1717
thiserror = "1.0"
18+
# v0.9.3 Axis 3 — aged-tier memory compression. Used by the MemoryStore
19+
# `compact_namespace` operation to rewrite old pool bodies as zlib-deflated
20+
# blobs. Adds <100KB to the binary.
21+
flate2 = { version = "1.0", default-features = false, features = ["rust_backend"] }
1822
serde_json = "1.0"
1923
sha2 = "0.10"
2024
base64 = "0.22"

0 commit comments

Comments
 (0)