Commit e25af0d
v0.10.0 memory-plus axes 1-4: Merkle manifests, dedup pool, aged zlib, substrate tokenizer
Pushing compression ceiling beyond v1.0's 297× context ratio along four
orthogonal axes. All four ship with round-trip verification.
Axis 1 — Merkle manifest hashes (omc_memory_create_manifest +
omc_memory_recall_manifest). One manifest hash references N leaf hashes.
Tested on 19 stored chapters: 1 manifest hash (5 tokens) replaces 19
flat hash refs (95 tokens). Context cost reduction: 19× over v1.0,
5,356× vs raw chapter content in context.
Axis 2 — Cross-namespace dedup pool. Bodies live in shared
~/.omc/memory/_pool/<shard>/<hash>.txt; namespace dirs hold only index.
Same content stored in K namespaces = 1 body file. Tested 5-way dup:
naive 29,505 bytes → pool 5,901 bytes (5× disk).
Axis 3 — fibtier-aware progressive compression (omc_memory_compact).
Aged pool bodies rewritten as zlib-deflated with `OMCZ` magic prefix.
Tested on real chapter: 6,008 → 2,743 bytes (2.19× disk).
Axis 4 — substrate-aware tokenizer compaction (omc_memory_compact_substrate).
Aged bodies re-encoded via tokenizer::encode + varint + deflate with
`OMCT` magic. Tested on OMC code: 8,018 → 3,383 bytes (2.37× disk).
Honest comparison: roughly ties OMCZ zlib (2.48× on similar content).
The substrate-native path is a substrate-tokenizer integration win
even though raw compression is on par with zlib.
Architecture:
recall path transparently inflates OMCZ + OMCT bodies
legacy per-namespace storage still works as fallback
6 new MCP tools (compact, compact_substrate, create_manifest,
recall_manifest, plus the existing 5)
flate2 added to omnimcode-core deps (rust_backend, no system zlib)
Compounding ceiling so far:
Axis 1: 5,356× context compression (manifest hash for 19 chapters)
Axis 2: 5× disk on duplicated content (linear with namespace count)
Axis 3: 2.19× disk on aged content via zlib
Axis 4: 2.37× disk on aged content via substrate tokenizer (ties Axis 3)
Tests: 1111/1111 OMC tests still pass; round-trip verified for OMCZ
and OMCT bodies via MCP integration test.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>1 parent 19bbe1f commit e25af0d
4 files changed
Lines changed: 551 additions & 9 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
18 | 22 | | |
19 | 23 | | |
20 | 24 | | |
| |||
0 commit comments