RandomCoder-lab
diff --git a/‎README.md‎
Lines changed: 1 addition & 0 deletions b/‎README.md‎
Lines changed: 1 addition & 0 deletions
diff --git a/‎docs/SUBSTRATE_NATIVE_AGENT.md‎
Lines changed: 216 additions & 0 deletions b/‎docs/SUBSTRATE_NATIVE_AGENT.md‎
Lines changed: 216 additions & 0 deletions
diff --git a/‎examples/lib/agent.omc‎
Lines changed: 140 additions & 0 deletions b/‎examples/lib/agent.omc‎
Lines changed: 140 additions & 0 deletions
@@ -355,6 +355,7 @@ Submit a package: PR an entry to [`registry/index.json`](registry/index.json).
 | **Prometheus: substrate-native ML framework** | **MVP shipped + 4 substrate-moat features verified** ([docs](omnimcode-core/src/prometheus/README.md)) — pure-OMC training (no PyTorch in the loop), content-addressed checkpoints, geodesic bias primitive, **harmonic SGD WINS 3/3 seeds at -13.2% vs vanilla SGD on tinyLM**, canonical-hash inference cache surviving model reload. |
 | **Parameter-free substrate attention WINS 3/3 (−21.5%)** | Four-way A/B: standard QKV → substrate-K → substrate-K+Q → fully substrate. Monotonic improvement at every step *down* the substrate ladder; the variant with ZERO learnable attention params (CRT-PE as K and Q, identity V) beats standard learned attention by 21.5% on 3/3 seeds. See [`SUBSTRATE_ATTENTION_4WAY.md`](experiments/prometheus_parity/SUBSTRATE_ATTENTION_4WAY.md). |
 | **Substrate-K attention WINS −8% val on TinyShakespeare** | The architectural sweet spot. K = CRT-Fibonacci positional table (no learnable K); Q and V stay learned. On 1.1MB corpus with 90/10 train/val split: L1 val=0.104 vs L0 val=0.113, **−8.0% with ~9% fewer params**. Fully-substrate (L3) catastrophically fails at scale; L1 is the architectural recommendation. See [`SUBSTRATE_K_FINDING.md`](experiments/prometheus_parity/SUBSTRATE_K_FINDING.md). |
+| **End-to-end substrate-native agent** | Two agents conversing over OMC-PROTOCOL with persistent fibtier memory across a simulated process restart. Every primitive shipped this week composed into one demonstrable system. 15 conversation turns → 7 bounded entries per agent; restart preserves state; memory query surfaces as fallback knowledge. See [`SUBSTRATE_NATIVE_AGENT.md`](docs/SUBSTRATE_NATIVE_AGENT.md). |
 | Self-hosting compiler V.9b | shipped, gen2 == gen3 byte-identical |
 | **Self-healing pass (7 classes, substrate-routed typo)** | shipped, `OMC_HEAL=1`, **10× typo lookup**, 16 tests, per-class pragmas |
 | **Substrate-keyed code codec + compressed messaging** | **shipped**, `omc_codec_encode/decode_lookup` + `omc_msg_sign_compressed/recover`, alpha-rename invariant, token-count ~N× (wire-byte breaks even at ≥500 B + N≥8); always-on win is library-lookup recovery; 13 tests, lossless on in-library content |
 
@@ -0,0 +1,216 @@
+# The substrate-native agent — every primitive composed
+
+> This is the demonstrable end of the week's substrate-native AI work.
+> Every primitive shipped earlier (kernel, codec, fibtier, OMC-PROTOCOL,
+> Prometheus, content-addressed checkpoints) is load-bearing in this
+> single demo. Each piece's value is visible because the others are
+> present.
+
+## The demo
+
+```bash
+omnimcode-standalone examples/substrate_agent_demo.omc
+```
+
+Two agents — **Curio** (questioner) and **Sage** (responder) — hold a
+15-turn conversation across a simulated process restart. Each agent
+runs the full substrate-native AI stack:
+
+| Layer | Primitive |
+|---|---|
+| identity | `fnv1a_hash(name)` → sender_id (no shared key needed) |
+| memory | Persistent fibtier (`~/.omc/fibtier/<name>/`) |
+| wire format | OMC-PROTOCOL substrate-signed messages |
+| persistence | Manifest JSON journaled per push; reload reconstructs full state |
+| responder | Knowledge-dispatch (could be Prometheus LM via one swap) |
+
+## What happens, scene by scene
+
+### Phase A — fresh start, 12-turn conversation
+
+Both agents are constructed from scratch. Each push to memory triggers:
+1. fibtier cascade — overflow folds upward through Fibonacci tiers
+2. manifest journal — current state written to disk
+3. content-addressed entry IDs — every entry has its canonical hash
+
+Sample turn:
+```
+[Curio → Sage] "What is CRT-PE?"
+[Sage → Curio] "CRT-PE is positional encoding using sin/cos pairs
+                over Fibonacci moduli {5,8,13,21,...}. It won -5.4%
+                val loss on TinyShakespeare in 3/3 seeds."
+```
+
+Behind that two-line exchange:
+- Curio signs a 1-line wire message (~200 bytes JSON, substrate-signed)
+- Sage receives, verifies signature (`omc_msg_verify` returns `valid: 1`)
+- Sage's responder looks up CRT-PE in its knowledge dict
+- Sage signs the reply, ships it
+- Curio verifies the reply, pushes Q+A into its fibtier
+- Sage also pushes the Q+A into its fibtier
+- Both manifests update on disk
+
+### Phase B — memory snapshot after 12 turns
+
+```
+[curio_agent | role=questioner | sender_id=410668497]
+  memory: 12 pushes, 6 folds, 6 entries
+  tier occupancy: [1, 1, 3, 1, 0, 0, 0]
+[sage_agent  | role=responder  | sender_id=144951395]
+  memory: 12 pushes, 6 folds, 6 entries
+  tier occupancy: [1, 1, 3, 1, 0, 0, 0]
+```
+
+**12 conversation turns → 6 stored entries** per agent. Memory is bounded
+by the Fibonacci tier capacities, not by conversation length.
+
+### Phase C — simulated process restart
+
+```
+(discarding in-memory state; reloading both agents from disk)
+
+Reloaded state:
+[curio_agent | ...]
+  memory: 12 pushes, 6 folds, 6 entries  ← identical to pre-restart
+  tier occupancy: [1, 1, 3, 1, 0, 0, 0]  ← identical
+```
+
+The fibtier_persistent_load reads the manifest JSON, rebuilds the
+in-memory representation, and the agent picks up exactly where it
+left off. No state lost; no shared key needed for verification.
+
+### Phase D — resume conversation
+
+Three more turns. Curio asks a question Sage has no direct knowledge
+match for ("Out of all those, which gave the biggest win?"). Sage's
+responder falls back to **querying its own fibtier memory** by
+substrate distance, retrieves the most relevant past entry, and uses
+it as the response:
+
+```
+[Sage → Curio] "That reminds me of: Q: What is L1 substrate-K? | 
+                A: L1 replaces attention's learned K matrix with the
+                CRT-PE positional table. On TinyShakespeare with proper
+                train/val split it wins -8.0% with ~9% fewer params,
+                3/3 seeds."
+```
+
+This is the moment all the pieces compose: **the agent's memory of
+past turns becomes its fallback knowledge** because fibtier stored
+the Q+A as a substrate-addressable entry, the query found it by
+substrate distance, and the responder used the stored content
+directly.
+
+### Final state
+
+```
+[curio_agent]
+  memory: 15 pushes, 8 folds, 7 entries
+  tier occupancy: [1, 2, 2, 2, 0, 0, 0]
+[sage_agent]
+  memory: 15 pushes, 8 folds, 7 entries
+  tier occupancy: [1, 2, 2, 2, 0, 0, 0]
+```
+
+15 conversation turns → 7 entries. Still bounded. Disk artifacts under
+`~/.omc/fibtier/{curio_agent, sage_agent}/manifest.json`.
+
+## What each primitive contributed
+
+| Primitive | Role in the demo | Without it, what fails |
+|---|---|---|
+| `fnv1a_hash` | Stable sender_id from agent name | Identity coordination requires shared keys |
+| `omc_msg_sign` / `omc_msg_verify` | Substrate-signed wire format | No integrity guarantee on inter-agent messages |
+| `fibtier_push` / `_cascade_overflow` | Bounded memory with Fibonacci tiering | Context grows linearly forever |
+| `fibtier_query` | Substrate-distance memory retrieval | Agent has no fallback for unknown queries |
+| `fibtier_persistent_*` | Manifest journaling | Memory dies with the process |
+| Canonical hash addressing | Per-entry content identity | No dedup, no integrity, no cross-agent reference |
+| `py_exec`/`py_eval` | OS path management (mkdir, env vars) | Persistence layer can't bootstrap its own paths |
+
+Remove any one and the demo breaks at a specific point. They're not
+independent features — they're a system.
+
+## What it would take to make Sage's responder a Prometheus LM
+
+Replace `_agent_respond` in `examples/lib/agent.omc`:
+
+```omc
+fn _agent_respond(agent, input_text) {
+    h ctx = fibtier_query(dict_get(agent, "memory"), input_text, 3);
+    h ctx_text = render_context(ctx);
+    h prompt = concat_many(ctx_text, "\n\nQ: ", input_text, "\nA:");
+    h tokens = prom_generate_greedy(
+        agent_model_forward,
+        dict_get(agent, "prom_model"),
+        encode_chars(prompt),
+        50,
+        VOCAB_SIZE
+    );
+    return decode(tokens);
+}
+```
+
+Plug in any trained Prometheus model (built with our L1 substrate-K
+attention as the default), and the agent generates substrate-native
+LM responses while keeping every other layer of the stack the same.
+
+## What this proves
+
+The substrate-native AI stack OMC built this week is **composable in
+practice, not just on a diagram.** Two agents share an OMC-PROTOCOL
+channel; each maintains a persistent fibtier; both survive process
+restart; the memory layer surfaces as a fallback knowledge source
+when the direct responder runs out of answers.
+
+Six primitives (codec, kernel, fibtier, protocol, prometheus,
+checkpoints) → one working agent demo → ~250 lines of OMC + ~150
+lines of agent.omc + ~200 lines of fibtier_persistent.omc.
+
+The architecture is the substrate. The substrate is the architecture.
+
+## Files
+
+| Path | What |
+|---|---|
+| `examples/lib/fibtier.omc` | In-memory Fibonacci-tier core |
+| `examples/lib/fibtier_persistent.omc` | Manifest-journaled persistence layer |
+| `examples/lib/agent.omc` | Agent abstraction (identity + memory + send/receive) |
+| `examples/substrate_agent_demo.omc` | The end-to-end demo script |
+| `examples/tests/test_fibtier.omc` | 8/8 tests |
+| `examples/tests/test_fibtier_persistent.omc` | 4/4 persistence tests |
+| `docs/SUBSTRATE_NATIVE_AGENT.md` | This file |
+
+## How to reproduce
+
+```bash
+PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 cargo build --release --bin omnimcode-standalone
+./target/release/omnimcode-standalone examples/substrate_agent_demo.omc
+```
+
+Memory artifacts persist under `~/.omc/fibtier/curio_agent/` and
+`~/.omc/fibtier/sage_agent/`. Re-running the demo from a clean state:
+
+```bash
+rm -rf ~/.omc/fibtier/curio_agent ~/.omc/fibtier/sage_agent
+./target/release/omnimcode-standalone examples/substrate_agent_demo.omc
+```
+
+Tests:
+```bash
+./target/release/omnimcode-standalone --test examples/tests/test_fibtier.omc
+./target/release/omnimcode-standalone --test examples/tests/test_fibtier_persistent.omc
+```
+
+## What's next (not part of this demo)
+
+- **LLM-summarization fold** — replace concat-fold with a py_callback
+  to Claude/GPT for true semantic compression. Substrate captures
+  structure; LLM captures meaning.
+- **MCP exposure** — wrap fibtier as MCP tools so any Claude Desktop
+  / Cursor session gets the bounded-memory architecture natively.
+- **Substrate transformer integration** — wire Prometheus' L1
+  substrate-K transformer as the agent's response generator.
+- **N-agent mesh** — extend from 2 agents to a network. OMC-PROTOCOL
+  handles arbitrary peers; fibtier handles arbitrary message volume.
+
+Each is a natural extension of what already works.
@@ -0,0 +1,140 @@
+# Substrate-native agent — composes fibtier + kernel + protocol + (optional) Prometheus.
+#
+# An agent is:
+#   - identity (sender_id derived from name)
+#   - memory (persistent fibtier; ~/.omc/fibtier/<name>/)
+#   - knowledge (a dict of fact → response, for the demo's generator)
+#   - role (string, used in introductions)
+#
+# Agents talk over OMC-PROTOCOL: every message is substrate-signed
+# (omc_msg_sign), serialized via omc_msg_serialize. Recipients verify
+# via omc_msg_verify with no shared key. The wire format and identity
+# guarantees come for free from primitives shipped earlier.
+#
+# Replace knowledge_dispatch with a Prometheus-trained generator
+# when you want the agent's responses to come from a substrate-native
+# LM. For now, the demo uses a dictionary-driven responder so the
+# conversation is intelligible.
+
+import "examples/lib/fibtier_persistent.omc";
+
+# ---------------------------------------------------------------------------
+# Identity: substrate-derived sender_id from the agent name.
+# ---------------------------------------------------------------------------
+
+fn _agent_sender_id(name) {
+    h hval = fnv1a_hash(name);
+    # Truncate to 31 bits (positive int32) to match omc_msg_sign expectation.
+    h pos = hval - (hval / 2147483648) * 2147483648;
+    if pos < 0 { pos = 0 - pos; }
+    return pos;
+}
+
+# ---------------------------------------------------------------------------
+# Agent construction
+# ---------------------------------------------------------------------------
+
+fn agent_new(name, role, knowledge_dict, max_tiers) {
+    fibtier_persistent_new(name, max_tiers);
+    h mem = fibtier_persistent_load(name);
+    h agent = dict_new();
+    dict_set(agent, "name", name);
+    dict_set(agent, "role", role);
+    dict_set(agent, "sender_id", _agent_sender_id(name));
+    dict_set(agent, "memory", mem);
+    dict_set(agent, "knowledge", knowledge_dict);
+    return agent;
+}
+
+# Reload an existing agent (preserves memory).
+fn agent_load(name, role, knowledge_dict) {
+    h mem = fibtier_persistent_load(name);
+    h agent = dict_new();
+    dict_set(agent, "name", name);
+    dict_set(agent, "role", role);
+    dict_set(agent, "sender_id", _agent_sender_id(name));
+    dict_set(agent, "memory", mem);
+    dict_set(agent, "knowledge", knowledge_dict);
+    return agent;
+}
+
+# ---------------------------------------------------------------------------
+# Response generator (knowledge-based for the demo)
+#
+# Looks up the input text against the agent's knowledge dict. If
+# nothing matches, returns a substrate-aware "I don't know"-style
+# response. Production agents would replace this with a Prometheus
+# forward pass over the input + retrieved context from memory.
+# ---------------------------------------------------------------------------
+
+fn _agent_respond(agent, input_text) {
+    h kn = dict_get(agent, "knowledge");
+    # Linear scan; keys are short substrings to match against input.
+    h keys = dict_keys(kn);
+    h i = 0;
+    while i < arr_len(keys) {
+        h key = arr_get(keys, i);
+        if re_match(key, input_text) == 1 {
+            return dict_get(kn, key);
+        }
+        i = i + 1;
+    }
+    # Fallback: query memory for closest related entry.
+    h hits = fibtier_query(dict_get(agent, "memory"), input_text, 1);
+    if arr_len(hits) > 0 {
+        h hit = arr_get(hits, 0);
+        h entry = dict_get(hit, "entry");
+        return concat_many("That reminds me of: ", dict_get(entry, "content"));
+    }
+    return concat_many("I haven't encountered \"", input_text, "\" yet.");
+}
+
+# ---------------------------------------------------------------------------
+# Inter-agent messaging via OMC-PROTOCOL
+# ---------------------------------------------------------------------------
+
+# Send a substrate-signed message. Returns the wire-serialized form
+# that the recipient deserializes via omc_msg_deserialize.
+fn agent_send(agent, content) {
+    h KIND_REQUEST = 1;
+    h msg = omc_msg_sign(content, dict_get(agent, "sender_id"), KIND_REQUEST);
+    return omc_msg_serialize(msg);
+}
+
+# Receive a wire message: deserialize, verify signature, push verified
+# content into the agent's memory, generate a response, sign + return
+# the response wire. This is the full inbound→outbound loop.
+fn agent_receive(agent, wire) {
+    h msg = omc_msg_deserialize(wire);
+    h check = omc_msg_verify(msg);
+    if dict_get(check, "valid") != 1 {
+        return omc_msg_serialize(
+            omc_msg_sign("[INVALID SIGNATURE]", dict_get(agent, "sender_id"), 8)
+        );
+    }
+    h content = dict_get(check, "content");
+    h reply_content = _agent_respond(agent, content);
+    # Push the Q+A pair into memory.
+    h qa = concat_many("Q: ", content, " | A: ", reply_content);
+    fibtier_persistent_push(dict_get(agent, "memory"), qa);
+    # Sign + serialize the reply.
+    h reply_msg = omc_msg_sign(reply_content, dict_get(agent, "sender_id"), 2);
+    return omc_msg_serialize(reply_msg);
+}
+
+# ---------------------------------------------------------------------------
+# Stats
+# ---------------------------------------------------------------------------
+
+fn agent_stats(agent) {
+    h mem_stats = fibtier_stats(dict_get(agent, "memory"));
+    h s = dict_new();
+    dict_set(s, "name", dict_get(agent, "name"));
+    dict_set(s, "role", dict_get(agent, "role"));
+    dict_set(s, "sender_id", dict_get(agent, "sender_id"));
+    dict_set(s, "memory_pushes", dict_get(mem_stats, "total_pushes"));
+    dict_set(s, "memory_folds", dict_get(mem_stats, "total_folds"));
+    dict_set(s, "memory_entries", dict_get(mem_stats, "total_entries"));
+    dict_set(s, "memory_occupancy", dict_get(mem_stats, "occupancy"));
+    return s;
+}