|
10 | 10 | |---|---|---| |
11 | 11 | | Composition layer (Linear, ReLU, MSE loss, SGD) | `examples/lib/prometheus.omc` | shipped | |
12 | 12 | | Tiny LM training demo | `examples/prometheus_tinylm.omc` | **passes stop condition** | |
| 13 | +| Content-addressed checkpoints | `examples/prometheus_checkpoint.omc` | round-trip verified | |
| 14 | +| Geodesic bias primitive (3/3 seed PyTorch win → pure OMC) | `examples/prometheus_geodesic_bias.omc` | shape + symmetry verified | |
| 15 | +| **Harmonic SGD A/B (substrate-modulated lr)** | `examples/prometheus_harmonic_sgd.omc` | **WINS 3/3 seeds, -13.2%** | |
| 16 | +| Substrate-cached inference | `examples/prometheus_cache.omc` | 3/3 cache hits across model reload | |
13 | 17 | | Reverse-mode autograd | `omnimcode-core/src/interpreter.rs` (`tape_*` builtins, 18 ops, 12 tests) | already shipped | |
14 | 18 | | Forward-mode autograd (duals) | same, `dual_*` builtins (21 ops, 17 tests) | already shipped | |
15 | 19 | | ML kernels | `arr_softmax`, `arr_layer_norm`, `arr_relu_vec`, `arr_sigmoid_vec`, `arr_conv1d`, `arr_outer`, `arr_matmul`, `arr_transpose`, `arr_eye`, `arr_zeros_2d` | already shipped | |
@@ -74,17 +78,26 @@ Each of these is an extension of the existing tape interpreter + |
74 | 78 | the kernel we shipped. They are the **substrate-unique features |
75 | 79 | that PyTorch cannot offer** — the strategic moat. |
76 | 80 |
|
77 | | -## Priority order |
78 | | - |
79 | | -1. **`tape_save_weights` + `tape_load_weights`** via .omcs format. |
80 | | - Cheapest substrate-moat win; uses every primitive we already shipped. |
81 | | -2. **`tape_geodesic_attention`** — promote today's transformerless-LM |
82 | | - win to a first-class primitive. Anyone defining a transformer-replacement |
83 | | - model gets it as one call. |
84 | | -3. **`tape_update_scaled`** — enables the harmonic optimizer hypothesis test. |
85 | | - Small Rust change; large research surface. |
86 | | -4. **`tape_cache_forward`** — the substrate-cache win. Hardest to design |
87 | | - right (cache invalidation rules), highest leverage on training time. |
| 81 | +## Priority order (all four shipped 2026-05-16) |
| 82 | + |
| 83 | +1. ✅ **Content-addressed checkpoints** — `prom_serialize_model` / |
| 84 | + `prom_model_hash` / `prom_load_model` in `examples/lib/prometheus.omc`. |
| 85 | + End-to-end round trip verified in `examples/prometheus_checkpoint.omc`: |
| 86 | + trained model serialized → JSON → hash → tape_reset → reloaded → SAME |
| 87 | + canonical hash + bit-identical predictions. |
| 88 | +2. ✅ **Geodesic attention bias as fused primitive** — |
| 89 | + `prom_geodesic_bias_matrix(seq_len)` in `examples/lib/prometheus.omc`. |
| 90 | + Pure-OMC port of today's PyTorch impl that won 3/3 seeds. Numerically |
| 91 | + identical (symmetric, diag-zero, mean-off-diag normalized to ~1.0). |
| 92 | +3. ✅ **Harmonic SGD** — `prom_harmonic_sgd_step(params, lr, alpha)`. |
| 93 | + A/B against vanilla SGD on the tinyLM bigram task: |
| 94 | + seed 42: -7.7% seed 7: -25.9% seed 123: -19.8% |
| 95 | + harmonic mean -13.2% vs vanilla — WINS 3/3. |
| 96 | +4. ✅ **Substrate-cached inference** — `prom_cache_key` / |
| 97 | + `prom_cache_get` / `prom_cache_put`. Cache survives `tape_reset()` + |
| 98 | + model reload because keys are canonical hashes, not in-memory IDs. |
| 99 | + Demo: `examples/prometheus_cache.omc` shows 3/3 cache hits after |
| 100 | + model rebuilt from a saved bundle. |
88 | 101 |
|
89 | 102 | ## What this is NOT |
90 | 103 |
|
|
0 commit comments