|
| 1 | +# MC-under-SUBC migration map: what collapses, what survives |
| 2 | + |
| 3 | +The grand picture for moving MC's transform into a SUBC Rust module. Companion to |
| 4 | +`ck-message-field-inventory.md` (message representation) and |
| 5 | +`cache-stability-golden-vectors.json` (cache mechanics). This one maps the |
| 6 | +**self-inflicted machinery** - the code that exists not because of provider or |
| 7 | +cache requirements, but because of MC's runtime POSITION today. |
| 8 | + |
| 9 | +## Thesis |
| 10 | + |
| 11 | +> Most of MC's transform is not reduction logic. It is **state-reconstruction |
| 12 | +> and multi-writer-defense machinery** that exists only because MC is a stateless |
| 13 | +> guest in someone else's process, re-handed a freshly-rebuilt message array |
| 14 | +> every pass, sharing one SQLite DB with sibling processes, over two different |
| 15 | +> message models. CK + a single stateful SUBC daemon + one canonical message |
| 16 | +> deletes that machinery. The cache discipline is not deleted - it is |
| 17 | +> centralized into the cache-policy core we already specced. |
| 18 | +
|
| 19 | +## The three collapse drivers |
| 20 | + |
| 21 | +Every self-inflicted quirk traces to exactly one of these, each changed by the migration: |
| 22 | + |
| 23 | +1. **MC holds no state between passes.** OpenCode/Pi re-hand a fresh array with |
| 24 | + ORIGINAL un-prefixed content every pass, so every mutation (tag prefix, strip, |
| 25 | + clear, drop, compress) must be RE-DERIVED deterministically each pass and |
| 26 | + gated by a persisted-DB watermark to stay byte-identical. -> CK: the module |
| 27 | + HOLDS the canonical array; it receives a tail-delta + fingerprint, not the |
| 28 | + full array. |
| 29 | +2. **Many processes share one session DB.** OpenCode + Pi + multiple instances |
| 30 | + race, so every persisted decision is written via CAS/delta and guarded by |
| 31 | + `BEGIN IMMEDIATE` and leases. -> CK: one MC daemon is the single writer. |
| 32 | +3. **Two message models** (OpenCode MessageV2, Pi AgentMessage). -> CK: one CK |
| 33 | + Message with thin per-harness encoders/decoders. |
| 34 | + |
| 35 | +## The fate table |
| 36 | + |
| 37 | +Fate ∈ {COLLAPSE (gone), RELOCATE (survives but moves/centralizes), SURVIVE |
| 38 | +(genuine, largely unchanged)}. |
| 39 | + |
| 40 | +| Machinery (today) | Driver | Fate | Why | |
| 41 | +|---|---|---|---| |
| 42 | +| `§N§` re-prefix every pass (`tag.loop`, O(parts)/pass, ~53ms on 2M sessions, note #283) | 1 | RELOCATE | The held array stays prefixed; prefixing becomes incremental on tail-entry only. The hottest perf path largely vanishes. | |
| 43 | +| Reasoning-clear replay (`cleared_reasoning_through_tag` watermark, re-applied every pass) | 1 | RELOCATE | The held array already has cleared reasoning; the watermark demotes from per-pass-hot to cold restart-recovery. | |
| 44 | +| Placeholder-strip replay (`stripped_placeholder_ids`) | 1 | RELOCATE | Same - decision embodied in held state. | |
| 45 | +| Stale-`ctx_reduce`-strip replay (`stale_reduce_stripped_ids`) | 1 | RELOCATE | Same. | |
| 46 | +| Processed-image-strip replay (`stripped_image_ids`) | 1 | RELOCATE | Same. | |
| 47 | +| Synthetic-todowrite byte-identical replay (`mc_synthetic_todo_<hash>` anchor) | 1 | RELOCATE | Same - held, not re-synthesized per pass. | |
| 48 | +| Emergency-drop idempotence latch (`last_emergency_input_sample`) | 1 | RELOCATE | Held-state makes the "did I already drop on this input" check trivial. | |
| 49 | +| Caveman depth replay (`tags.caveman_depth`) + `drop_mode` replay (`tags.drop_mode`) | 1 | RELOCATE | Held compressed/dropped state replaces per-pass re-derivation. | |
| 50 | +| **All of the above unify into the cache-policy core's single frozen-set** | 1 | RELOCATE | Per-strip watermark tables -> one `frozen_units` set in the Rust core (the golden-vector state). N tables -> 1. | |
| 51 | +| Empty-text **sentinels for index stability** (`makeSentinel`/`isSentinel`/`replaySentinelByMessageIds`) | 1 | COLLAPSE | They exist ONLY to keep array-index shape stable across re-handed arrays. We hold post-strip state -> just omit the part. Gone. (The VISIBLE `[dropped §N§]` placeholder is NOT this - it survives as a frozen-unit render.) | |
| 52 | +| Sentinel-in-place (OpenCode) **vs** splice-out (Pi) drop mechanism split | 3 | COLLAPSE | One CK drop mechanism. The split was pure message-model accident. | |
| 53 | +| Pi omits `stripStructuralNoise` / `stripReasoningFromMergedAssistants` | 3 | COLLAPSE | Becomes an encoder property (CK carries no structural-noise parts; merge is a serializer property), not divergent transform code. | |
| 54 | +| CAS/delta writes (`applyStrippedPlaceholderDelta`, `casChannel2NudgeState`, "sibling concurrent discovery" guards) | 2 | COLLAPSE | One writer -> no races -> no CAS/delta defense. | |
| 55 | +| Lease subsystem (Channel-2 claims + `healWedgedChannel2Claims`, compartment leases, dream leases, `BEGIN IMMEDIATE` coordination) | 2 | RELOCATE/COLLAPSE | Single-daemon serialization replaces cross-process leases; intra-daemon task ordering is in-process, not DB-lease. | |
| 56 | +| Per-harness duplicated transform code (the whole PARITY.md tax) | 3 | COLLAPSE | One transform + two thin adapters. PARITY.md mostly retires. | |
| 57 | +| Provider-quirk residual (empty-content `[dropped]`, reasoning shape) | (prev round) | RELOCATE | Downstream quirk pass; ZERO on owned path if llm-runner heals like Pi (see inventory healing profiles). | |
| 58 | +| Compaction-marker coordination with OpenCode (`filterCompacted` boundary) | 3 | SURVIVE (plugin-path only) | Still needed where MC is a guest in OpenCode; internal/absent on the owned harness. | |
| 59 | + |
| 60 | +## What genuinely SURVIVES (not artifacts) |
| 61 | + |
| 62 | +These are real session facts, not reconstruction machinery - they persist under CK too: |
| 63 | + |
| 64 | +- The reduction/reclaim **decisions** (what to drop, when, the scheduler) - the |
| 65 | + actual product logic. |
| 66 | +- The cache classification (SOFT+/SOFT/HARD) and frozen-set discipline - |
| 67 | + **hoisted** into the SUBC cache core, not deleted. |
| 68 | +- The VISIBLE `[dropped §N§]` placeholder content (model awareness) - a |
| 69 | + frozen-unit render. |
| 70 | +- Usage tracking, protected-tail boundary, overflow detection, historian-failure |
| 71 | + state, emergency-drain latch. |
| 72 | +- The note system, auto-search hints, nudge cooldown bands (the cooldown is |
| 73 | + genuine; only the multi-writer claim machinery collapses). |
| 74 | +- The `tags` table identity + cached token counts (agent references + sums) - |
| 75 | + genuine; only the re-prefix-every-pass loop relocates. |
| 76 | +- Durable restart-recovery persistence - still required, but COLD path |
| 77 | + (reconstruct on daemon restart), not per-pass. |
| 78 | + |
| 79 | +## Honest risk relocation (complexity moves, does not vanish) |
| 80 | + |
| 81 | +The migration trades MANY shallow per-pass defenses for a FEW deep invariants. |
| 82 | +The new load-bearing risks: |
| 83 | + |
| 84 | +1. **Delta-anchor correctness.** The tail-delta + fingerprint protocol must be |
| 85 | + exactly right or it is a cache catastrophe - this is the golden-vector |
| 86 | + anchor-validity (host-trim-below-coverage vs in-coverage-revert). One bug |
| 87 | + here replaces a whole class of small ones. |
| 88 | +2. **Encode/decode byte-fidelity.** The keystone invariant |
| 89 | + `render(project(ck)) == render(project(strip_harness_flags(ck)))` must hold, |
| 90 | + or held-state diverges from wire. |
| 91 | +3. **Restart recovery.** The stateful daemon must reconstruct canonical + the |
| 92 | + frozen-set from durable store on restart, byte-identically. |
| 93 | + |
| 94 | +These are fewer, centralized, and golden-vector-testable - which is the good |
| 95 | +trade. The self-inflicted machinery is replaced by three invariants we have |
| 96 | +already specced. |
| 97 | + |
| 98 | +## Headline |
| 99 | + |
| 100 | +CK + single stateful daemon + one canonical message deletes the |
| 101 | +state-reconstruction half of MC's transform and unifies the multi-writer-defense |
| 102 | +and per-harness-divergence halves. The perf hot paths (re-prefix, full-scan |
| 103 | +re-derivation) largely vanish because the held array is incremental. What remains |
| 104 | +is reduction logic + the centralized cache core + three load-bearing invariants. |
0 commit comments