Skip to content

Commit cf0a80a

Browse files
docs(cache-policy): MC-under-SUBC migration map (collapse/relocate/survive)
The grand picture of the self-inflicted transform machinery and its fate under the SUBC migration. Establishes that most of MC's transform is state-reconstruction + multi-writer-defense + per-harness-divergence machinery that exists only because MC is a stateless guest in someone else's process (re-handed a fresh array every pass), sharing one DB with sibling processes, over two message models. Three collapse drivers (no held state / shared DB / two message models), a per-machinery fate table, what genuinely survives, and the honest risk relocation (many shallow per-pass defenses -> three deep golden-vector-testable invariants: delta-anchor correctness, encode/decode byte-fidelity, restart recovery). Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>
1 parent f810335 commit cf0a80a

1 file changed

Lines changed: 104 additions & 0 deletions

File tree

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
# MC-under-SUBC migration map: what collapses, what survives
2+
3+
The grand picture for moving MC's transform into a SUBC Rust module. Companion to
4+
`ck-message-field-inventory.md` (message representation) and
5+
`cache-stability-golden-vectors.json` (cache mechanics). This one maps the
6+
**self-inflicted machinery** - the code that exists not because of provider or
7+
cache requirements, but because of MC's runtime POSITION today.
8+
9+
## Thesis
10+
11+
> Most of MC's transform is not reduction logic. It is **state-reconstruction
12+
> and multi-writer-defense machinery** that exists only because MC is a stateless
13+
> guest in someone else's process, re-handed a freshly-rebuilt message array
14+
> every pass, sharing one SQLite DB with sibling processes, over two different
15+
> message models. CK + a single stateful SUBC daemon + one canonical message
16+
> deletes that machinery. The cache discipline is not deleted - it is
17+
> centralized into the cache-policy core we already specced.
18+
19+
## The three collapse drivers
20+
21+
Every self-inflicted quirk traces to exactly one of these, each changed by the migration:
22+
23+
1. **MC holds no state between passes.** OpenCode/Pi re-hand a fresh array with
24+
ORIGINAL un-prefixed content every pass, so every mutation (tag prefix, strip,
25+
clear, drop, compress) must be RE-DERIVED deterministically each pass and
26+
gated by a persisted-DB watermark to stay byte-identical. -> CK: the module
27+
HOLDS the canonical array; it receives a tail-delta + fingerprint, not the
28+
full array.
29+
2. **Many processes share one session DB.** OpenCode + Pi + multiple instances
30+
race, so every persisted decision is written via CAS/delta and guarded by
31+
`BEGIN IMMEDIATE` and leases. -> CK: one MC daemon is the single writer.
32+
3. **Two message models** (OpenCode MessageV2, Pi AgentMessage). -> CK: one CK
33+
Message with thin per-harness encoders/decoders.
34+
35+
## The fate table
36+
37+
Fate ∈ {COLLAPSE (gone), RELOCATE (survives but moves/centralizes), SURVIVE
38+
(genuine, largely unchanged)}.
39+
40+
| Machinery (today) | Driver | Fate | Why |
41+
|---|---|---|---|
42+
| `§N§` re-prefix every pass (`tag.loop`, O(parts)/pass, ~53ms on 2M sessions, note #283) | 1 | RELOCATE | The held array stays prefixed; prefixing becomes incremental on tail-entry only. The hottest perf path largely vanishes. |
43+
| Reasoning-clear replay (`cleared_reasoning_through_tag` watermark, re-applied every pass) | 1 | RELOCATE | The held array already has cleared reasoning; the watermark demotes from per-pass-hot to cold restart-recovery. |
44+
| Placeholder-strip replay (`stripped_placeholder_ids`) | 1 | RELOCATE | Same - decision embodied in held state. |
45+
| Stale-`ctx_reduce`-strip replay (`stale_reduce_stripped_ids`) | 1 | RELOCATE | Same. |
46+
| Processed-image-strip replay (`stripped_image_ids`) | 1 | RELOCATE | Same. |
47+
| Synthetic-todowrite byte-identical replay (`mc_synthetic_todo_<hash>` anchor) | 1 | RELOCATE | Same - held, not re-synthesized per pass. |
48+
| Emergency-drop idempotence latch (`last_emergency_input_sample`) | 1 | RELOCATE | Held-state makes the "did I already drop on this input" check trivial. |
49+
| Caveman depth replay (`tags.caveman_depth`) + `drop_mode` replay (`tags.drop_mode`) | 1 | RELOCATE | Held compressed/dropped state replaces per-pass re-derivation. |
50+
| **All of the above unify into the cache-policy core's single frozen-set** | 1 | RELOCATE | Per-strip watermark tables -> one `frozen_units` set in the Rust core (the golden-vector state). N tables -> 1. |
51+
| Empty-text **sentinels for index stability** (`makeSentinel`/`isSentinel`/`replaySentinelByMessageIds`) | 1 | COLLAPSE | They exist ONLY to keep array-index shape stable across re-handed arrays. We hold post-strip state -> just omit the part. Gone. (The VISIBLE `[dropped §N§]` placeholder is NOT this - it survives as a frozen-unit render.) |
52+
| Sentinel-in-place (OpenCode) **vs** splice-out (Pi) drop mechanism split | 3 | COLLAPSE | One CK drop mechanism. The split was pure message-model accident. |
53+
| Pi omits `stripStructuralNoise` / `stripReasoningFromMergedAssistants` | 3 | COLLAPSE | Becomes an encoder property (CK carries no structural-noise parts; merge is a serializer property), not divergent transform code. |
54+
| CAS/delta writes (`applyStrippedPlaceholderDelta`, `casChannel2NudgeState`, "sibling concurrent discovery" guards) | 2 | COLLAPSE | One writer -> no races -> no CAS/delta defense. |
55+
| Lease subsystem (Channel-2 claims + `healWedgedChannel2Claims`, compartment leases, dream leases, `BEGIN IMMEDIATE` coordination) | 2 | RELOCATE/COLLAPSE | Single-daemon serialization replaces cross-process leases; intra-daemon task ordering is in-process, not DB-lease. |
56+
| Per-harness duplicated transform code (the whole PARITY.md tax) | 3 | COLLAPSE | One transform + two thin adapters. PARITY.md mostly retires. |
57+
| Provider-quirk residual (empty-content `[dropped]`, reasoning shape) | (prev round) | RELOCATE | Downstream quirk pass; ZERO on owned path if llm-runner heals like Pi (see inventory healing profiles). |
58+
| Compaction-marker coordination with OpenCode (`filterCompacted` boundary) | 3 | SURVIVE (plugin-path only) | Still needed where MC is a guest in OpenCode; internal/absent on the owned harness. |
59+
60+
## What genuinely SURVIVES (not artifacts)
61+
62+
These are real session facts, not reconstruction machinery - they persist under CK too:
63+
64+
- The reduction/reclaim **decisions** (what to drop, when, the scheduler) - the
65+
actual product logic.
66+
- The cache classification (SOFT+/SOFT/HARD) and frozen-set discipline -
67+
**hoisted** into the SUBC cache core, not deleted.
68+
- The VISIBLE `[dropped §N§]` placeholder content (model awareness) - a
69+
frozen-unit render.
70+
- Usage tracking, protected-tail boundary, overflow detection, historian-failure
71+
state, emergency-drain latch.
72+
- The note system, auto-search hints, nudge cooldown bands (the cooldown is
73+
genuine; only the multi-writer claim machinery collapses).
74+
- The `tags` table identity + cached token counts (agent references + sums) -
75+
genuine; only the re-prefix-every-pass loop relocates.
76+
- Durable restart-recovery persistence - still required, but COLD path
77+
(reconstruct on daemon restart), not per-pass.
78+
79+
## Honest risk relocation (complexity moves, does not vanish)
80+
81+
The migration trades MANY shallow per-pass defenses for a FEW deep invariants.
82+
The new load-bearing risks:
83+
84+
1. **Delta-anchor correctness.** The tail-delta + fingerprint protocol must be
85+
exactly right or it is a cache catastrophe - this is the golden-vector
86+
anchor-validity (host-trim-below-coverage vs in-coverage-revert). One bug
87+
here replaces a whole class of small ones.
88+
2. **Encode/decode byte-fidelity.** The keystone invariant
89+
`render(project(ck)) == render(project(strip_harness_flags(ck)))` must hold,
90+
or held-state diverges from wire.
91+
3. **Restart recovery.** The stateful daemon must reconstruct canonical + the
92+
frozen-set from durable store on restart, byte-identically.
93+
94+
These are fewer, centralized, and golden-vector-testable - which is the good
95+
trade. The self-inflicted machinery is replaced by three invariants we have
96+
already specced.
97+
98+
## Headline
99+
100+
CK + single stateful daemon + one canonical message deletes the
101+
state-reconstruction half of MC's transform and unifies the multi-writer-defense
102+
and per-harness-divergence halves. The perf hot paths (re-prefix, full-scan
103+
re-derivation) largely vanish because the held array is incremental. What remains
104+
is reduction logic + the centralized cache core + three load-bearing invariants.

0 commit comments

Comments
 (0)