You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hardening release from a partitioned adversarial audit pass across every
subsystem: emergency overflow now fails closed on provider-proven limits
(migration v52), historian gap-healing rejects narrative-bearing gaps instead
of silently absorbing them, first-person historian voice (v8.7.4), compartment
title sanitization with a cache-neutral render-epoch bump, TUI/RPC lifecycle
and upgrade-skew fixes, smart-note sandbox/egress hardening (IPv6 denied),
concurrent-startup migration safety (v51), CLI config protection, Pi image-only
tool-result fix, and the Rust module lane's wire-v2 cutover with paged shadow
seeds and a durable reduce-command ledger.
Squashed from the private working branch; full notes in
.alfonso/release-notes/v0.32.1.md.
Co-authored-by: Alfonso <alfonso-magic-context@users.noreply.github.com>
A hardening release. A partitioned adversarial audit pass covered every subsystem — transform core, historian, emergency paths, storage, TUI/RPC, smart-notes, CLI — and this patch ships the resulting fixes: two silent-data-loss classes closed, a fail-closed overflow gate, a batch of lifecycle races, and cleaner historian summaries.
4
+
5
+
## Context safety
6
+
7
+
### Emergency overflow now fails closed
8
+
At ≥95% usage with no fold available, Magic Context previously did its best-effort reduction and let the request through — which could still overflow the provider and crash the turn mid-stream. The emergency path now blocks the request outright and interrupts the turn cleanly when three conditions hold together: the context limit is provider-proven (a persisted overflow detection or a real 400, never the token estimator), usage is genuinely ≥95%, and no fold materialized in the current pass. The token estimator is demoted to telemetry on this path, so estimate drift can never abort a healthy request. Restart-safe via a new schema column (migration v52).
9
+
10
+
### Historian gaps are rejected, not absorbed
11
+
When the historian's output left small gaps between compartments, a healing step silently extended the previous compartment to cover them. Tool-only noise absorbed that way is fine — but the same path could absorb up to 15 messages of real narrative into a compartment whose summary never saw them: silently missing history. Gap healing is now strict: tool-only gaps still absorb, narrative-bearing gaps reject the chunk and the range is re-read on the next run. Nothing user-visible changes on healthy sessions; degenerate outputs now retry instead of losing content.
12
+
13
+
## Historian
14
+
15
+
### Summaries in the first person
16
+
The historian writes memory for the agent's own future self, but its summaries drifted into third-person reporting ("The agent investigated X..."). The prompt now enforces first-person voice with named actors ("I traced the timeout to readLoop; Alice rejected the load theory"), validated on the production model class: cleaner to read back, and the agent treats recalled history as its own memory rather than a report about someone else.
17
+
18
+
### Compartment titles are sanitized
19
+
Titles emitted by the historian render into the history block verbatim; a degenerate title carrying markup or control characters could distort the rendered structure. Titles are now sanitized at render time (control characters collapsed, markup escaped) in both the TypeScript and Rust renderers, and a render-epoch bump re-renders any cached history containing an unsanitized title once, cache-neutrally.
20
+
21
+
## Pi
22
+
23
+
-**Image-only tool results no longer 400 on Anthropic.** A tool result whose only content was an image (screenshot tools) could serialize to empty content after image handling and be rejected by Anthropic. Those results now carry an `[image stripped]` placeholder.
24
+
- Status/progress notices remain display-only under the stricter delivery path shipped in v0.32.0; a residual leak through the fallback renderer on older Pi versions was closed.
25
+
26
+
## TUI, notifications, and RPC
27
+
28
+
-**Dialogs no longer vanish across restarts.** The notification protocol now resets its cursor on reconnect handshake and acknowledges by exact ID — previously a restart could mark undelivered dialogs (upgrade prompts, warnings) as consumed.
29
+
-**Upgrade-skew compatibility both directions.** A newer TUI can discover an older server and an older TUI no longer 401-loops against a newer server (one-release auth bridge, negotiated legacy mode).
30
+
-**Sidebar no longer zeroes out under DB contention.** A busy database during a snapshot read returned an all-zero snapshot that purged the TUI's cache; it now returns an error envelope and the sidebar keeps its last good state.
31
+
- Fixed a WebSocket sink leak on re-hello (lost TUI actions), a reconnect loop against dead endpoints, port-file collisions between instances, and cross-session cursor pruning.
32
+
33
+
## Smart notes
34
+
35
+
-**Sandbox and egress hardening:** the file denylist re-checks canonical paths (symlink bypass closed), FIFO/special files can no longer wedge the check mutex, note transitions are CAS-guarded against concurrent evaluators, and IPv6 egress is denied entirely (closes a NAT64/Pref64 literal-IP bypass) while dual-stack hosts keep IPv4 reachability.
36
+
37
+
## Storage and CLI
38
+
39
+
-**Concurrent startup is clean.** Multiple instances booting at once could collide on schema migrations (`SQLITE_BUSY`); migrations now take immediate transactions with proper backoff (migration v51), and session deletion no longer leaks a bookkeeping row.
40
+
-**The CLI can't eat your config.** A malformed JSONC config now aborts the write path instead of being replaced with defaults; doctor opens the shared database read-only so a newer schema can never be tripped by an older CLI; issue-report sanitization enforces its size cap post-condition.
41
+
- Background maintenance lanes (search indexing, session backfill) no longer block the event loop on large sessions and recover cleanly from partial failures.
42
+
43
+
## Dashboard
44
+
45
+
- The cache page's session list filtering fix (managed-session detection across Claude Code / Codex stores) rides the next dashboard release.
46
+
47
+
## Under the hood
48
+
49
+
- The Rust module lane (runs under the subconscious daemon; not part of the npm packages) moved to the subc wire-v2 protocol, gained a durable command ledger for idempotent reduction commands, hardened its MCP facade memory operations and historian producer validation, and its shadow-mode dev lane — off by default — now pages large cold-start seeds, bounds its work queues, and rides the shared subc client (fixes a memory-growth path when the daemon stalls).
Copy file name to clipboardExpand all lines: ARCHITECTURE.md
+4-4Lines changed: 4 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -12,21 +12,21 @@ Magic Context is an `@opencode-ai/plugin` (entry `src/index.ts`) that rewrites t
12
12
-**Hidden subagents** (`historian`, `historian-editor`, `dreamer`, `sidekick`) do the heavy LLM work out of band; the transform itself does no LLM calls.
13
13
-**Runtime SQLite backend** (`src/shared/sqlite.ts`): `bun:sqlite` under Bun, `node:sqlite` (`DatabaseSync`) under Node (Pi) and Electron (Desktop). The non-Bun branch adds a savepoint-aware `transaction()` shim and `readonly`→`readOnly` mapping; otherwise identical. No native module, no prebuild.
14
14
-**Pi parity:**`packages/pi-plugin/` mirrors OpenCode semantics, importing shared core from `@magic-context/core`. Intentional divergences are tracked in `packages/pi-plugin/PARITY.md`. Provider prefixes are translated between canonical (OpenCode) and Pi configuration models via `src/shared/harness-provider-map.ts` at configuration read/write edges to keep shared model configurations portable. Pi implements session state inheritance on branch clone forks by copying filtered tags, compartments, and pending marker states, whereas OpenCode `/fork` does not yet inherit context state.
15
-
-**Rust module / subc integration.** A harness-agnostic Rust workspace in `crates/` re-implements the cache-stability transform and autonomous historian to run under the subconscious daemon (`subc`), communicating via the `subc` command and subscribe wire protocols.
15
+
-**Rust module / subc integration.** A harness-agnostic Rust workspace in `crates/` re-implements the cache-stability transform and autonomous historian to run under the subconscious daemon (`subc`), communicating via the `@cortexkit/subc-client` library on the TypeScript side and wire v2 protocols on the Rust side.
16
16
17
17
## Layers
18
18
19
19
-**Bootstrap** (`src/index.ts`): load config, register hidden agents + hooks + tools, start RPC server, dream-timer, auto-update checker; detect conflicting plugins (DCP / OMO / OpenCode auto-compaction) and disable the runtime if any is active.
-**Runtime** (`src/hooks/magic-context/`): the transform pipeline, postprocess phase, event/command handlers, system-prompt injection, compartment runners, decay rendering, strip-and-replay, nudges, m[0]/m[1] injection, and the shadow-transform sender (a flag-gated dev lane that mirrors transform passes to the Rust subc module).
21
+
-**Runtime** (`src/hooks/magic-context/`): the transform pipeline, postprocess phase, event/command handlers, system-prompt injection, compartment runners, decay rendering, strip-and-replay, nudges, m[0]/m[1] injection, and the shadow-transform sender (a flag-gated dev lane that mirrors transform passes to the Rust subc module using the `@cortexkit/subc-client` library).
22
22
-**Feature services** (`src/features/magic-context/`): storage, scheduler, tagger, memory, dreamer, sidekick, git-commit + message FTS indexes, unified search, overflow detection, session-project mapping and backfill, migrations, clone-state copy helpers (`src/features/magic-context/storage-clone.ts`), project identity resolution (resolves `git:<sha>` or fallback `dir:<md5-12>` identifiers, caching directory fallbacks, and utilizing a cooldown period for transient git errors).
23
23
-**Tools** (`src/tools/`): `ctx_reduce`, `ctx_expand`, `ctx_note`, `ctx_memory`, `ctx_search`. Gating: when `memory.enabled` is false, `ctx_memory` is not registered (on Pi, it registers but refuses queries for memory-off projects), and all memory-related guidance is stripped from the system prompt.
24
24
-**Config + shared** (`src/config/`, `src/shared/`): Zod config (deep-merge raw JSONC before validation; invalid leaves fall back to defaults with warnings, never disable the plugin). **Project-tier trust boundaries** (`src/config/project-security.ts`) strip unsafe fields from untrusted repository configs before merge — preventing escalation via `sqlite` PRAGMAs, hidden-agent reprogramming (`prompt`, `tools`), `embedding` destinations, or developer-only shadow transform settings. Project-level compaction thresholds are raise-only to prevent cloned repos from forcing extra historian cost. Also includes logger, data paths, SQLite selector, harness id, RPC transport, conflict detector, tag-transcript primitive (shared with Pi), provider-id translation map (`src/shared/harness-provider-map.ts`), and exit-abort listener coordinator (`src/shared/exit-abort-registry.ts`).
25
25
-**TUI** (`src/tui/`): sidebar + `/ctx-status` / `/ctx-recomp` dialogs, RPC-backed; shipped as raw TS via the `./tui` export (not bundled into `dist/index.js`).
26
26
-**CLI** (`packages/cli/`, separate `@cortexkit/magic-context` package): `npx` setup / doctor / migrate wizard, including local embedding runtime checks for missing or broken `onnxruntime-node` native bindings.
27
-
-**Rust Core & Store** (`crates/mc-core/`, `crates/mc-store/`): Re-implement cache-stability transform/classification logic and durable SQLite schemas with row-version CAS in Rust.
27
+
-**Rust Core & Store** (`crates/mc-core/`, `crates/mc-store/`): Re-implement cache-stability transform/classification logic and durable SQLite schemas with row-version CAS in Rust. It implements a durable `mc_reduce_command_ledger` table (via migration 16) to provide command-id idempotency for `ctx_reduce` agent-drop requests.
28
28
-**Rust Tokenizer** (`crates/mc-tokenizer/`): Tiktoken byte-BPE Claude token estimator.
29
-
- **Rust subc Module** (`crates/mc-module/`): Integrate with the `subc` daemon, handle transform requests, route MCP facade requests (for `ctx_memory`, `ctx_search`, `ctx_expand`, and `ctx_note`), and orchestrate the autonomous historian. Serve prompt guidance with support for a `"no_reduce"` variant on reduce-less surfaces. Expose co-owned profile and feature epochs (`PROFILE_EPOCH_CLAUDE_CODE_ANTHROPIC`, `TAGGER_FEATURE_EPOCH`) during status checks to assert compatibility, folding profile-specific serializer epochs to coordinate hard transitions on format changes. Maintain a durable pass trace (`mc_pass_trace` table in `mc-store`) to audit session receive/complete/reject events separately from cache state. Use harness model codecs to translate between harness-specific JSON (OpenCode, Pi) and canonical `CkWireMessage` values, reject client-reported context limits below `MIN_PLAUSIBLE_CONTEXT_LIMIT` (1024) by falling back to 200,000, and use serializer profiles to selectively disable tail mutations for byte-splice consumers (e.g. Claude Code) to prevent phantom reclaims while absorbing covered system-role messages into the `m0` baseline.
29
+
- **Rust subc Module** (`crates/mc-module/`): Integrate with the `subc` daemon, handle transform requests, route MCP facade requests (for `ctx_memory`, `ctx_search`, `ctx_expand`, and `ctx_note`), and orchestrate the autonomous historian. Serve prompt guidance with support for a `"no_reduce"` variant on reduce-less surfaces. Expose co-owned profile and feature epochs (`PROFILE_EPOCH_CLAUDE_CODE_ANTHROPIC`, `TAGGER_FEATURE_EPOCH`) during status checks to assert compatibility, folding profile-specific serializer epochs to coordinate hard transitions on format changes. Maintain a durable pass trace (`mc_pass_trace` table in `mc-store`) to audit session receive/complete/reject events separately from cache state. Use harness model codecs to translate between harness-specific JSON (OpenCode, Pi) and canonical `CkWireMessage` values, reject client-reported context limits below `MIN_PLAUSIBLE_CONTEXT_LIMIT` (1024) by falling back to 200,000, and use serializer profiles to selectively disable tail mutations for byte-splice consumers (e.g. Claude Code) to prevent phantom reclaims while absorbing covered system-role messages into the `m0` baseline. It supports `agent_drops.append` for queuing tag drops, with support for server-side canonicalization of raw range/list drop strings and command-id idempotency checks, and enforces epoch-aware route channel tracking (via `OpenedRoute` / `RouteHandle`) to reject stale route executions under the wire v2 protocol.
30
30
31
31
<!-- mc:protected START — hand-authored cache-stability core. The dreamer's maintain-docs task MUST NOT edit, reword, reorder, trim, or drop anything between mc:protected START and mc:protected END; carry it forward byte-for-byte on any rewrite. Only a human edits this region, deliberately. -->
<imgalt="Star History Chart"src="https://api.star-history.com/chart?repos=cortexkit/magic-context&type=date&legend=bottom-right&sealed_token=8tgpBwlCfG4Vl1P1ink58E2ocwGiJZ7EfwQLdovNyFh8gqF6In4EhvfaIx8p_C6KztsJtU389KpouCB0HKrjDqN06WULjeHrc1uGhnHHpqDJHv_eAa2zWA" />
<imgalt="Star History Chart"src="https://api.star-history.com/chart?repos=cortexkit/magic-context&type=date&legend=bottom-right&sealed_token=8tgpBwlCfG4Vl1P1ink58E2ocwGiJZ7EfwQLdovNyFh8gqF6In4EhvfaIx8p_C6KztsJtU389KpouCB0HKrjDqN06WULjeHrc1uGhnHHpqDJHv_eAa2zWA" />
<imgalt="Star History Chart"src="https://api.star-history.com/chart?repos=cortexkit/magic-context&type=date&legend=bottom-right&sealed_token=8tgpBwlCfG4Vl1P1ink58E2ocwGiJZ7EfwQLdovNyFh8gqF6In4EhvfaIx8p_C6KztsJtU389KpouCB0HKrjDqN06WULjeHrc1uGhnHHpqDJHv_eAa2zWA" />
0 commit comments