Body links get the /magic-context base via a remark plugin so authors
keep writing site-absolute paths; hero frontmatter links carry the base
explicitly. Discord invite corrected to the canonical README one.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

The flat VERBATIM_PROBE_BONUS=0.5 sat 30x above the RRF scale (max list
contribution 1/60), so every verbatim hit saturated; divide-by-max then
flattened all message scores into a ~0.95-1.0 band, and at the unified
layer those ~1.0 scores x MESSAGE_SOURCE_BOOST crowded memories out of
the results entirely.

Verbatim containment is now worth one rank-0 list appearance (1/RRF_K)
of the best matching probe, probe lists are weighted by a smooth
document-frequency falloff (a probe matching 2% of the corpus carries a
third of a rare probe's signal - the 'AFT' acronym-flood case), and
fused results map onto the same linear 0..1 band the single-query path
emits so cross-source scales stay comparable.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

The shared DB can be migrated to v34 by a plugin process while the
dashboard stays open; caching 'false' permanently would hide workspaces
until an app restart. Only the true verdict is cached (schemas never
un-migrate); not-ready re-probes on each call.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

The create row and workspace list added their own 20px inline padding on
top of .scroll-area's 0 20px, indenting content 40px relative to every
other page's content edge.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…dings superseded it

Chunk embeddings (per-compartment raw [ordinal] U:/A: vectors) became
ctx_search's semantic substrate, leaving p1_embedding (the per-compartment
SUMMARY vector) with zero readers — no SELECT of p1_embedding exists in
either package. It was still computed on every historian publish and
re-computed by four recomp/upgrade blocks, a second embedding call per
publish that hammered local endpoints for nothing.

Removed embedAndStoreCompartments and all five call sites (incremental
publish, full/partial recomp, Pi historian). The p1_embedding column is
left inert (no migration mid-cycle); dreamer v2 decides whether to
repopulate it for cross-compartment linking or drop it.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…am timer

A bounded chunk-embedding batch per 15-min tick is a slow, bursty trickle
that repeatedly hammers local embedding endpoints (LM Studio etc.) while
never finishing a large historical corpus quickly. New compartments still
embed on publish; historical backfill moves to an on-demand command
(/ctx-embed-history) so the user controls when the endpoint is hit. The
backfill helpers stay exported for that command.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…ddings on demand

Replaces the timer-driven trickle (removed in the prior commit) with a
user-initiated command that embeds ALL of one session's un-embedded
compartments in a single pass, oldest-first. Reuses the recomp progress
surface (sidebar + status bar) with a new kind="embed" so OpenCode shows a
live N/M bar; Pi reports a single completion message (no progress sidebar).

Core: embedSessionCompartmentChunks (shared by both harnesses) runs under
the per-project embedding coordinator lease, yields between batches so the
multi-core MiniLM burst stays interruptible, and is idempotent/resumable
via chunk_hash. Session-scoped candidate loader + count added to
compartment-chunk-embedding; the candidate-embed core is factored out of
the project sweep so both paths share it.

For users with no external provider, the in-process local MiniLM model
embeds a large session in ~2 minutes; for external providers this gives
explicit control over when the endpoint is hit.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

Three operational risks in the unbounded session drain:

- Lease expiry mid-run: the run holds the coordinator lease with no per-run
  cap and could exceed the 5-min TTL, letting a sibling/passive sweep
  acquire the 'expired' lease and duplicate work. Added a renewal interval
  (cleared in finally).
- Unbounded provider payload: batchSize capped compartments, not windows, so
  one large compartment (or big batch) could build an enormous padded
  tensor / JSON body. Provider calls are now sub-batched by window count
  (MAX_WINDOWS_PER_EMBED_CALL=16); a compartment is never split across calls.
- Abort only between batches: threaded AbortSignal through
  embedCandidateChunkBatch into embedBatchForProject and re-checked before
  persist.

Also: the no-progress break could falsely return 'done' with embeddable
candidates still pending. Now distinguishes no-work skips (empty canonical
text / already-current — excluded from re-selection so they can't block the
oldest-first cursor) from provider failures, returning a new 'stalled'
outcome with a retry hint surfaced in both harness commands. Stale p1
comments in recomp updated.

+2 tests (window cap across compartments, provider-failure stall).

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…ilter leak, TOCTOU)

Three Oracle audits on the workspaces + dashboard work. Real fixes:

- HIGH (cache): mustMaterialize keyed the HARD memory gate only on the
  CURRENT pass's isWorkspaced, so a cached workspaced m[0] whose session
  left its workspace fell through to the integer-epoch compare and could
  keep rendering the stale union. Now compares the workspace fingerprint
  whenever EITHER cached or current is non-null (matches renderM1 soft-
  refresh and mustMaterializePi).
- HIGH (dashboard): an empty/zero-member workspace filter omitted the IN
  clause and surfaced ALL memories/stats under that workspace (bulk
  archive/delete hazard). A requested filter that resolves to zero paths
  now forces an empty result in both get_memories and get_memory_stats.
- MED (dashboard TOCTOU): workspace member fan-out for memory-status
  changes was resolved BEFORE BEGIN IMMEDIATE; a concurrent add-member
  could leave a new member's epoch un-bumped. Moved the resolution inside
  the write transaction (single + bulk paths).
- LOW: rename_workspace bumped every member epoch though the workspace
  name is never rendered (spurious hard fold) — removed. ctx_memory
  archive now de-dupes ids (both harnesses) so [42,42] queues one row.

Accepted-by-design (documented A41-A44): dashboard archive rides the
supersede-delta not an epoch bump; foreign-member embeddings aren't
backfilled from a member session; m[1] new-memories uses the flat trim;
ctx_search RRF band parked pending message-embedding recall.

Tests: workspace→single fold regression, empty-workspace zero-result,
rename no-fold. Plugin 1997, Pi 441, dashboard 118.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…y on github-copilot

stripStructuralNoise/stripClearedReasoning/stripProcessedImages/
dropStaleReduceCalls replace parts with empty-text sentinels assuming
OpenCode drops them before the wire. OpenCode only does that for
@ai-sdk/anthropic + @ai-sdk/amazon-bedrock — github-copilot
(@ai-sdk/github-copilot) forwards the empty-text part as a real content
block. When it lands after a tool_use (e.g. a step-finish that natively
converts to nothing) it breaks Anthropic's tool_use/tool_result adjacency
after copilot's server-side Bedrock re-translation: 400 on a FRESH session.

Gate all four empty-sentinel producers on modelAcceptsEmptyContent
(anthropic-only). Resolve the provider ONCE (reusing the budget path's
map+cold-only DB recovery — no new DB read) and share it across the main
transform AND postprocess so cold and hot passes make the same decision
(step-start is a message-boundary marker in AI SDK 6.0.168: native vs
sentinel diverge on the wire, so a map-only gate would bust a warm
anthropic cache on the first hot pass post-restart). The stale-reduce
gate covers BOTH frozen-id replay AND persistence so a prior anthropic
pass's persisted ids can't re-sentinelize on a later copilot pass. Pi has
no makeSentinel wire path (PARITY.md noted). Anthropic behavior byte-unchanged.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…ent drop starvation

In an active session the protected tail's newest message changes every
turn, so a boundary snapshot captured at trigger time fails validation on
the "last ordinal id changed" check by the time the historian runs — even
though the eligible head it would compact is untouched. The runner no-op'd
forever while the trigger refired each turn, queued drop ops accumulated
(observed: 27 consecutive stale no-ops, 179 pending drops, zero reduction).

- compartment-runner-incremental: on a stale_snapshot validation result,
  re-resolve the boundary once from current session state and adopt the
  fresh snapshot when it still exposes a runnable head. The refreshed
  snapshot recomputes protectedTailStart/eligibleEnd from live messages,
  so the protected-tail guarantee is preserved (the head can never include
  a message now in the live tail). Historian makes real progress, publishes,
  and queues drops so the accumulated pending ops drain.
- startCompartmentAgent: a synchronous runner no-op cleared compartmentInProgress
  but left the activeRuns registration alive until a microtask, so the same
  transform pass deferred queued drop ops for a run that already finished.
  Signal onHistorianRunStarted at the real-run commit point and drop the
  registration synchronously when the runner no-ops.
- Regression tests: stale-tail snapshot re-derives + publishes instead of
  no-op'ing; synchronous no-op clears the active-run belief in the same pass.

Co-authored-by: tracycam <gtracycam@gmail.com>
Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…rce attribution

Two Pi-only parity gaps from a workspaces council audit:

- cachedPiRowMatchesSnapshot compared the session's own projectMemoryEpoch but
  NOT the workspace fingerprint, unlike OpenCode's cachedRowMatchesState. A
  FOREIGN workspace member's epoch bump changes the fingerprint without touching
  this session's epoch, so a sibling-materialized row under different membership
  could pass the CAS and be adopted with the wrong union baseline. Added the
  fingerprint compare (both sides already carry it in markers).
- Pi ctx_search memory line omitted the source= attribution OpenCode emits, so a
  foreign member's memory wasn't labeled with its origin project.

Verified mustMaterializePi already checks the fingerprint symmetrically (so a
membership change is a correct HARD fold, not a soft-refresh race — the council's
HIGH #2 is not exploitable). Documented the two post-ship edge cases (v22 rekey
fan-out, epoch-0 fingerprint collision) as A45/A46. Pi 441/0.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…, dry-run

Four problems from the cortexkit#144 report, all in the setup wizard:

- ROOT CAUSE of 'preselected list, none available for me': pi-helpers
  parseModelListOutput required each token to contain '/', but
  `pi --list-models` prints a fixed-width TABLE (provider + model in
  separate columns, no slash). It matched ZERO rows and fell back to a
  hardcoded STATIC_MODELS list the user didn't have. Rewrote the parser to
  join the provider/model columns (skips header, keeps forward-compat with
  a pre-joined first token). Verified: 0 -> 22 real models live.
- Removed the per-role recommendation tree (buildModelSelection's hardcoded
  'recommended' ids) in BOTH harnesses; replaced with a shared model-picker
  that shows the user's FULL sorted catalog.
- 'type the model name to pick': new selectAutocomplete (clack autocomplete)
  — a scrollable visible list you also narrow by typing. Used for every
  model pick. Each role first shows a short explanation of what it does and
  that it does NOT need a frontier model (smaller/cheaper is fine).
- Pi dreamer-after-no bug: chooseModel ran unconditionally even when the
  user declined the dreamer. Now gated on dreamerEnabled (matches OpenCode).

Plus: --dry-run for setup (both harnesses) — runs the full interactive
wizard but writes no files / registers no package, printing
'[dry-run] would write …'. Verified both flows end-to-end via PTY.

Tests: pi-helpers table-parser suite, dreamer-decline regression, updated
setup-pi expectations for the no-recommendation-tree behavior. CLI 151/0.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

A content change to ARCHITECTURE.md/STRUCTURE.md was a HARD mustMaterialize
trigger, so any on-disk docs change folded m[0] and busted the entire cached
prefix. In production this fired every time the dreamer's maintain-docs task
rewrote the docs — a real m[0] bust for every active session on that project
(confirmed live: cached prefix 599,885B → 28,187B, reason=project_docs_hash).

Project docs are slowly-changing reference material living in m[0]; they now
follow the 'Deliberately NOT triggers' pattern (like new compartments /
additive memories): ride along and fold into m[0] on the NEXT natural hard
bust, never force one. Removed projectDocsHash as a DECISION input at 6 sites
(both harnesses): mustMaterialize, the m[1] Phase-3 contention stale-check, and
the sibling CAS. KEPT computed-at-fold + stored, so a natural fold always reads
fresh docs (readProjectDocsCanonical) and persists the hash matching the bytes
it rendered. CAS removal is safe because the byte compare runs first — a
byte-different m[0] still rejects; only docs-hash-only drift with identical
bytes now matches.

Also fixes analyze-cache-busts.ts: verdict now compares the cached prefix
against the PREVIOUS request's last breakpoint (the old check used the current
request's moved-tail breakpoint and stamped normal tail growth as BUST);
timestamps include the date (multi-day dump sets were ambiguous).

Plugin 2010/0, Pi 445/0.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

Co-authored-by: Ren yiwei <85666259+SSDWGG@users.noreply.github.com>
Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

A workspace governs WHICH memory categories its members share to each other.
A member session renders its OWN project's memories in all 5 categories, but
FOREIGN members' memories only in the workspace's share_categories (new
workspaces.share_categories JSON column, default ["CONSTRAINTS"]). [] = foreign
share nothing; absent column (pre-v35) = all.

Cache-stability core (m[0]/m[1]):
- One shared SQL predicate (buildWorkspaceMemorySqlFilter) on ALL union readers
  — baseline getMemoriesByProjects, m[1] delta readNewMemoriesForM1Union,
  watermark getMaxMemoryIdForProjects, AND the FTS union — at EVERY call site
  incl. both OpenCode marker sites (snapshot + Phase-3) and all Pi marker/render
  sites, so a hidden foreign category can't render in one path while advancing
  another path's cursor.
- ownIdentities = anchor + its v22 aliases (not just [projectIdentity]), so an
  own-project legacy-alias memory is never wrongly category-filtered.
- A share_categories change folds m[0] exactly once: hashed into
  computeWorkspaceEpochFingerprint (null -> "ALL"), flowing into mustMaterialize
  + CAS on both harnesses, then byte-stable.
- No-workspace sessions are a provable no-op (single-identity fast paths
  unchanged). resolveWorkspaceShareCategories fails open to null on parse error.

Schema: migration v35 (LATEST_SUPPORTED_VERSION 35) — CREATE TABLE IF NOT EXISTS
(fresh-safe) + columnExists-guarded ADD COLUMN (upgrade-safe) + NULL backfill +
one-tx epoch fan-out over existing workspace members; fresh DDL + ensureColumn
carry the default. Pi parity. plugin 2015/0, Pi 446/0.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

Replaces per-action immediate writes (each ran its own epoch fan-out -> one
m[0] fold per click) with a staged editor + one batch command.

- New apply_workspace_changes Tauri command: one IMMEDIATE tx that builds the
  FINAL member map in memory (old union adds minus removes), validates display-
  name uniqueness / membership against THAT final state (so 'remove A, add B
  reusing A's freed name' passes), and runs CONDITIONAL epoch fan-out over
  old union new ONLY when membership, member display-names, or share_categories
  changed — rename-only / no-op Saves bump no epochs. Removes the old
  add/remove/set-name single-action commands (UI was their only caller).
- 5 'Shared categories' checkboxes (default CONSTRAINTS) + share_categories
  written as a canonical JSON array matching the plugin's V2 category set.
- Readiness bumped to v35 + share_categories column check; the card hides
  entirely pre-column (never writes NULL sharing). Degrades, never migrates.
- Staged-editor card uses <Index> (not <For>) for member rows + checkboxes to
  avoid the prior mid-edit reactivity bug; Save gated on dirty, Discard resets.

cargo check + 29 Rust tests + staging unit tests + bun check all green.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

A Pi session overflowed (Codex context_length_exceeded) while MC reported 68%.
Root cause: Pi sources pressure from lastContextPercentage persisted at
message_end (end of the whole turn), which stays FROZEN during a long
multi-step tool-heavy turn while the assembled request balloons past the
window — so nothing trips the execute/85%/95% thresholds and the request
overflows. OpenCode never hits this: its message.updated fires per-step at
step-finish (confirmed against OpenCode source), so its pressure climbs
mid-turn and it sheds before overflowing.

Fix (Pi-only): floor pressure with ctx.getContextUsage().tokens, which is
FORWARD (last assistant usage + estimateTokens of every trailing message,
recomputed from the live array each pass) — approximating OpenCode's per-step
climb. Shared applyForwardPressureFloor helper applied in BOTH the scheduler
pressure block AND maybeFireHistorian (the historian trigger independently
re-read the same stale persisted pressure, so it needed the floor too). Keeps
only .tokens (forward, input-side); .percent is discarded (counts output on
Pi's denominator). Scales the LIMIT (×0.85 FORWARD_PRESSURE_LIMIT_FACTOR) for
the forward percentage to compensate estimate-token undercount — does NOT
mutate the real limit (history-budget/emergency-ceiling rely on it) and passes
the raw forward tokens onward (emergency drop needs the true assembled size).
max() so it never lowers; null/missing forward usage = today's behavior.
Emergency-recovery bump changed to Math.max(_,95) so it can't cap a higher
forward reading.

Immune to the NULL token_count tag undercount (uses the live message array,
not the tag store). Emergency drops stay cache-stable: same-sample force
passes are latched, fresh same-turn growth re-triggers (intended), no-candidate
force leaves wire bytes unchanged — all locked by tests. PARITY.md documents
the divergence. Pi 455/0, plugin 2015/0 (OpenCode untouched).

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…version-manager shims

The Pi config page showed empty model comboboxes for users whose pi (or
opencode) binary is installed by a version manager (mise/nvm/fnm/volta/asdf):
those write the binary under a per-version dir (e.g.
~/.local/share/mise/installs/node/VER/bin/pi) that the dashboard's hardcoded
candidate paths can't enumerate, and the Tauri GUI doesn't inherit the shell
PATH so the bare "pi"/"opencode" candidate also fails. get_available_pi_models
then returned [] -> empty dropdowns.

The table parser and the App.tsx -> getAvailablePiModels -> ConfigEditor wiring
were both correct; only binary discovery failed. Fix:
- Add version-manager shim dirs (mise/asdf/volta) to the candidate list.
- Add a login-shell fallback (run_via_login_shell): when the candidates miss,
  run the tool through the user's login shell, which resolves it exactly as the
  user's terminal does. Bounded by an 8s timeout so a slow shell rc can't hang
  the dropdown; unix-only (gated cfg unix); skipped when SHELL is unset.
- Treat an empty parse from a found binary as "keep trying" so a stub on PATH
  doesn't shadow a working install.

Applied to BOTH pi and opencode discovery (same latent bug). Verified E2E that
the login-shell command in a GUI-like stripped env resolves the mise pi and
returns the model table. cargo check + 46 Rust tests green.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

- Shared-categories checkboxes used raw browser defaults that clashed with the
  dark theme and aligned poorly with their labels. Reuse the existing custom
  checkbox styling (.tri-checkbox / .memory-card-checkbox) by extending its
  selectors to .share-category-label inputs, and add a flex label class for
  proper checkbox-to-text alignment.
- Save changes / Discard are now HIDDEN until there are unsaved edits (were
  rendered disabled-but-looking-active). This also disambiguates Discard from
  Delete: Discard only appears next to Save when there are unsaved edits to
  revert, while Delete (remove the workspace) is always shown — the two no
  longer sit together at rest.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

A processed-screenshot user message lost its image blocks on a DEFER pass,
collapsing the Anthropic cached prefix mid-conversation (verified on the wire:
message[48]'s two base64 PNGs present at 18:58:11, gone at the 18:58:31 defer
pass → cachedPrefix dropped to that message).

Root cause: stripProcessedImages sentinelized images whenever a message's
maxTag crossed the live watermark. The empty sentinel is filtered off the
Anthropic wire, so that FIRST strip removes the image blocks — a real byte
change. Keying it on the live watermark let tail growth advance the boundary
across an older image message on a defer pass (which must replay
byte-identically), busting the cache.

Fix mirrors the adjacent dropStaleReduceCalls frozen-id pattern exactly:
- DETECT (cache-busting passes only): find newly-aged processed-image messages,
  strip them, return their ids.
- REPLAY (every pass incl. defer): re-strip only already-frozen ids,
  byte-identical regardless of how the live array grew.
- Frozen id set persisted in session_meta.processed_image_stripped_ids via
  CAS-merge helpers (sibling-process safe), cleared with the session row.
OpenCode-only (Pi intentionally does not strip images). +1 regression test
asserting a defer pass never first-strips an aged image; existing defer/replay
integration tests updated to the freeze-then-replay contract. plugin 2016/0.

Also harden analyze-cache-busts.ts: default to printing ONLY bust rows (+ a
bust-count summary), so a real prefix bust is never buried under ordinary
tail-growth STABLE rows; --all-rows restores the full table.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…ode)

When m[0] is going to HARD-fold this pass (epoch / model / system-hash /
ttl-idle / mutation-id / upgrade — whatever mustMaterialize decides), the
Anthropic prefix is being re-cached regardless. Previously the drop decision
(shouldApplyPendingOps / shouldRunHeuristics, computed at L236/L265) didn't know
the pass was about to bust m[0] (discovered later inside injectM0M1), so a
hard fold landing on a sub-threshold defer pass busted the whole prefix while
reclaiming NOTHING, then a later execute pass busted AGAIN to drain the queued
tool drops. Two busts where one would do (wire-confirmed: workspace-delete epoch
fold re-billed 611K tokens, then +83s a 2068-op drop pass busted again).

Fix: compute the existing mustMaterialize ONCE early (advisory) →
m0HardFoldThisPass, and OR it into shouldReadPendingOps / shouldApplyPendingOps /
shouldRunHeuristics so a known-bust pass drains the drops into the SAME bust.

Council-reviewed (bg_0f5c5832); four corrections folded:
- ADVISORY-ONLY: the early call only WIDENS the gates. injectM0M1 keeps its own
  independent late mustMaterialize recheck — early-false never suppresses it, so a
  cross-process (dashboard) epoch/mutation bump arriving after the early read still
  folds. Correctness is never worse than today; cost is one extra mustMaterialize
  (indexed reads + a cached docs stat).
- SEPARATE OR-TERM: never folded into materializationRequested (which drives the
  lastResponseTime TTL reset + pendingMaterialization cleanup) — folding in would
  suppress those and oscillate.
- Reuses mustMaterialize itself, so the HARD trigger set can't drift (project_docs_hash
  is correctly NOT a trigger).
- Respects the existing compartment-run gate (no new bypass): a hard fold while a
  historian runs at <85% defers the drain to the next pass — still ≤ today.

+2 integration tests: a model-key change on a DEFER scheduler pass drains a queued
drop; an unchanged-marker defer pass leaves it queued. plugin 2018/0.

Pi parity follows in a separate commit. Design doc:
.alfonso/plans/cache-fold-and-image-strip-review.md

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…rity)

Pi mirror of 4ce3d73. When Pi's m[0] will HARD-fold this pass (mustMaterializePi:
model / system-hash / ttl-idle / project-memory epoch / mutation-id / upgrade),
the Anthropic prefix re-caches regardless, so drain the queued tool drops + run
heuristics into THAT bust instead of busting again on a later execute pass.

Hoisted the piHardSignals block above the exec gates; computes an advisory
m0HardFoldThisPass via an early mustMaterializePi (gated on args.injection, its
own getCompartments snapshot). injectM0M1Pi keeps its OWN authoritative late
mustMaterializePi recheck unchanged — early-false never suppresses it (a
cross-process epoch/mutation bump after the early read still folds late). The
late injectM0M1Pi reuses the SAME hoisted piHardSignals const, so the advisory
and authoritative hard signals are byte-identical. m0HardFoldThisPass is a
separate OR-term into shouldRunHeuristics + baseShouldApplyPendingOps only —
never folded into the deferred/explicit materialization signals; deferred-drain
and compaction-marker ordering untouched. Full parity, no PARITY.md divergence.

+2 Pi integration tests (model-key change on a DEFER pass drains a queued drop;
unchanged-marker defer leaves it queued). Pi 457/0, plugin 2018/0.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…A + workspaces)

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…-shared categories

The memory-mutation tools (update/archive/merge) gated visibility on workspace
MEMBERSHIP only, never on the share-category policy. So an agent in project A
could update/archive a memory belonging to foreign member B in a category B does
NOT share with the workspace — a memory the agent cannot even SEE in its
rendered <project-memory> (the render path filters foreign memories by
shareCategories, the tool did not). Tool visibility now mirrors render
visibility exactly: own-project memories are mutable in every category; foreign
member memories only when shared (shareCategories===null shares all, [] shares
none, otherwise only the listed categories) — the same own/foreign split as
buildWorkspaceMemorySqlFilter. Both harnesses. +3 regression tests (foreign
non-shared category refused, foreign shared allowed, own always allowed).
plugin 38/0 in the ctx-memory suite.

Co-authored-by: Alfonso [Magic Context] <288211368+alfonso-magic-context@users.noreply.github.com>

…r-config-only

Engine-agnostic external memory config (provider/endpoint/banks/retain_sources/
tags) + recall sub-block (timeout, per-slice budget, dedup threshold,
global_tags, global_from_prompt, search, mental models). Whole block stripped
from project-level config (a hostile repo must not redirect a user's memory
store). Schema asset regenerated.

Neutral backend interface (retain/recall/remove/mentalModels) with a Hindsight
implementation: bank routing (per-project banks + main bank), idempotent
content-hash document_ids, circuit breaker, SSRF guard, 422 no-retry,
benign-404 read paths, mental-model seeding after first retain. All three
durable creation points tee fire-and-forget (historian promotion, ctx_memory
write, dreamer user-memory promotion); origin provenance on global-scope items
(origin-* tags + extraction-context entity link). Never throws, never blocks a
local write.

…er, not memory.enabled

Compartment embeddings are the ctx_search substrate, not memory-store data.
embedding-on/memory-off users now get search embeddings + project registration;
promotion stays gated by memory flags (issue cortexkit#44 preserved).

…safe m[0]/m[1] injection

Once-per-session 3-slice recall (project/profile/global; mental-model fast
path; optional first-prompt-enriched global query), deduped against local rows
(cosine + hash fallback), trimmed, sorted, frozen into session_meta (migration
v33 — renumbered from pre-v0.23 v31). Rendered as <external-memory> in m[0]
with marker-hash m[1] delta for late arrivals; NOT a mustMaterialize trigger;
excluded from the m[1] pressure-refold math; A-path await bounded and
first-render-only. Defer passes replay frozen bytes.

…e writes

Local memory truth signals propagate to the external store: archive (incl.
v0.23.0 batch archive — one batched DELETE per call) removes documents by
original-content document_id; update removes the stale doc and tees the
corrected fact; NEW dreamer-only verify action records local verification and
verbatim-upserts (refreshes engine recency, verifiedAt metadata); user-memory
dismiss/update propagate so the profile recall slice cannot resurrect dismissed
memories. merge and memory-migration deletes deliberately propagate NOTHING.
ctx_memory write gains scope="global": main-bank-only tee (no local row) with
origin-project provenance. Removes/upserts bypass retain_sources; the update
re-tee respects it.

Provider/endpoint/circuit-state/session-recall-state plus a best-effort
failed-retain count from the operations endpoint (benign-404, breaker-safe,
null on any failure). executeStatus goes async; both harness call paths await
it.

…for read tools

session.created parentID registrations (bounded LRU, cycle-safe walk);
ctx_search and ctx_expand resolve child sessions (sidekick, task subagents) to
the ROOT conversation so message-history search, the visible-memory filter,
and the injected-external-recall filter target the session that actually owns
that state. Mutating tools stay child-scoped. Sidekick registers synchronously
at spawn (event-race belt-and-braces).

Renders the StatusDetail.externalMemory payload that buildStatusDetail
already ships over RPC: provider, endpoint, circuit-breaker state (warning
color when not closed), this session's recall state, and the server-side
failed-retain count. TUI parity with the text-mode /ctx-status section.

isMagicContextEntry only matched the npm package name
(opencode-magic-context), so a dev entry like
file:///…/magic-context/packages/plugin was invisible to the detector
and ensureTuiPluginEntry re-appended @latest on every startup,
double-loading the plugin. Match magic-context as a whole path
segment too.

When opencode runs inside the user config directory itself
(~/.config/opencode as project root), project-config discovery
resolves to the same magic-context.jsonc as the user config. It was
loaded a second time as untrusted repo config, so the project
security strips fired against the user's own file: {file:} tokens
left literal and memory.external dropped — silently disabling the
external memory backend. Skip the project load when its resolved
path equals the user config path.

…sh in background

…gration fence, search merge

…on test

Previous filter admitted ALL stored vectors when queryModelId was falsy
or 'off', producing meaningless cosine scores across different embedding
spaces. Fix: only populate localVectors when queryModelId is known and
not 'off'; otherwise fall back to hash-only dedup.

Regression tests: hash-dedup baseline (integration), model-guard unit
tests for off/empty queryModelId and known queryModelId with two stored
model IDs.

…al parity

- index.ts: call initializeExternalMemory(config.memory?.external) at
  startup (mirrors OpenCode). Config flows through the shared
  MagicContextConfig schema which already includes memory.external.

- ctx-memory.ts: mirror OpenCode's external wiring:
  - write (project scope): tee to external backend after local insert
  - write (global scope): external-only write, no local row; gated on
    getExternalMemoryStatus()
  - update: W2 corrective propagation (remove old + tee new)
  - archive: removeFromExternalBackend for all archived memories
  - verify (dreamer-only): upsertToExternalBackend to refresh recency
  - scope param added to schema (project/global)
  - verify added to ALL_ACTIONS and DREAMER_ONLY_ACTIONS

All external calls are fire-and-forget (void), never blocking local
mutations. memoryVisibleToTool gate respected (external calls go after
local mutation, same as OpenCode).

…rch source

context-handler.ts:
- Fire startSessionRecall on first context pass per session (mirrors
  OpenCode transform.ts loadedSessions gate). Extracts first user
  prompt text for global_from_prompt enrichment.
- Call maybeAwaitExternalRecall before injectM0M1Pi when injection is
  enabled (A-path await: waits only when no cached m[0] exists, bounded
  by recall.timeout_ms). Mirrors OpenCode transform.ts timing.

inject-compartments-pi.ts:
- Add externalRecallHash to PiM0SnapshotMarkers (NOT a mustMaterializePi
  trigger — rides m[1] external delta only, invariant preserved).
- Add cached_m0_external_recall_hash to CachedPiM0M1Row + SQL SELECT.
- readFrozenM0InputsPi: read external recall snapshot in-transaction
  (TOCTOU-safe, same as projectDocsHash override in OpenCode).
- renderM0Pi: accept externalRecallOverride param; merge profile slice
  into user-profile; render renderExternalMemoryBlock after project-memory.
- renderUserProfileBlock: accept externalProfileLines param for profile
  slice merge (mirrors OpenCode renderUserProfileBlock signature).
- materializeM0Pi: pass frozen.externalRecall to renderM0Pi; persist
  externalRecallHash in persistCachedM0.
- renderM1PiWithMetadata: render external memory delta when live hash
  differs from m[0] baseline hash.
- markersFromCachedPiRow: populate externalRecallHash from DB row.
- readCurrentMarkersFromCompartments: read live externalRecallHash.

ctx-search.ts: already wired (explicitSearch + projectName + external
source in schema + formatResult). No changes needed.

Biome import ordering and formatting fixes for:
- pi-plugin/src/context-handler.ts (import order + indentation)
- pi-plugin/src/index.ts (import order)
- pi-plugin/src/inject-compartments-pi.ts (import order + SQL formatting)
- pi-plugin/src/tools/ctx-memory.ts (indentation)
- plugin/src/.../external-recall.test.ts (import order + arrow fn)

…, STRUCTURE, CONFIGURATION, README)

Adds the external-memory-backend feature to root docs:

- ARCHITECTURE.md: new 'External-memory flow' subsection under Data Flow
  (tee-on-write → 3-slice recall + dedup + persist-before-settle → m[1]
  <external-memory> render → W2 correctives → ctx_search external source);
  new 'External memory backend (Hindsight)' Key Abstraction; cached_m0_
  external_recall_hash marker added to the schema-migrations note
  (deliberately NOT a mustMaterialize trigger); project-config trust
  boundary updated to include memory.external in the user-only strip
  list; unified search entry updated with the external source.

- STRUCTURE.md: new external-memory files listed under src/features/
  plus a Key File Locations block (engine-agnostic provider,
  Hindsight impl, recall orchestrator, snapshot read, W2 hooks,
  project-security strip) and the Pi mirror path
  (@magic-context/core re-export).

- CONFIGURATION.md: full memory.external block (provider, endpoint,
  api_key, project_bank template, main_bank, retain_sources, tags,
  recall sub-block) and the memory.external.recall defaults; full
  failure-semantics + cross-harness note; user-config-only security
  callout; example updated.

- README.md: concise mention in the Recall section + a new callout
  for long-term memory; external added to the four-source ctx_search
  description and the agent-tools table.

…, not external-memory block

Three-layer fix for the recurring 'failed to load plugin
@cortexkit/opencode-magic-context error="database is locked"' errors
that crash opencode startup when multiple processes cold-open context.db
concurrently.

Root cause: during a concurrent cold open (two opencode processes
booting, or a child session spawning while the parent is mid-checkpoint),
the WAL writer lock is held by a sibling. The first read in openDatabase()
— enforceSchemaFence() -> getPersistedSchemaVersion() — ran BEFORE
busy_timeout was installed (it was set later inside initializeDatabase()),
so SQLite returned SQLITE_BUSY immediately instead of waiting.

Layer 1 (storage-db.ts): Set PRAGMA busy_timeout=5000 immediately after
new Database(), before enforceSchemaFence()'s schema_migrations read. This
was the actual race window.

Layer 2 (dream-timer.ts): openTimerDatabaseOrNull() only handled
openDatabase()'s null return (schema-fence path) but NOT its throw path
(fatal open error). The throw propagated through
startDreamScheduleTimer() -> plugin() -> opencode's plugin loader, which
logged 'failed to load plugin' and disabled magic-context for the entire
session. Now wrapped in try/catch.

Layer 3 (index.ts): Defensive try/catch around the
startDreamScheduleTimer() await in the plugin entry, so even if a future
code path inside the timer setup throws, plugin() survives and the rest
of magic-context (hooks, tools, RPC) still initializes.

…USY retry

Replaces 9 hand-rolled BEGIN IMMEDIATE / COMMIT / finally-ROLLBACK blocks
with a single shared runWriteTransaction helper that adds a bounded
SQLITE_BUSY retry loop (4 attempts, 100ms backoff). This is a better
concurrency solution than the previous per-site pattern because:

1. Centralized retry policy: transient SQLITE_BUSY from cross-process WAL
   writer lock contention (sibling migration, dreamer run, Channel-2 bulk
   delivery) now waits-and-retries instead of propagating up and disabling
   Magic Context for the run. Previously each site either swallowed,
   propagated, or retried inconsistently.

2. Composition-safe: if called inside an existing db.transaction() or
   another runWriteTransaction, the body runs inline WITHOUT issuing a
   nested BEGIN (SQLite doesn't allow nested BEGIN; the outer transaction
   already holds the writer lock). Detected via the bun:sqlite
   inTransaction flag or the node:sqlite shim's isTransaction flag.

3. One correct transaction plumbing pattern instead of 9 slightly-different
   copies. Several sites were missing the ROLLBACK path or had subtle
   ordering bugs at the edges.

Sites converted (all sync write paths with hand-rolled BEGIN IMMEDIATE):
- dreamer/lease.ts (acquireLease, renewLease, releaseLease)
- git-commits/sweep-coordinator.ts (acquireGitSweepLease, renewGitSweepLease,
  markGitSweepSuccessAndRelease)
- message-index.ts (indexMessagesAfterOrdinal — cross-process FTS dedup)
- workspaces.ts (bumpEpochsForWorkspaceMembers, bumpEpochsForWorkspaceMemberSet)
- compartment-storage.ts (replaceAllCompartmentStateAndBumpDepth,
  promoteRecompStaging)
- key-files/project-key-files.ts (replaceAllKeyFiles)
- key-files/identify-key-files.ts (commitKeyFilesUnderLease)
- hooks/compartment-runner-recomp.ts (promoteRecompStagingWithM0Mutation)
- hooks/compartment-runner-incremental.ts (historian publish path)

inject-compartments.ts is intentionally NOT converted: its two BEGIN IMMEDIATE
blocks have complex early-return-with-rollback-and-fallback-read patterns
(Phase 3 materialization contention retry, soft-refresh cache replay) that
don't map cleanly onto the shared helper without significant restructuring.
The helper is still a net win across the 9 simpler sites.

Verification:
- bun run typecheck (tsc --noEmit + scripts) passes
- bun run lint (biome check) passes
- 12 new unit tests for runWriteTransaction/runWriteTransactionAsync pass
  (BEGIN/COMMIT, rollback on throw, transient BUSY retry, max-attempts
  give-up, nested-transaction composition, non-transient error passthrough)
- All existing tests in the converted files' test suites pass:
  dreamer/lease.test.ts (7), git-commits (19), compartment-storage-atomic
  (8), compartment-storage-v6, compartment-lease, compartment-runner-recomp-fk
  (4), compartment-runner-partial-recomp, key-files (14), workspaces,
  storage-db (11), message-index

Net: -199 lines / +40 lines across the 9 converted sites, +201 lines for
the shared helper + 156 lines for its tests.

…ifest

The external-memory-backend branch added the memory.external schema subtree
(provider/endpoint/banks/retain_sources/tags + recall sub-block) but did not
add a corresponding entry to the dashboard's config-field-coverage manifest.
The config-parity guard (config-parity.test.ts) enforces that every schema
leaf is either RENDERED by the ConfigEditor form or listed in
OMITTED_BY_DESIGN, so CI failed with 16 uncovered leaves.

memory.external is USER-config-only and operator-configured (no form widgets
exist for it), so the correct classification is OMITTED_BY_DESIGN with the
whole subtree covered by the 'memory.external' prefix.

The seedMemory helper opened context.db without setting PRAGMA busy_timeout,
so under the live opencode process's concurrent writer (historian checkpoint
or dreamer run) the seed INSERT could throw SQLITE_BUSY and fail the
'memory injection > injects <project-memory>' E2E test. This is the same
contention pattern fixed in the plugin itself; the test's external write
just wasn't opt-in to the retry window.

Set busy_timeout=5000 immediately after open, matching the pattern already
used by writeContextDb in cache-invariants.test.ts.