fix(storage): retry cold-open under SQLITE_BUSY instead of disabling plugin - #1
Closed
codeo1io wants to merge 24 commits into
Closed
fix(storage): retry cold-open under SQLITE_BUSY instead of disabling plugin#1codeo1io wants to merge 24 commits into
codeo1io wants to merge 24 commits into
Conversation
…r-config-only Engine-agnostic external memory config (provider/endpoint/banks/retain_sources/ tags) + recall sub-block (timeout, per-slice budget, dedup threshold, global_tags, global_from_prompt, search, mental models). Whole block stripped from project-level config (a hostile repo must not redirect a user's memory store). Schema asset regenerated.
Neutral backend interface (retain/recall/remove/mentalModels) with a Hindsight implementation: bank routing (per-project banks + main bank), idempotent content-hash document_ids, circuit breaker, SSRF guard, 422 no-retry, benign-404 read paths, mental-model seeding after first retain. All three durable creation points tee fire-and-forget (historian promotion, ctx_memory write, dreamer user-memory promotion); origin provenance on global-scope items (origin-* tags + extraction-context entity link). Never throws, never blocks a local write.
…er, not memory.enabled Compartment embeddings are the ctx_search substrate, not memory-store data. embedding-on/memory-off users now get search embeddings + project registration; promotion stays gated by memory flags (issue cortexkit#44 preserved).
…safe m[0]/m[1] injection Once-per-session 3-slice recall (project/profile/global; mental-model fast path; optional first-prompt-enriched global query), deduped against local rows (cosine + hash fallback), trimmed, sorted, frozen into session_meta (migration v33 — renumbered from pre-v0.23 v31). Rendered as <external-memory> in m[0] with marker-hash m[1] delta for late arrivals; NOT a mustMaterialize trigger; excluded from the m[1] pressure-refold math; A-path await bounded and first-render-only. Defer passes replay frozen bytes.
Project + main bank recall in parallel under a 5s bound, rank-scored below curated local memories, deduped against the session's injected <external-memory> snapshot. The auto-search hot path never reaches the network. Shared tool description documents the source for both harnesses.
…e writes Local memory truth signals propagate to the external store: archive (incl. v0.23.0 batch archive — one batched DELETE per call) removes documents by original-content document_id; update removes the stale doc and tees the corrected fact; NEW dreamer-only verify action records local verification and verbatim-upserts (refreshes engine recency, verifiedAt metadata); user-memory dismiss/update propagate so the profile recall slice cannot resurrect dismissed memories. merge and memory-migration deletes deliberately propagate NOTHING. ctx_memory write gains scope="global": main-bank-only tee (no local row) with origin-project provenance. Removes/upserts bypass retain_sources; the update re-tee respects it.
Provider/endpoint/circuit-state/session-recall-state plus a best-effort failed-retain count from the operations endpoint (benign-404, breaker-safe, null on any failure). executeStatus goes async; both harness call paths await it.
…for read tools session.created parentID registrations (bounded LRU, cycle-safe walk); ctx_search and ctx_expand resolve child sessions (sidekick, task subagents) to the ROOT conversation so message-history search, the visible-memory filter, and the injected-external-recall filter target the session that actually owns that state. Mutating tools stay child-scoped. Sidekick registers synchronously at spawn (event-race belt-and-braces).
Renders the StatusDetail.externalMemory payload that buildStatusDetail already ships over RPC: provider, endpoint, circuit-breaker state (warning color when not closed), this session's recall state, and the server-side failed-retain count. TUI parity with the text-mode /ctx-status section.
isMagicContextEntry only matched the npm package name (opencode-magic-context), so a dev entry like file:///…/magic-context/packages/plugin was invisible to the detector and ensureTuiPluginEntry re-appended @latest on every startup, double-loading the plugin. Match magic-context as a whole path segment too.
When opencode runs inside the user config directory itself
(~/.config/opencode as project root), project-config discovery
resolves to the same magic-context.jsonc as the user config. It was
loaded a second time as untrusted repo config, so the project
security strips fired against the user's own file: {file:} tokens
left literal and memory.external dropped — silently disabling the
external memory backend. Skip the project load when its resolved
path equals the user config path.
…gration fence, search merge
…on test Previous filter admitted ALL stored vectors when queryModelId was falsy or 'off', producing meaningless cosine scores across different embedding spaces. Fix: only populate localVectors when queryModelId is known and not 'off'; otherwise fall back to hash-only dedup. Regression tests: hash-dedup baseline (integration), model-guard unit tests for off/empty queryModelId and known queryModelId with two stored model IDs.
…al parity
- index.ts: call initializeExternalMemory(config.memory?.external) at
startup (mirrors OpenCode). Config flows through the shared
MagicContextConfig schema which already includes memory.external.
- ctx-memory.ts: mirror OpenCode's external wiring:
- write (project scope): tee to external backend after local insert
- write (global scope): external-only write, no local row; gated on
getExternalMemoryStatus()
- update: W2 corrective propagation (remove old + tee new)
- archive: removeFromExternalBackend for all archived memories
- verify (dreamer-only): upsertToExternalBackend to refresh recency
- scope param added to schema (project/global)
- verify added to ALL_ACTIONS and DREAMER_ONLY_ACTIONS
All external calls are fire-and-forget (void), never blocking local
mutations. memoryVisibleToTool gate respected (external calls go after
local mutation, same as OpenCode).
…rch source context-handler.ts: - Fire startSessionRecall on first context pass per session (mirrors OpenCode transform.ts loadedSessions gate). Extracts first user prompt text for global_from_prompt enrichment. - Call maybeAwaitExternalRecall before injectM0M1Pi when injection is enabled (A-path await: waits only when no cached m[0] exists, bounded by recall.timeout_ms). Mirrors OpenCode transform.ts timing. inject-compartments-pi.ts: - Add externalRecallHash to PiM0SnapshotMarkers (NOT a mustMaterializePi trigger — rides m[1] external delta only, invariant preserved). - Add cached_m0_external_recall_hash to CachedPiM0M1Row + SQL SELECT. - readFrozenM0InputsPi: read external recall snapshot in-transaction (TOCTOU-safe, same as projectDocsHash override in OpenCode). - renderM0Pi: accept externalRecallOverride param; merge profile slice into user-profile; render renderExternalMemoryBlock after project-memory. - renderUserProfileBlock: accept externalProfileLines param for profile slice merge (mirrors OpenCode renderUserProfileBlock signature). - materializeM0Pi: pass frozen.externalRecall to renderM0Pi; persist externalRecallHash in persistCachedM0. - renderM1PiWithMetadata: render external memory delta when live hash differs from m[0] baseline hash. - markersFromCachedPiRow: populate externalRecallHash from DB row. - readCurrentMarkersFromCompartments: read live externalRecallHash. ctx-search.ts: already wired (explicitSearch + projectName + external source in schema + formatResult). No changes needed.
Biome import ordering and formatting fixes for: - pi-plugin/src/context-handler.ts (import order + indentation) - pi-plugin/src/index.ts (import order) - pi-plugin/src/inject-compartments-pi.ts (import order + SQL formatting) - pi-plugin/src/tools/ctx-memory.ts (indentation) - plugin/src/.../external-recall.test.ts (import order + arrow fn)
…, STRUCTURE, CONFIGURATION, README) Adds the external-memory-backend feature to root docs: - ARCHITECTURE.md: new 'External-memory flow' subsection under Data Flow (tee-on-write → 3-slice recall + dedup + persist-before-settle → m[1] <external-memory> render → W2 correctives → ctx_search external source); new 'External memory backend (Hindsight)' Key Abstraction; cached_m0_ external_recall_hash marker added to the schema-migrations note (deliberately NOT a mustMaterialize trigger); project-config trust boundary updated to include memory.external in the user-only strip list; unified search entry updated with the external source. - STRUCTURE.md: new external-memory files listed under src/features/ plus a Key File Locations block (engine-agnostic provider, Hindsight impl, recall orchestrator, snapshot read, W2 hooks, project-security strip) and the Pi mirror path (@magic-context/core re-export). - CONFIGURATION.md: full memory.external block (provider, endpoint, api_key, project_bank template, main_bank, retain_sources, tags, recall sub-block) and the memory.external.recall defaults; full failure-semantics + cross-harness note; user-config-only security callout; example updated. - README.md: concise mention in the Recall section + a new callout for long-term memory; external added to the four-source ctx_search description and the agent-tools table.
…, not external-memory block
…ll_memory takes v37)
…plugin Root cause of recurring "failed to load plugin ... database is locked" errors in opencode.log (25 occurrences 2026-06-14..2026-06-18): When two plugin processes (OpenCode + Pi, or two OpenCode instances) cold-open the shared context.db at the same time, the loser's first write-locking operation inside initializeDatabase() — PRAGMA journal_mode=WAL or CREATE TABLE IF NOT EXISTS — throws SQLITE_BUSY because busy_timeout was only set INSIDE initializeDatabase, AFTER the sibling already held the WAL writer lock. The thrown error propagates through openDatabase()'s outer try/catch and disables Magic Context for the entire process lifetime on every cold start under multi-process contention. Three layered fixes: 1. Install PRAGMA busy_timeout on the raw Database handle BEFORE enforceSchemaFence / initializeDatabase run. The first PRAGMA on a fresh connection is now busy_timeout, so transient lock contention waits instead of throwing immediately. 2. Wrap the cold-open sequence (new Database + enforceSchemaFence + initializeDatabase + runMigrations) in a bounded retry loop (3 attempts, 250ms backoff). A long-running sibling transaction (large migration, dreamer run, Channel-2 bulk delivery) can hold the writer lock past the 5s busy_timeout; retrying recovers in the common case instead of disabling Magic Context for the run. 3. Enable PRAGMA wal_autocheckpoint=1000 in initializeDatabase so the WAL file doesn't grow unbounded under sustained concurrent writes (observed: 40MB+ WAL on long-running multi-process installs), which amplifies BUSY windows because checkpoint passes hold the write lock. Adds three regression tests covering the retry path, busy_timeout value, and wal_autocheckpoint. All 14 storage-db tests pass; typecheck and lint clean.
codeo1io
force-pushed
the
external-memory-backend
branch
from
June 19, 2026 17:41
62e49f9 to
db3213e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
Recurring
failed to load plugin ... database is lockederrors inopencode.log(25 occurrences 2026-06-14 through 2026-06-18). Each occurrence disabled Magic Context for the entire process lifetime.When two plugin processes (OpenCode + Pi, or two OpenCode instances) cold-open the shared
~/.local/share/cortexkit/magic-context/context.dbat the same time, the loser's first write-locking operation insideinitializeDatabase()—PRAGMA journal_mode=WALorCREATE TABLE IF NOT EXISTS— throwsSQLITE_BUSYimmediately because:busy_timeoutwas only set insideinitializeDatabase, afterenforceSchemaFence's readsinitializeDatabaseruns, the sibling already holds the WAL writer lock (mid-migration, dreamer lease, Channel-2 delivery)openDatabase()'s outer try/catch and disables Magic Context for the entire process lifetimeLive evidence from this host:
~/.local/share/cortexkit/magic-context/context.db— 2.6 GB, held open by 4opencode+ 4piprocesses simultaneouslycontext.db-wal— 43 MB uncheckpointed (amplifies BUSY windows because checkpoint passes hold the write lock)opencode.log— 25database is lockedplugin-load failures over 4 daysFix
Three layered changes in
packages/plugin/src/features/magic-context/storage-db.ts:Install
PRAGMA busy_timeouton the rawDatabasehandle BEFOREenforceSchemaFence/initializeDatabaserun. The first PRAGMA on a fresh connection is nowbusy_timeout, so transient lock contention waits up to 5s instead of throwingSQLITE_BUSYimmediately.Wrap the cold-open sequence in a bounded retry loop (3 attempts, 250ms backoff). A long-running sibling transaction can hold the writer lock past the 5s
busy_timeout; retrying recovers in the common case instead of disabling Magic Context for the run. OnlySQLITE_BUSY/SQLITE_LOCKED/database is lockederrors are retried; other errors fail-closed per the existing contract.Enable
PRAGMA wal_autocheckpoint=1000ininitializeDatabaseso the WAL file doesn't grow unbounded under sustained concurrent writes. Unbounded WAL growth amplifies BUSY windows because checkpoint passes hold the write lock.Tests
Adds three regression tests in
storage-db.test.ts:#when a sibling holds the writer lock #then openDatabase retries and succeeds after lock release#when busy_timeout is set #then the connection reports the configured timeout#when WAL mode is active #then wal_autocheckpoint is enabledAll 14 storage-db tests pass. typecheck and lint clean. Existing migration-race, storage, and message-index-async tests still pass (30/30 across the three files).
Files changed
packages/plugin/src/features/magic-context/storage-db.ts(+131 / -14)packages/plugin/src/features/magic-context/storage-db.test.ts(+59)Risk
Low. The retry loop only fires on the cold-open path (once per process boot), adds at most ~750ms to startup in the worst recoverable case, and only retries
SQLITE_BUSY-shaped errors. All other failure modes (schema fence, corrupt file, unwritable path) still fail-closed per the existing contract. Thewal_autocheckpoint=1000value is SQLite's default, made explicit for discoverability.