docs: centralize database authority and version logs

Author: ScriptedAlchemy

docs/plans/2026-07-09-tracedecay-brain-rewrite.md

@@ -201,7 +201,7 @@ The plan treats merged rows below as required base semantics at publication base
 | `#438` merged — restart-safe applied-manifest retirement | Validates and transactionally retires only proven schema-2 `Applied` source/target manifest owners under an exclusive lifecycle capability while leaving original shard data untouched and the destination canonical; final head `4f7b2b2c`, merge `3bea5ec7`. | Import restart-safe retirement as accepted anti-corruption behavior: retries are idempotent, ambiguous ownership fails closed with doctor evidence, registry rows and manifests change atomically, and V2 preserves the exact applied-ledger/retirement receipt (FM-111). |
 | `#439` merged — derive orphan stores from registry reconstruction | Reuses the read-only registry-reconstruction preflight to count only manifests actually missing project/alias/store/scope/artifact rows, replacing incomplete token-accounting/path proxies; final head `de55e376`, merge `974d423b`. | Doctor, health, migration, and repair share one per-manifest typed diff and population. Complete registry rows never produce an orphan warning; a reported orphan links the exact missing rows and reconstruction plan (FM-114). |
 | `#440` merged — isolate registry diff conflicts | Independently preflights each eligible reconstruction plan so one conflicting manifest does not hide missing rows in unrelated manifests; final head `7a56db8e`, merge `0dd1fd7d`. | Preserve each conflict, continue classifying every unrelated manifest, and expose per-manifest reconstruction/detection receipts through the shared catalog/doctor truth (FM-114). |
-| `#441` open — Hermes memory/context routing, with local follow-up | Remote PR head `152bfacc` gates first-turn guidance, routes memory/LCM from the logical Hermes session workspace, generalizes handle dereference, and isolates cloned context-engine state. Local worktree `/fast/projects/tracedecay` had amended head `26626698` plus user-owned follow-up for named Hermes profiles and a transitional profile-level `user-memory.db`; one open review finding shows generalized handle dereference can drop the original project selector. | Treat the failures as fixtures, not the draft implementation as V2 authority: host profile/config target is distinct from TraceDecay data profile; every invocation carries immutable session/workspace scope; singleton/clone state has explicit per-session versus process-shared ownership; non-code greetings produce useful silence; compatibility-handle retrieval preserves the exact canonical scope/auth binding; projectless activity remains activity-owned while durable user facts use explicit `DeclaredScope::Profile`; active-project memory composes profile plus exact-project facts with provenance. If the V1 `user-memory.db` ships, PR 33 migration imports it once into activity knowledge tables and retires the separate runtime store (FM-138–FM-143). |
+| `#441` open — Hermes memory/context routing, with local follow-up | Remote PR head `152bfacc` gates first-turn guidance, routes memory/LCM from the logical Hermes session workspace, generalizes handle dereference, and isolates cloned context-engine state. Local worktree `/fast/projects/tracedecay` had amended head `26626698` plus user-owned follow-up for named Hermes profiles and a transitional profile-level `user-memory.db`; one open review finding shows generalized handle dereference can drop the original project selector. A later local fix found branch creation copied/reflinked only the live main SQLite file while another connection could checkpoint WAL pages, so its branch lock did not fence database writers and the snapshot could be half-checkpointed or omit committed WAL data. | Treat the failures as fixtures, not the draft implementation as V2 authority: host profile/config target is distinct from TraceDecay data profile; every invocation carries immutable session/workspace scope; singleton/clone state has explicit per-session versus process-shared ownership; non-code greetings produce useful silence; compatibility-handle retrieval preserves the exact canonical scope/auth binding; projectless activity remains activity-owned while durable user facts use explicit `DeclaredScope::Profile`; active-project memory composes profile plus exact-project facts with provenance. If the V1 `user-memory.db` ships, PR 33 migration imports it once into activity knowledge tables and retires the separate runtime store. V2 never creates a branch database copy and never copies/reflinks a live main file: the daemon-owned snapshot service uses SQLite's online backup API or a bounded `VACUUM INTO` operation, pins a source watermark, verifies the destination, and publishes only a signed fsynced manifest (FM-138–FM-145). |
 
 PR `#409` was closed without merge and superseded by release PRs `#413`/`#416`; PRs #418, #425–#440 listed as merged above are accepted history where applicable. Latest audited `origin/master` is `273f50c0372f063b97f4755563a3ded65ef324d5` at 0.0.53; draft plan PR #421 and Hermes PR #441 are open. The plan branch must be rebased to this or a newer accepted base before final verification. The implementation lead refreshes open PRs, merge bases, changed files, checks, and TraceDecay semantic context immediately before each program phase. If GitHub and TraceDecay disagree, record both snapshots and reconcile index/ref freshness before changing the plan.
 
@@ -432,11 +432,14 @@ Ordering and concurrency contracts:
 - Each provider artifact/source instance has a monotonic source sequence; each session/agent stream has a projected sequence. There is no invented total global order.
 - Every observation carries occurred time, ingested time, source sequence, rewrite generation, and causation/correlation links. Late events are inserted without rewriting prior history.
 - One bounded writer actor owns each SQLite shard connection. Concurrent agents enqueue append batches through a private spool; read services use short-lived read-only snapshot pools.
+- The daemon is the only ordinary live process allowed to construct the V2 store factory, own mutable SQLite connections, checkpoint WAL, run query read pools, publish snapshots, or swap generations. CLI, MCP, hooks, dashboard, SDKs, automations, installers, and provider plugins call the daemon application service over authenticated local IPC or the protected HTTP API; they never open SQLite or fall back to an embedded writer/reader when the daemon is unavailable.
 - WAL, busy timeout, queue depth, maximum batch bytes, maximum transaction duration, and checkpoint thresholds are explicit configuration with safe defaults and telemetry.
+- Database snapshots/backups/branch evidence are transactionally consistent products, never filesystem clones: no copy, reflink, hard link, main-file-only archive, or DB-without-WAL family operation is legal on a live shard. The snapshot coordinator uses the SQLite online backup API for live backups and may use `VACUUM INTO` only as a bounded daemon/offline-maintenance primitive, then validates header/application/schema/page counts, integrity policy, source watermark and row-family manifests, fsyncs file plus parent, and atomically publishes the manifest.
 - Queue saturation applies tiered backpressure: coalesce rebuildable notifications, spill canonical observations durably, and reject/mark optional enrichment. It never silently drops prompts, tool events, approvals, edits, or outcomes.
 - Idempotency is per source record. Projectors and outbox consumers are at-least-once and idempotent. Leases are fenced by generation; a crashed owner cannot resume writes after a new owner takes over.
 - Readers receive vector watermarks and may request frozen, live, or eventual consistency. No read transaction survives UI think time or cursor pagination.
 - Crash tests cover process death before/after spool fsync, observation commit, outbox commit, hint render, host acknowledgement, projector checkpoint, and WAL checkpoint.
+- Corruption-resistance tests additionally race writes, readers, passive/TRUNCATE checkpoints, online backup/`VACUUM INTO`, generation publication, daemon restart, disk-full, short write, bit flip, truncation, and killpoints. They prove no architecture-induced corruption or committed-WAL omission; unavoidable media faults are detected before publication, quarantined as a whole DB/WAL/SHM family, and recovered from a verified snapshot without hiding healthy shards.
 
 The hint engine is a deterministic, versioned policy pipeline:
 
@@ -1583,6 +1586,7 @@ Current planning evidence: `gitleaks 8.30.1` found zero findings in the plan set
 
 Record and expose:
 
+- Every TraceDecay-owned log record carries the exact originating `ComponentVersion` and component/build identity. This is mandatory at emission for daemon, CLI, MCP, hook, provider integration, installer/updater, API/server, dashboard backend, automation, migration, remote-sync, crash, and test-harness logs; forwarding preserves the producer version instead of overwriting it with the collector version.
 - Source discovery, offsets, rewrite generations, ingest rate/lag/errors/quarantine.
 - Identity conflicts, unresolved aliases, ambiguous lineages.
 - Outbox/projector lag, retries, dead letters, rebuild progress.
@@ -1593,7 +1597,7 @@ Record and expose:
 - Privacy classifications, redactions, locked content, denied export/integration.
 - Migration receipts, parity gaps, rollback events.
 
-Do not log sensitive query literals or payloads. Use safe query fingerprints and sampled `EXPLAIN QUERY PLAN` metadata.
+Do not log sensitive query literals or payloads. Use safe query fingerprints and sampled `EXPLAIN QUERY PLAN` metadata. Log/diagnostic queries support exact/range/include/exclude/current-runtime version predicates and report excluded plus legacy-unknown counts, so old-version evidence can be removed from a diagnostic view without being deleted or silently hidden.
 
 ## 22. Target Repository Structure
 

docs/plans/tracedecay-v2/00-plan-set-index.md

@@ -39,7 +39,7 @@ Core product surfaces:
 | [`11-dashboard-frontend.md`](11-dashboard-frontend.md) | Concept-led Evidence Cartography product; stable profile atlas, linked Atlas/Trace/Compare/Lab/Triage compositions, complete memory/skill/automation navigation in Brain and Explorer, Loom replay player, composable lenses, hermetic experiment cockpit, visual ontology/renderers/charts, accessibility/mobile/export, perceptual and comprehension gates. |
 | [`12-root-compatibility-migration.md`](12-root-compatibility-migration.md) | Root binary/daemon/CLI/MCP composition and deployment/probe/config/service effect adapters; application-owned integration lifecycle execution; V1 data migration, cutover/rollback/retirement. |
 | [`13-research-provenance-and-context-anchors.md`](13-research-provenance-and-context-anchors.md) | Research manifest, durable retrieval anchors, subagent context, corpus hashes/cutoff, source recovery, future implementation handoff. |
-| [`14-historical-failure-regression-matrix.md`](14-historical-failure-regression-matrix.md) | Historical problem -> prevention owner -> visible detection/recovery -> cutover regression gate; 143 stable rows (`FM-001` through `FM-143`) with no gaps. |
+| [`14-historical-failure-regression-matrix.md`](14-historical-failure-regression-matrix.md) | Historical problem -> prevention owner -> visible detection/recovery -> cutover regression gate; 146 stable rows (`FM-001` through `FM-146`) with no gaps. |
 | [`15-search-quality-evaluation-and-retrieval-research.md`](15-search-quality-evaluation-and-retrieval-research.md) | Real local precision corpus, primary retrieval research, hybrid pipeline, qrels/metrics/holdouts, shadow/online evaluation, Search Quality Lab. |
 | [`16-cross-project-repository-worktree-scope.md`](16-cross-project-repository-worktree-scope.md) | Exceptional multi-repo/project/worktree/ref/store behavior, `ScopeSelectorV2`, routed retrieval, graph federation, CLI/MCP UX, Rspack/Rsbuild/React Router corpus. |
 | [`17-official-public-api-and-sdks.md`](17-official-public-api-and-sdks.md) | Official direct-agent/public API, contract IR/OpenAPI/JSON Schema, stable IDs/errors/cursors/batch/SSE, Rust/TS/Python SDKs, docs/sandbox/conformance. |

docs/plans/tracedecay-v2/01-domain-crate.md

@@ -201,6 +201,21 @@ pub struct ProfileId(pub uuid::Uuid);
 pub struct BrainId(pub uuid::Uuid);
 pub struct BrainNodeId(pub uuid::Uuid);
 pub struct StoreAuthorityId(pub uuid::Uuid);
+pub enum StoreIsolationModeV1 {
+    DedicatedServiceIdentity,
+    RemoteAuthorityOnly,
+    SameUserDegraded,
+}
+pub struct StoreIsolationStatusV1 {
+    pub mode: StoreIsolationModeV1,
+    pub database_read_denied_to_clients: bool,
+    pub database_write_denied_to_clients: bool,
+    pub service_identity_verified: bool,
+    pub endpoint_acl_verified: bool,
+    pub key_authority_verified: bool,
+    pub last_probe_at: UtcMicros,
+    pub evidence_digest: ManifestDigest,
+}
 pub struct ShardId(pub uuid::Uuid);
 pub struct PrivacyDomainId(pub uuid::Uuid);
 pub struct SourceInstanceId(pub uuid::Uuid);
@@ -309,6 +324,30 @@ pub struct HostSurfaceKindV1(String); // private, grammar-validated opaque host
 pub struct LocaleId(String); // private, grammar-validated canonical BCP-47 language tag
 pub struct NativeAgentId(pub PrivacyDomainBoundLocatorDigest); // provider-native alias, never literal public text
 pub struct ComponentVersion(String); // bounded ASCII semver/build grammar
+pub struct TraceDecayBuildRefV1 {
+    pub version: ComponentVersion,
+    pub component: RegistryEntryId,
+    pub build_manifest_digest: Option<ManifestDigest>,
+}
+pub struct ComponentVersionRequirementV1(String); // bounded canonical semver requirement grammar
+pub enum TraceDecayVersionSelectionBasisV1 { All, CurrentRuntime, CompatibleProtocol }
+pub enum LegacyUnknownVersionPolicyV1 { Include, Exclude, Only }
+pub struct DiagnosticLogEventV1 {
+    pub event_id: EntityId,
+    pub occurred_at: UtcMicros,
+    pub producer: TraceDecayBuildRefV1,
+    pub collector: Option<TraceDecayBuildRefV1>,
+    pub severity: RegistryEntryId,
+    pub event_code: RegistryEntryId,
+    pub correlation_id: Option<EntityId>,
+    pub safe_message: LogSafeText,
+}
+pub struct TraceDecayVersionSelectorV1 {
+    pub basis: TraceDecayVersionSelectionBasisV1,
+    pub include: BoundedVec<ComponentVersionRequirementV1, 16>,
+    pub exclude: BoundedVec<ComponentVersionRequirementV1, 16>,
+    pub legacy_unknown: LegacyUnknownVersionPolicyV1,
+}
 pub struct MediaTypeCode(String); // allowlisted IANA/media grammar, no parameters with literals
 pub struct LegacyBindingCode(String); // bounded historical CLI/MCP/HTTP identifier grammar
 pub struct SanitizationReceiptId(pub uuid::Uuid);
@@ -902,6 +941,8 @@ pub struct ModelCapabilityRefV1 {
 }
 ```
 
+`TraceDecayBuildRefV1.version` is required on every newly emitted TraceDecay log event and uses semantic-version precedence with prerelease/build handling; development artifacts use an explicit valid development/build version rather than an empty or inferred value. A forwarder sets `collector` but preserves `producer` byte-for-byte. Multi-line human diagnostics are one typed event or independently version-stamped continuation events. `TraceDecayVersionSelectionBasisV1::CurrentRuntime` resolves against the application server/CLI runtime captured at request admission; `CompatibleProtocol` resolves through the versioned compatibility manifest, never a string-prefix guess. Empty `include` means the selected basis population, then `exclude` subtracts. `LegacyUnknownVersionPolicyV1` exists only to query imported pre-contract records; a new `DiagnosticLogEventV1` cannot represent an unknown version. Metric labels still exclude arbitrary build/version cardinality—version is an indexed diagnostic predicate and evidence boundary, not a free-form metric dimension.
+
 `SavedViewV1` and `SavedViewDefinitionV1` are the one persisted/wire saved-view envelope. Plan 11 owns the UI-neutral `InvestigationStateV1` codec, bounded scene-trail interaction semantics, and `ExperimentViewSpecV1` presentation; plan 24 owns `TaskViewSpecV1` validation/lenses. All three variants live under this domain contract and share identity, name/owner scope, classification/redaction, live/frozen snapshot, optimistic version, expiry, revoke/reauthorize, and sharing lifecycle. Experiment views reference immutable experiment/run/cell/stage/comparison/comparison-cell/reduction/playhead identities and never embed inputs or outputs. A variant cannot introduce another saved-view ID, table, query scope, grant, route family, or command namespace. `PendingSanitization` is an automated safety state, not a human approval queue.
 
 Deterministic derivation uses fixed UUIDv5 namespaces published by `id.rs`. Input encoding is version byte `1`, then big-endian length-prefixed UTF-8 fields and fixed-width hash/integer fields. Enum tags use their registry snake-case names. No locale, platform path syntax, JSON object order, wall clock, or process randomness participates.