diff --git a/.beads/interactions.jsonl b/.beads/interactions.jsonl index c00e752..54e4c21 100644 --- a/.beads/interactions.jsonl +++ b/.beads/interactions.jsonl @@ -23,3 +23,17 @@ {"id":"int-826011e5ee8ee8d205e7e838693c8fa1","kind":"field_change","created_at":"2026-07-17T14:52:45.121813Z","actor":"Val Alexander","issue_id":"threads-986.17.9","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"Phase 4 FREEZE executed 2026-07-17 (Val approval recorded on this bead 2026-07-15; gate:val-phase4-freeze satisfied). Epic AC verified: (1) all four surfaces live+functional behind fixtures adapter (daemon flip filed as threads-v3g) — /weaves rail, thread pane, strand inspector, /proposals approval, merged in coven-cave PR #3223; (2) metaphor-referent binding preserved — Nova sign-off .17.8 closed 2026-07-17 with all three coherence checks PASS, no findings; source-pin suites hold referents in CI; (3) Charm pass complete (.17.7); (4) proposal flow E2E over real staged fixtures (.17.6); contract spec landed on main via PR #1 (a55d99af1). Children .17.1-.17.8 + .17.10 all closed with evidence. goals.md updated."}} {"id":"int-a0bacc8f5edaee7a580c1a218a43157a","kind":"field_change","created_at":"2026-07-17T14:52:46.121787Z","actor":"Val Alexander","issue_id":"threads-986.17","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"EPIC SUMMARY — Phase 4 Cave UX complete. Built: PHASE-4-CAVE-SURFACES.md contract (PR #1, a55d99af1); read plumbing with dual adapters (fixtures-first default, daemon behind same interface); weave rail with tension rollup (snapped>frayed>unknown>stale>holds); thread pane Holds/Frayed/Snapped with strands + channel bindings; strand inspector with tri-state diff + R7 lineage; proposal approval flow (staged DegradeToProposal, approve/reject daemon-forwarders, fail-closed everywhere) — all merged via coven-cave PR #3223 (645a1894 head, 18 checks green, test:app 740/740). Gates: Charm voice pass .17.7; Nova coherence sign-off .17.8 (PASS, no findings); Val UX-accept .17.10; Val freeze approval .17.9. Rendering rules R1-R11 enforced fail-closed; PatternDescriptor never rendered as enforcement; Gate-4 unknown states render Blocked. Follow-up: threads-v3g (daemon endpoints + adapter flip). Screenshots 01-07 at ~/.openclaw/workspace/sage/goals/phase4-artifacts/."}} {"id":"int-8181a70aeede0f1b0c58cd5fa4521a68","kind":"field_change","created_at":"2026-07-17T14:53:21.464406Z","actor":"Val Alexander","issue_id":"threads-986","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"PROGRAM EPIC COMPLETE — all four AC met. (1) Phases 0-4 delivered: v0.2 design freeze (.4); Phase 1 crate freeze (.18, coven-threads-core through v0.1.3); Phase 2 daemon integration freeze (.20, coven PR #382 merged f745117 — gate operational on main, + WARD-C6 compaction hook PR #393); Phase 3 portability freeze (.21, + jq4 surfaces map/.af exporter PR #2); Phase 4 Cave UX freeze (.17, coven-cave PR #3223). (2) RFC-0001 §5 conformance mirror 14/14 (.11). (3) Daemon imports validator: tier-0 edits route through validate_fail_closed, pinned v0.1.3. (4) Portability ships with C7 survives-serialization enforced (c7_roundtrip suite; C1-C7 jointly canonical in grimoire §9 per .12). 30 child beads closed with evidence. Remaining non-child follow-up: threads-v3g (daemon endpoints + adapter flip)."}} +{"id":"int-66ac26cbefee070c08321748420bad15","kind":"field_change","created_at":"2026-07-17T19:21:41.010774Z","actor":"Val Alexander","issue_id":"threads-eql","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"MERGED coven PR #409 (a9724b09), all checks green. ward_manifest baselines advance on every committed protected write (store_baseline upsert + advance_surface_baseline post-apply); refusals keep pending files and append proposal-scoped audit rows. Independently live-verified on the PR branch with a hermetic daemon over the real socket: baseline hash advanced after approve; the previously-broken second cycle (drift->stage->restore-to-advanced-baseline->approve) applied 200; refused approve kept file + audited and retry stayed viable. Clarified: tier-0 POST /edits is HeldForCoherence by design — approve is the sole tier-0 commit point."}} +{"id":"int-8c4cb8b33a94f385798ab5be6d435d8d","kind":"field_change","created_at":"2026-07-17T19:21:42.881808Z","actor":"Val Alexander","issue_id":"threads-v3g","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"COMPLETE, both sides merged. Daemon: coven PR #408 (85566ba) GET /api/v1/threads/weaves + POST /api/v1/threads/proposals/{id}/{approve|reject} per spec §3/§3.7, + P1 follow-up PR #409 (a9724b09, threads-eql) making multi-cycle approves work. Cave: coven-cave PR #3362 (3db8547c8) — daemon adapter is the DEFAULT (spec §1 fixtures-first condition lapsed; COVEN_THREADS_ADAPTER=fixtures opts out), daemon semantic refusals surface verbatim (proposal-refused/not-found/proposal-corrupt, 409/404) instead of transport-shaped 503. Live-verified end to end through the real socket with a hermetic daemon: reads normalize, staging, drift refusal, approve apply, corrupt guard, unknown-id fail-closed. Validation: coven gates green; cave test:app 758/758 + typecheck on merged head. Stowaway Desktop commit rescued to rescue/val-ui-tweaks-2026-07-17 (needs its own PR)."}} +{"id":"int-be6d3f843b0a440d8eb1d1c81d475d32","kind":"field_change","created_at":"2026-07-18T07:02:18.244791Z","actor":"Val Alexander","issue_id":"threads-vmf","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"MERGED coven PR #418 (b98504cd, all checks green incl. windows after cfg fix 2001da04). Root cause: handler Err propagated out of threads_weaves_response on the first WardConfig::load failure and daemon::handle_http_stream ?-returned before writing any HTTP bytes (empty reply, curl exit 52). Fix: per-familiar isolation (log to stderr + daemon-recovery.log, skip, continue — never abort fleet read) and handle_http_stream converts api handler errors to JSON 500 envelopes for every endpoint. Independently live-verified on the PR branch: fleet of healthy sage + Val's REAL v0.1-dialect nova ward.toml returns 200 with sage listed. Tests: threads_weaves_skips_malformed_ward_without_aborting_fleet_read, handle_http_stream_converts_api_handler_error_to_json_500. Surfacing of malformed-config familiars in the UI deferred to threads-v60 migration lane."}} +{"id":"int-b1b336460d93e610124263159faeada3","kind":"field_change","created_at":"2026-07-18T07:02:19.14866Z","actor":"Val Alexander","issue_id":"threads-wro","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"MERGED coven-cave PR #3407 (d1edf3e6, 19/19 checks). preferences-store + theme-store suites now set a hermetic temp COVEN_HOME so cave-home reconciliation stops walking the real ~/.coven legacy symlinks against temp canonical overrides. Verified locally on a migrated machine: both suites + backdrop/migration siblings green; remaining test:api failure is the separately-tracked pre-existing tts flake (cave-k7i3 pattern)."}} +{"id":"int-b0352cdd9a72fadc88de97b11bb2f4b9","kind":"field_change","created_at":"2026-07-18T08:11:35.780927Z","actor":"Val Alexander","issue_id":"threads-v60","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"COMPLETE. Command: coven ward migrate merged in coven PR #419 (74b09cd1, all checks green incl. windows after TOML-escape fixture fix 023dc5bb); shipping in v0.1.5 (signed tag pushed, release pipeline running). Fleet migration APPLIED on Val's machine 2026-07-18 with Val's decisions (SSH signing-key fingerprint via runtime arg; protected->tier 0 + protected_surface; editable->tier 2 Logged; backup-only legacy disposition): all nine familiars migrated with byte-exact ward.toml.v01.bak backups; thoth additionally needed a familiars.toml workspace= line (added, backup kept). Live verification post-kickstart: GET /api/v1/threads/weaves HTTP 200 with nine weaves, 4-8 threads each, all tensions Holds, all coherence Coherent — the authority-boundary protection layer is now live over real familiar memory. v0.1 approval_tiers/harness_blocks/invariants semantics intentionally not carried (no Phase-2 equivalent; preserved in backups)."}} +{"id":"int-7c8ce4baece71955acd514e5c34b1e12","kind":"field_change","created_at":"2026-07-18T08:24:30.113404Z","actor":"Val Alexander","issue_id":"threads-2q5","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"MERGED coven PR #393 2026-07-15 (daemon main; verified via git log 04e0ce7 ancestor of main). WARD-C6 CompactionLedger events append in store::vacuum_store_path with pre/post thread commitments; tests vacuum_appends_compaction_ledger_for_warded_surface, vacuum_without_warded_surfaces_appends_no_compaction_ledger, compaction_ledger_keeps_ward_audit_append_only. Full evidence in bead comments. Closing stale in_progress claim from coordinator session."}} +{"id":"int-45fee6c5015cd238bdf2b4f49197d1c5","kind":"field_change","created_at":"2026-07-18T08:24:30.930406Z","actor":"Val Alexander","issue_id":"threads-jq4","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"MERGED coven-threads PR #2 2026-07-15 (5b4a51a on main), tag v0.1.3 cut at that commit and coven pin bumped (30474c3). Surfaces content map + export_af lossy one-way Letta exporter, no .af import path, C7 conformance in tests/c7_roundtrip.rs. Full evidence in bead comments. Closing stale in_progress claim from coordinator session."}} +{"id":"int-96e600594080674425edaa10b7a4934c","kind":"field_change","created_at":"2026-07-18T12:03:18.346804Z","actor":"Val Alexander","issue_id":"threads-k9s","extra":{"field":"status","new_value":"open","old_value":"in_progress"}} +{"id":"int-3992491181076736d0b8c9d8072dd308","kind":"field_change","created_at":"2026-07-18T12:03:18.909381Z","actor":"Val Alexander","issue_id":"threads-3xd","extra":{"field":"status","new_value":"open","old_value":"in_progress"}} +{"id":"int-019bebf2d54257621bcb1e7d3bf06f74","kind":"field_change","created_at":"2026-07-18T12:03:19.445615Z","actor":"Val Alexander","issue_id":"threads-uqx","extra":{"field":"status","new_value":"open","old_value":"in_progress"}} +{"id":"int-1805bcc20e8bca84ac7425b1877298a9","kind":"field_change","created_at":"2026-07-18T14:42:35.772492Z","actor":"Val Alexander","issue_id":"threads-k9s","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"COMPLETE, both halves merged. Spec: §2.7 DegradedFamiliarView + §4.R12 + route-1 WeaveListView shape (coven-threads main 6476a67 + 6829e5d). Daemon: coven#422 merged e04a7af8 (reopened after accidental close; branch restored from verified head e5dd8fc5) — degraded entries with const reason + sanitized single-line errors, recovery-log breadcrumbs kept, no-ward familiars skipped. Cave: coven-cave#3415 merged 037bca078 — WeaveListView {weaves, degraded}, R12 blocked rail rows with exact copy 'ward unreadable — protection not verifiable', traceForDegradedFamiliar drawer, degraded fixture, forward-compat unknown reasons; typecheck + test:app 774/774. Fail-visible gap from threads-vmf closed end to end."}} +{"id":"int-c49d97f567aa70cc2b8d289e925f4114","kind":"field_change","created_at":"2026-07-18T14:45:05.028849Z","actor":"Val Alexander","issue_id":"threads-0ip","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"MERGED coven PR #426 (7ad55434), all checks green. Root causes fixed, not papered over: (1) windows pty_runner codex_json_batch_shim flake — side-effect files (args.txt/stdin.txt) were read before reliably visible on windows; now condition-based read_side_effect_file polling with deadline; (2) ubuntu smoke session-kill timeout — record_session_exit's natural-exit path could overwrite a concurrent 'killed' status; now update_session_status_if_current CAS (UPDATE ... WHERE status='running') with unit test conditionally_updates_session_status_only_from_expected_current_status; daemon.rs exit recorder uses it. Coordinator-reviewed the product diff. No assertions weakened, no quarantines needed."}} +{"id":"int-eefa76ec3968a5d984f3da51e113add6","kind":"field_change","created_at":"2026-07-18T21:24:21.080221Z","actor":"Val Alexander","issue_id":"threads-3ym","extra":{"field":"status","new_value":"in_progress","old_value":"open"}} +{"id":"int-17b3b70454cfdb7bbdaca849bc30ad2f","kind":"field_change","created_at":"2026-07-18T21:25:07.575924Z","actor":"Val Alexander","issue_id":"threads-3ym","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"PR #7 opened: https://github.com/OpenCoven/coven-threads/pull/7. 75+17+14 tests green, clippy clean. Awaiting merge + v0.1.4 tag."}} diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..fc3aecb --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,133 @@ +# Changelog + +All notable changes to `coven-threads-core` are documented here. + +## [0.2.0] — unreleased + +Compatibility note: `AuditEventType::ApplyAudit` and `WardAuditRecord::detail` +expand public exhaustive APIs. Downstream `match` arms and struct literals can +break under Cargo `^0.1`, so this ships on the `0.2.x` line rather than as a +`0.1.x` patch release. + +### Added + +- `AuditEventType::ApplyAudit` variant + `"apply_audit"` tag in `WARD_AUDIT_SCHEMA_SQL`'s CHECK constraint (coven-threads#5). + - Enables the coven daemon to persist Gate-4 Tier-2 applied-write records into `ward.audit` (downstream: coven#414). +- `WardAuditRecord::detail` field — nullable JSON column for event-type-specific payload. + - For `ApplyAudit` events: `{"prev_sha256":"","bytes_written":N}`. +- `WardAuditRecord::for_apply(...)` constructor — builds the audit row for a Gate-4 applied write. +- `WardAuditRecord::apply_prev_sha256_hex()` / `apply_bytes_written()` — decode helpers for `ApplyAudit` detail. +- `APPLY_AUDIT_DETAIL_KEY_PREV` / `APPLY_AUDIT_DETAIL_KEY_BYTES` — stable JSON key constants. +- `WARD_AUDIT_SCHEMA_STATE_SQL` plus stable state tags (`missing`, + `legacy_v013`, `current_v020`, `unknown`) — reusable table-local schema + fingerprint contract for daemon callers, using exact stored + `main.sqlite_master.sql` fingerprints for the durable `main.ward_audit` + table, explicit main indexes, and append-only main triggers, plus ordered + durable column metadata discovered through schema-qualified PRAGMAs. + Across all durable states, the reserved main-schema `ward_audit` / + `ward_audit_*` namespace is whitelisted to exactly `main.ward_audit`, its two + explicit indexes, and its two append-only triggers attached to + `main.ward_audit`; every other reserved main object (including + `ward_audit_new`, views, backup/shadow tables, or reserved-name + indexes/triggers attached elsewhere) fail-closes as `unknown`. `missing` now + means `main.ward_audit` is absent, no unexpected durable reserved object + exists, **and** no temp shadow/reserved temp object exists; any temp-schema + table/view/index/trigger named `ward_audit` or `ward_audit_*` also + fail-closes as `unknown`. Only the controlled fresh/migrated v0.2.0 table SQL + variants and the shipped v0.1.3 table SQL are accepted; no + whitespace-destroying normalization is applied. +- `WARD_AUDIT_SCHEMA_SQL` — atomic, self-guarding schema initialization for the + exact `current_v020` fingerprint. It now begins with `BEGIN IMMEDIATE`, so + the main-database write reservation is acquired before any guard read or + classification. That makes concurrent initializers serialize cleanly: the + second caller waits, re-runs the guard against the winner's committed schema, + and idempotently sees exact `current_v020` instead of racing into + `sqlite_master` lock errors. The SQL still permits only `missing` or exact + `current_v020` before any mutation, uses idempotent `IF NOT EXISTS` DDL for + daemon compatibility, targets durable schema objects in `main` wherever + SQLite syntax permits, then requires exact `current_v020` before `COMMIT`. + Exact `legacy_v013`, every drifted `unknown` shape, any unexpected durable + reserved object, and any temp shadow/reserved temp object fail closed; + callers must `ROLLBACK` after any init error. +- `WARD_AUDIT_MIGRATION_V020_SQL` — transaction SQL for a detail-preserving + rebuild of `main.ward_audit` guarded by the exact `legacy_v013` fingerprint + plus durable-whitelist and temp-shadow rejection. The daemon should query + `WARD_AUDIT_SCHEMA_STATE_SQL`, initialize when missing, migrate only + `legacy_v013`, continue on `current_v020`, and fail closed on `unknown`. The + migration independently re-checks `legacy_v013` before any `ALTER`, now + inside `BEGIN IMMEDIATE` so concurrent migrators serialize before the guard + read. After the rebuild, a distinct TEMP postcondition guard reruns the same + shared schema-state CTE/predicates and requires exact `current_v020` before + `COMMIT`, so a caller cannot durably commit a self-unknown rebuilt schema if + unexpected `main.ward_audit` / `main.ward_audit_*` drift appears mid- + transaction. After the first caller upgrades, a second caller waits, re-runs + the guard against exact current, and fails only at the legacy guard instead + of racing into `sqlite_master` lock errors. The SQL otherwise uses the same + exact stored-table + column/index/trigger predicate plus durable namespace + whitelist and temp-shadow rejection, then copies rows into the replacement + table without discarding evidence. +- Exhaustiveness test `schema_names_all_event_tags` extended to cover `ApplyAudit`. +- New tests: `for_apply_produces_correct_shape`, + `for_apply_roundtrips_json`, + `schema_state_query_returns_missing_on_empty_db`, + `fresh_schema_sql_initializes_current_schema_atomically_and_enforces_append_only`, + `exact_legacy_fixture_returns_legacy_v013`, + `exact_current_schema_returns_current_v020`, + `current_schema_sql_reruns_idempotently_and_preserves_rows_and_objects`, + `fresh_and_migrated_current_schemas_use_controlled_exact_sql_variants`, + `current_schema_with_spaced_event_type_literal_is_unknown`, + `legacy_schema_with_spaced_event_type_literal_is_unknown_and_guard_preserves_state`, + `legacy_schema_sql_rejects_and_rollback_preserves_state`, + `fresh_schema_preserves_user_version_zero_and_creates_current_shape`, + `fresh_schema_preserves_user_version_ninety_nine_and_creates_current_shape`, + `legacy_plus_extra_column_and_data_is_unknown_and_guard_preserves_state`, + `legacy_schema_with_extra_table_check_is_unknown_and_guard_preserves_constraint`, + `legacy_schema_with_extra_unique_is_unknown_and_guard_preserves_constraint`, + `current_schema_with_extra_table_check_is_unknown`, + `current_schema_with_extra_unique_is_unknown`, + `current_schema_missing_append_only_trigger_is_unknown_and_update_succeeds`, + `current_schema_with_extra_index_is_unknown`, + `current_schema_with_desc_index_is_unknown`, + `current_schema_with_collated_index_is_unknown`, + `current_schema_with_extra_reserved_main_view_or_table_is_unknown_and_schema_sql_preserves_state`, + `current_schema_with_altered_trigger_error_literal_is_unknown`, + `current_schema_with_altered_trigger_body_is_unknown`, + `schema_and_migration_sql_use_begin_immediate`, + `migration_sql_uses_distinct_pre_and_post_guards_before_commit`, + `absent_ward_audit_with_reserved_index_collision_is_unknown_and_schema_sql_preserves_other_objects`, + `absent_ward_audit_with_reserved_trigger_collision_is_unknown_and_schema_sql_preserves_other_objects`, + `unknown_partial_current_schema_rejects_schema_sql_and_preserves_state`, + `migration_rejects_current_schema_rows_with_detail_and_preserves_state`, + `legacy_schema_with_preexisting_main_ward_audit_new_is_unknown_and_guard_rejects_before_alter`, + `legacy_schema_upgrade_passes_post_guard_and_preserves_append_only_behavior`, + `post_guard_rejects_durable_drift_and_rollback_restores_exact_legacy_state`, + `rerunning_migration_after_legacy_upgrade_errors_and_preserves_rows`, + `concurrent_schema_initialization_serializes_without_locked_errors`, + `concurrent_legacy_migration_waits_then_rejects_current_at_guard`, + `sqlite_post_alter_failure_rollback_restores_legacy_schema_for_production_migration_contract`, + `schema_qualified_table_valued_pragmas_resolve_main_and_temp_separately`, + `current_main_with_temp_shadow_is_unknown_and_guards_preserve_main_and_temp_rows`, + `missing_main_with_temp_ward_audit_shadow_is_unknown_and_schema_sql_rejects_without_creating_main`, + `missing_main_with_reserved_temp_objects_is_unknown_and_schema_sql_preserves_temp_objects`, + `legacy_main_with_temp_shadow_is_unknown_and_migration_rejects_before_mutating_either_schema`, and + `unqualified_insert_targets_temp_shadow_while_schema_state_stays_unknown`. + +### Design notes + +- **Column approach (Option A):** `diff_hash` carries `next_sha256`; `prev_sha256` + `bytes_written` ride in the new `detail` TEXT column as JSON. This avoids a second table rebuild by not adding typed hash columns while keeping the data query-accessible via SQLite JSON functions. +- **Migration approach (Option B-lite):** exports + `WARD_AUDIT_SCHEMA_STATE_SQL`, `WARD_AUDIT_SCHEMA_SQL`, and + `WARD_AUDIT_MIGRATION_V020_SQL` so the daemon can classify `missing` / + `legacy_v013` / `current_v020` / `unknown`, perform a quiet fail-closed init + or upgrade, and recover cleanly with explicit rollback after any init or + migration error. The durable contract is explicitly `main.ward_audit`, and + the reserved durable namespace is valid only for that table plus its two + explicit indexes and two append-only triggers; any extra durable reserved + object or TEMP shadow/reserved temp object keeps the contract fail-closed as + `unknown`. Production dependencies stay unchanged, while `coven-threads-core` + now carries bundled `rusqlite` as a dev-dependency for executable migration + tests. + +## [0.1.3] — prior release + +Initial audit module with seven `AuditEventType` variants (`proposal_submitted`, `proposal_approved`, `proposal_rejected`, `proposal_vetoed`, `ward_updated`, `validation_verdict`, `compaction_ledger`). diff --git a/Cargo.lock b/Cargo.lock index a5a514b..0925248 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2,6 +2,18 @@ # It is not intended for manual editing. version = 3 +[[package]] +name = "ahash" +version = "0.8.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a15f179cd60c4584b8a8c596927aadc462e27f2ca70c04e0071964a73ba7a75" +dependencies = [ + "cfg-if", + "once_cell", + "version_check", + "zerocopy", +] + [[package]] name = "arrayref" version = "0.3.9" @@ -14,6 +26,12 @@ version = "0.7.8" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "d3fb67a6e08acf24fdeccbac2cb6ac4305825bd1f117462e0e6f2f193345ad56" +[[package]] +name = "bitflags" +version = "2.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da" + [[package]] name = "blake3" version = "1.8.5" @@ -67,9 +85,10 @@ checksum = "3d52eff69cd5e647efe296129160853a42795992097e8af39800e1060caeea9b" [[package]] name = "coven-threads-core" -version = "0.1.0" +version = "0.2.0" dependencies = [ "blake3", + "rusqlite", "serde", "serde_json", "sha2", @@ -125,6 +144,18 @@ dependencies = [ "crypto-common", ] +[[package]] +name = "fallible-iterator" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2acce4a10f12dc2fb14a218589d4f1f62ef011b2d0cc4b3cb1bba8e94da14649" + +[[package]] +name = "fallible-streaming-iterator" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7360491ce676a36bf9bb3c56c1aa791658183a54d2744120f27285738d90465a" + [[package]] name = "find-msvc-tools" version = "0.1.9" @@ -176,6 +207,24 @@ dependencies = [ "r-efi", ] +[[package]] +name = "hashbrown" +version = "0.14.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" +dependencies = [ + "ahash", +] + +[[package]] +name = "hashlink" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ba4ff7128dee98c7dc9794b6a411377e1404dba1c97deb8d1a55297bd25d8af" +dependencies = [ + "hashbrown", +] + [[package]] name = "itoa" version = "1.0.18" @@ -199,6 +248,17 @@ version = "0.2.186" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" +[[package]] +name = "libsqlite3-sys" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c10584274047cb335c23d3e61bcef8e323adae7c5c8c760540f73610177fc3f" +dependencies = [ + "cc", + "pkg-config", + "vcpkg", +] + [[package]] name = "memchr" version = "2.8.3" @@ -223,6 +283,12 @@ version = "0.2.17" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" +[[package]] +name = "pkg-config" +version = "0.3.33" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "19f132c84eca552bf34cab8ec81f1c1dcc229b811638f9d283dceabe58c5569e" + [[package]] name = "powerfmt" version = "0.2.0" @@ -253,6 +319,20 @@ version = "6.0.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" +[[package]] +name = "rusqlite" +version = "0.31.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b838eba278d213a8beaf485bd313fd580ca4505a00d5871caeb1457c55322cae" +dependencies = [ + "bitflags", + "fallible-iterator", + "fallible-streaming-iterator", + "hashlink", + "libsqlite3-sys", + "smallvec", +] + [[package]] name = "rustversion" version = "1.0.23" @@ -325,6 +405,12 @@ version = "0.4.12" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" +[[package]] +name = "smallvec" +version = "1.15.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90" + [[package]] name = "syn" version = "2.0.119" @@ -410,6 +496,12 @@ dependencies = [ "wasm-bindgen", ] +[[package]] +name = "vcpkg" +version = "0.2.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "accd4ea62f7bb7a82fe23066fb0957d48ef677f6eeb8215f372f52e48bb32426" + [[package]] name = "version_check" version = "0.9.5" @@ -461,6 +553,26 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "zerocopy" +version = "0.8.54" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b7cbbc0a705a0fd05cc3676525980d2bf5a9bc4adac6d6475209a7887cf59d19" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.54" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e2e817b7b52d0c7358d3246da9d69935ebb18116b2b102b4230dac079b4862f5" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + [[package]] name = "zmij" version = "1.0.23" diff --git a/Cargo.toml b/Cargo.toml index e1c41ea..a13a952 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,7 +3,7 @@ resolver = "2" members = ["crates/coven-threads-core"] [workspace.package] -version = "0.1.0" +version = "0.2.0" edition = "2021" rust-version = "1.75" license = "MIT" diff --git a/crates/coven-threads-core/Cargo.toml b/crates/coven-threads-core/Cargo.toml index 345d0e5..30e2582 100644 --- a/crates/coven-threads-core/Cargo.toml +++ b/crates/coven-threads-core/Cargo.toml @@ -19,3 +19,6 @@ uuid.workspace = true blake3.workspace = true sha2.workspace = true time.workspace = true + +[dev-dependencies] +rusqlite = { version = "0.31", features = ["bundled", "hooks"] } diff --git a/crates/coven-threads-core/src/audit.rs b/crates/coven-threads-core/src/audit.rs index fab7fc0..55b34b9 100644 --- a/crates/coven-threads-core/src/audit.rs +++ b/crates/coven-threads-core/src/audit.rs @@ -14,6 +14,85 @@ //! //! WARD-C6 (compaction ledger appends to `ward.audit`) rides the same table: //! `AuditEventType::CompactionLedger`. +//! +//! **Gate-4 applied writes** are also auditable here via +//! `AuditEventType::ApplyAudit` (see §3.4, coven-threads#5). +//! Because [`AuditEventType`] and [`WardAuditRecord`] are public exhaustive +//! surfaces, that new variant plus [`WardAuditRecord::detail`] define the +//! v0.2.0 contract rather than a v0.1.x-compatible patch. +//! +//! ## Schema shape and migration gating +//! +//! `WARD_AUDIT_SCHEMA_SQL` initializes or verifies the exact v0.2.0 +//! `main.ward_audit` shape in one `BEGIN IMMEDIATE` transaction, without +//! mutating database-wide `PRAGMA user_version`. It is safe for the daemon to +//! execute unconditionally on every store open: a pre-install guard permits +//! only `missing` or exact `current_v020`, the durable DDL runs explicitly in +//! `main` with `IF NOT EXISTS` compatibility, and a post-install guard requires +//! exact `current_v020` before `COMMIT`. The IMMEDIATE reservation means +//! concurrent initializers serialize before any guard/classification read, so a +//! second caller waits, re-runs the guard against the winner's committed +//! schema, and then idempotently sees `current_v020`. If either guard fails, +//! callers must explicitly `ROLLBACK` before continuing. +//! `WARD_AUDIT_SCHEMA_STATE_SQL` is the reusable, table-local fingerprint query +//! that returns one of four stable tags: +//! - `missing` — `main.ward_audit` does not exist, no unexpected durable +//! main-schema object named `ward_audit` or `ward_audit_*` exists, and no +//! temp-schema shadow/reserved object exists; initialize with +//! `WARD_AUDIT_SCHEMA_SQL`; +//! - `legacy_v013` — `main.ward_audit` exactly matches the v0.1.3 legacy +//! fingerprint, the durable reserved namespace contains only the expected +//! table/index/trigger objects attached to `main.ward_audit`, and no temp +//! shadow exists; run `WARD_AUDIT_MIGRATION_V020_SQL`; +//! - `current_v020` — `main.ward_audit` exactly matches the v0.2.0 current +//! fingerprint, the durable reserved namespace contains only the expected +//! table/index/trigger objects attached to `main.ward_audit`, and no temp +//! shadow exists; continue without schema work; +//! - `unknown` — every other shape, including any extra or missing declared +//! table constraint, column, index, or trigger, any unexpected durable +//! main-schema object named `ward_audit` or `ward_audit_*`, and any +//! temp-schema table/view/index/trigger named `ward_audit` or +//! `ward_audit_*`; fail closed. +//! +//! The fingerprint uses exact `main.sqlite_master.sql` text for the durable +//! table, explicit durable indexes, and append-only durable triggers, plus +//! ordered `pragma_table_info('ward_audit', 'main')` metadata and +//! `pragma_index_list('ward_audit', 'main')` for explicit index discovery. It +//! does **not** normalize whitespace: the only accepted `current_v020` +//! table-SQL variants are the fresh `CREATE TABLE ward_audit` form and the +//! quoted `CREATE TABLE "ward_audit"` form SQLite stores after the exact legacy +//! migration path, and the `legacy_v013` fingerprint includes the inline +//! comments preserved from the shipped v0.1.3 DDL. +//! `WARD_AUDIT_MIGRATION_V020_SQL` exists only for the exact `legacy_v013` +//! fingerprint with no unexpected durable reserved-namespace object and no temp +//! shadow. It independently re-checks that fingerprint inside one +//! `BEGIN IMMEDIATE` transaction before any `ALTER`, then adds `detail` and +//! rebuilds `main.ward_audit` so the current CHECK set is installed and every +//! existing row is preserved. Before `COMMIT`, a distinct TEMP postcondition +//! guard reruns the shared schema-state CTE/predicates and requires exact +//! `current_v020`, so callers cannot durably commit a rebuilt-but-self-unknown +//! namespace if extra `ward_audit` / `ward_audit_*` objects appear mid-transaction. +//! The IMMEDIATE reservation means concurrent migrators serialize before any +//! legacy-guard read: the winner upgrades first, and a second caller waits, +//! reclassifies the durable table as current when the guard re-runs, then fails +//! closed at the legacy guard without a `sqlite_master` lock race. If the +//! postcondition guard or any later migration step fails after +//! `ALTER TABLE main.ward_audit ADD COLUMN detail`, callers must explicitly roll +//! back the failed transaction before continuing so SQLite restores the +//! untouched legacy table. +//! +//! ## Where content hashes ride for applied writes +//! +//! `WardAuditRecord::diff_hash` carries `next_sha256` (the post-write content +//! hash, matching the RFC-0001 §5.6 `diff_hash` semantic for verdict rows). +//! The complementary `prev_sha256` and `bytes_written` ride in `detail` as a +//! compact JSON object `{"prev_sha256":"","bytes_written":N}`. This +//! choice: +//! - avoids new columns that would require another SQLite table rebuild; +//! - keeps `diff_hash` as the canonical single-hash field (consistent with +//! verdict rows where it holds the proposal diff hash); +//! - makes `prev_sha256` query-accessible via SQLite JSON functions when +//! needed (`json_extract(detail, '$.prev_sha256')`). use serde::{Deserialize, Serialize}; use time::OffsetDateTime; @@ -22,6 +101,11 @@ use crate::channel::Channel; use crate::ids::{FamiliarId, ProposalId, SurfaceId, ThreadId, WriterId}; use crate::validate::{MutationRequest, RejectReason, Verdict}; +/// Stable JSON key for the `detail` field of an `apply_audit` row; +pub const APPLY_AUDIT_DETAIL_KEY_PREV: &str = "prev_sha256"; +/// Stable JSON key for the bytes-written count in an `apply_audit` detail. +pub const APPLY_AUDIT_DETAIL_KEY_BYTES: &str = "bytes_written"; + /// Event types recorded in `ward.audit`. /// /// The first five are RFC-0001 §5.6's named set, verbatim. The last two are the @@ -44,6 +128,18 @@ pub enum AuditEventType { ValidationVerdict, /// WARD-C6 compaction ledger entry (§3.3, inherited by reference). CompactionLedger, + /// A Gate-4 Tier-2 write was applied and its content snapshot recorded + /// (coven-threads#5, coven#414). + /// + /// Field mapping for this event type: + /// - `diff_hash` — `next_sha256` (post-write content hash of the surface). + /// - `detail` — JSON `{"prev_sha256":"","bytes_written":N}`; + /// `prev_sha256` is the pre-write content hash, `bytes_written` is the + /// number of bytes written to the surface. + /// - `tier` — typically `"tier_2"` (logged-write tier). + /// - `decision` — `"applied"` (the write completed successfully). + /// - `files_touched` — the resolved surface ids that were written. + ApplyAudit, } impl AuditEventType { @@ -57,12 +153,17 @@ impl AuditEventType { AuditEventType::WardUpdated => "ward_updated", AuditEventType::ValidationVerdict => "validation_verdict", AuditEventType::CompactionLedger => "compaction_ledger", + AuditEventType::ApplyAudit => "apply_audit", } } } /// One row of `ward.audit` (RFC-0001 §5.6 field set). /// +/// For `ApplyAudit` events, the extra per-apply fields (`prev_sha256`, +/// `bytes_written`) are encoded in the `detail` field as a JSON object (see +/// module-level docs and [`APPLY_AUDIT_DETAIL_KEY_PREV`]). +/// /// `ward_hash` is RFC-0001 §5.6's audit-log field; at this layer it is the /// `weave_hash` of the weave the verdict was issued against — the commitment /// binding "which authority state decided this." @@ -86,7 +187,22 @@ pub struct WardAuditRecord { /// Who approved/decided, when a principal or tier actor is involved. pub approver: Option, /// Hash of the proposed diff, when the event carries one. + /// + /// For `ApplyAudit` events this is the `next_sha256` — the post-write + /// content hash of the surface. pub diff_hash: Option>, + /// Opaque JSON detail payload, event-type–specific. + /// + /// **Invariant for `ApplyAudit` rows:** `detail` MUST be a JSON object + /// containing exactly: + /// - `"prev_sha256"` — hex string (64 lower-case ASCII chars), the + /// pre-write SHA-256 of the surface content, or `""` when unknown. + /// - `"bytes_written"` — u64, number of bytes written to the surface. + /// + /// Other event types MAY leave `detail` as `None`, or use event-type- + /// specific shapes documented at their construction site. Callers MUST + /// NOT assume `detail` is non-null for non-`ApplyAudit` rows. + pub detail: Option, /// The surfaces this event touches. pub files_touched: Vec, /// The channel the triggering request arrived on, if any (§2.4). @@ -137,6 +253,7 @@ impl WardAuditRecord { decision, approver: None, diff_hash: None, + detail: None, files_touched: vec![request.surface.clone()], channel: Some(request.channel), thread_id, @@ -144,6 +261,82 @@ impl WardAuditRecord { decided_at, } } + + /// Build the audit row for a Gate-4 Tier-2 applied write (coven-threads#5). + /// + /// `next_hash` is the post-write content hash (SHA-256) of the surface; + /// it rides in `diff_hash` matching the single-hash pattern used for + /// verdict rows. + /// + /// `prev_hash` (pre-write content hash) and `bytes_written` are encoded in + /// `detail` as `{"prev_sha256":"","bytes_written":N}` because adding + /// typed columns would require an immediate SQLite table rebuild in all + /// consumers. + /// + /// `weave_hash` is the Ward/weave version the Gate-4 decision was made + /// against (RFC-0001 §5.6 `ward_hash`). + #[allow(clippy::too_many_arguments)] + pub fn for_apply( + familiar_id: FamiliarId, + weave_hash: &[u8], + surface: SurfaceId, + tier: &str, + prev_hash: Option<&[u8]>, + next_hash: Option<&[u8]>, + bytes_written: u64, + channel: Option, + submitted_at: OffsetDateTime, + decided_at: OffsetDateTime, + ) -> Self { + let prev_hex = prev_hash.map(bytes_to_hex); + let detail = serde_json::json!({ + APPLY_AUDIT_DETAIL_KEY_PREV: prev_hex.unwrap_or_default(), + APPLY_AUDIT_DETAIL_KEY_BYTES: bytes_written, + }) + .to_string(); + Self { + event_type: AuditEventType::ApplyAudit, + proposal_id: None, + familiar_id, + ward_version: None, + ward_hash: weave_hash.to_vec(), + tier: Some(tier.to_string()), + decision: "applied".to_string(), + approver: None, + diff_hash: next_hash.map(|h| h.to_vec()), + detail: Some(detail), + files_touched: vec![surface], + channel, + thread_id: None, + submitted_at, + decided_at, + } + } + + /// Decode `prev_sha256` from the `detail` JSON for `ApplyAudit` rows. + /// + /// Returns `None` for non-`ApplyAudit` events or if the field is missing. + pub fn apply_prev_sha256_hex(&self) -> Option { + let detail = self.detail.as_deref()?; + let v: serde_json::Value = serde_json::from_str(detail).ok()?; + v.get(APPLY_AUDIT_DETAIL_KEY_PREV) + .and_then(|x| x.as_str()) + .filter(|s| !s.is_empty()) + .map(|s| s.to_string()) + } + + /// Decode `bytes_written` from the `detail` JSON for `ApplyAudit` rows. + /// + /// Returns `None` for non-`ApplyAudit` events or if the field is missing. + pub fn apply_bytes_written(&self) -> Option { + let detail = self.detail.as_deref()?; + let v: serde_json::Value = serde_json::from_str(detail).ok()?; + v.get(APPLY_AUDIT_DETAIL_KEY_BYTES).and_then(|x| x.as_u64()) + } +} + +fn bytes_to_hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() } fn reject_tag(reason: &RejectReason) -> &'static str { @@ -158,12 +351,617 @@ fn reject_tag(reason: &RejectReason) -> &'static str { } } -/// DDL for the `ward.audit` table inside `coven.sqlite3` (§3.4). +/// Stable tag returned by [`WARD_AUDIT_SCHEMA_STATE_SQL`] when `main.ward_audit` +/// is absent, no unexpected durable main-schema `ward_audit` / +/// `ward_audit_*` object exists, and no temp shadow/reserved object blocks the +/// durable contract. +pub const WARD_AUDIT_SCHEMA_STATE_MISSING: &str = "missing"; +/// Stable tag returned by [`WARD_AUDIT_SCHEMA_STATE_SQL`] for the exact +/// `main.ward_audit` v0.1.3 legacy schema fingerprint, with no unexpected +/// durable reserved-namespace object and no temp shadow. +pub const WARD_AUDIT_SCHEMA_STATE_LEGACY_V013: &str = "legacy_v013"; +/// Stable tag returned by [`WARD_AUDIT_SCHEMA_STATE_SQL`] for the exact +/// `main.ward_audit` v0.2.0 current schema fingerprint, with no unexpected +/// durable reserved-namespace object and no temp shadow. +pub const WARD_AUDIT_SCHEMA_STATE_CURRENT_V020: &str = "current_v020"; +/// Stable tag returned by [`WARD_AUDIT_SCHEMA_STATE_SQL`] for every other +/// `main.ward_audit` shape, plus any unexpected durable reserved-namespace +/// object or any temp-schema `ward_audit` / `ward_audit_*` shadow object. +pub const WARD_AUDIT_SCHEMA_STATE_UNKNOWN: &str = "unknown"; + +macro_rules! ward_audit_reserved_name_predicate_sql { + () => { + r#"(lower(name) = 'ward_audit' OR lower(name) GLOB 'ward_audit_*')"# + }; +} + +macro_rules! ward_audit_expected_durable_object_predicate_sql { + () => { + r#"( + (type = 'table' AND name = 'ward_audit') + OR (type = 'index' AND name = 'ward_audit_event_idx' AND tbl_name = 'ward_audit') + OR (type = 'index' AND name = 'ward_audit_familiar_idx' AND tbl_name = 'ward_audit') + OR (type = 'trigger' AND name = 'ward_audit_append_only_update' AND tbl_name = 'ward_audit') + OR (type = 'trigger' AND name = 'ward_audit_append_only_delete' AND tbl_name = 'ward_audit') + )"# + }; +} + +macro_rules! ward_audit_schema_state_ctes_sql { + () => { + concat!( + r#" +WITH + ward_audit_table AS ( + SELECT sql + FROM main.sqlite_master + WHERE type = 'table' AND name = 'ward_audit' + ), + ward_audit_exists AS ( + SELECT EXISTS(SELECT 1 FROM ward_audit_table) AS ok + ), + ward_audit_column_fingerprint AS ( + SELECT COALESCE( + group_concat( + printf( + '%d|%s|%s|%d|%s|%d', + cid, + name, + type, + "notnull", + COALESCE(dflt_value, ''), + pk + ), + '||' + ), + '' + ) AS fp + FROM ( + SELECT cid, name, type, "notnull", dflt_value, pk + FROM pragma_table_info('ward_audit', 'main') + ORDER BY cid + ) + ), + ward_audit_index_fingerprint AS ( + SELECT COALESCE(group_concat(item, '||'), '') AS fp + FROM ( + SELECT printf('%s|%s', il.name, sm.sql) AS item + FROM pragma_index_list('ward_audit', 'main') AS il + JOIN main.sqlite_master AS sm + ON sm.type = 'index' AND sm.name = il.name + WHERE il.origin = 'c' AND sm.sql IS NOT NULL + ORDER BY il.name + ) + ), + ward_audit_trigger_fingerprint AS ( + SELECT COALESCE(group_concat(item, '||'), '') AS fp + FROM ( + SELECT printf('%s|%s', name, COALESCE(sql, '')) AS item + FROM main.sqlite_master + WHERE type = 'trigger' AND tbl_name = 'ward_audit' + ORDER BY name + ) + ), + ward_audit_unexpected_durable_namespace_object_count AS ( + SELECT COUNT(*) AS count + FROM main.sqlite_master + WHERE type IN ('table', 'index', 'trigger', 'view') + AND "#, + ward_audit_reserved_name_predicate_sql!(), + r#" + AND NOT "#, + ward_audit_expected_durable_object_predicate_sql!(), + r#" + ), + ward_audit_temp_shadow_object_count AS ( + SELECT COUNT(*) AS count + FROM temp.sqlite_master + WHERE type IN ('table', 'index', 'trigger', 'view') + AND "#, + ward_audit_reserved_name_predicate_sql!(), + r#" + ), + ward_audit_shape AS ( + SELECT + (SELECT ok FROM ward_audit_exists) AS table_exists, + COALESCE((SELECT sql FROM ward_audit_table), '') AS table_sql, + (SELECT fp FROM ward_audit_column_fingerprint) AS column_fp, + (SELECT fp FROM ward_audit_index_fingerprint) AS index_fp, + (SELECT fp FROM ward_audit_trigger_fingerprint) AS trigger_fp, + (SELECT count FROM ward_audit_unexpected_durable_namespace_object_count) + AS unexpected_durable_namespace_object_count, + (SELECT count FROM ward_audit_temp_shadow_object_count) AS temp_shadow_count + ) +"# + ) + }; +} + +macro_rules! ward_audit_exact_legacy_table_sql_sql { + () => { + r#"'CREATE TABLE ward_audit ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + event_type TEXT NOT NULL CHECK (event_type IN ( + ''proposal_submitted'',''proposal_approved'',''proposal_rejected'', + ''proposal_vetoed'',''ward_updated'',''validation_verdict'', + ''compaction_ledger'')), + proposal_id TEXT, + familiar_id TEXT NOT NULL, + ward_version TEXT, + ward_hash BLOB NOT NULL, + tier TEXT, + decision TEXT NOT NULL, + approver TEXT, + diff_hash BLOB, + files_touched TEXT NOT NULL, -- JSON array of surface ids + channel TEXT, + thread_id TEXT, + submitted_at TEXT NOT NULL, -- RFC 3339 + decided_at TEXT NOT NULL, -- RFC 3339 + recorded_at TEXT NOT NULL DEFAULT (strftime(''%Y-%m-%dT%H:%M:%fZ'',''now'')) +)'"# + }; +} + +macro_rules! ward_audit_exact_current_fresh_table_sql_sql { + () => { + r#"'CREATE TABLE ward_audit ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + event_type TEXT NOT NULL CHECK (event_type IN ( + ''proposal_submitted'',''proposal_approved'',''proposal_rejected'', + ''proposal_vetoed'',''ward_updated'',''validation_verdict'', + ''compaction_ledger'',''apply_audit'')), + proposal_id TEXT, + familiar_id TEXT NOT NULL, + ward_version TEXT, + ward_hash BLOB NOT NULL, + tier TEXT, + decision TEXT NOT NULL, + approver TEXT, + diff_hash BLOB, + detail TEXT, + files_touched TEXT NOT NULL, + channel TEXT, + thread_id TEXT, + submitted_at TEXT NOT NULL, + decided_at TEXT NOT NULL, + recorded_at TEXT NOT NULL DEFAULT (strftime(''%Y-%m-%dT%H:%M:%fZ'',''now'')) +)'"# + }; +} + +macro_rules! ward_audit_exact_current_migrated_table_sql_sql { + () => { + r#"'CREATE TABLE "ward_audit" ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + event_type TEXT NOT NULL CHECK (event_type IN ( + ''proposal_submitted'',''proposal_approved'',''proposal_rejected'', + ''proposal_vetoed'',''ward_updated'',''validation_verdict'', + ''compaction_ledger'',''apply_audit'')), + proposal_id TEXT, + familiar_id TEXT NOT NULL, + ward_version TEXT, + ward_hash BLOB NOT NULL, + tier TEXT, + decision TEXT NOT NULL, + approver TEXT, + diff_hash BLOB, + detail TEXT, + files_touched TEXT NOT NULL, + channel TEXT, + thread_id TEXT, + submitted_at TEXT NOT NULL, + decided_at TEXT NOT NULL, + recorded_at TEXT NOT NULL DEFAULT (strftime(''%Y-%m-%dT%H:%M:%fZ'',''now'')) +)'"# + }; +} + +macro_rules! ward_audit_exact_explicit_index_fp_sql { + () => { + r#"'ward_audit_event_idx|CREATE INDEX ward_audit_event_idx ON ward_audit (event_type, recorded_at)||ward_audit_familiar_idx|CREATE INDEX ward_audit_familiar_idx ON ward_audit (familiar_id, recorded_at)'"# + }; +} + +macro_rules! ward_audit_exact_trigger_fp_sql { + () => { + r#"'ward_audit_append_only_delete|CREATE TRIGGER ward_audit_append_only_delete +BEFORE DELETE ON ward_audit +BEGIN + SELECT RAISE(ABORT, ''ward_audit is append-only (RFC-0001 §5.6)''); +END||ward_audit_append_only_update|CREATE TRIGGER ward_audit_append_only_update +BEFORE UPDATE ON ward_audit +BEGIN + SELECT RAISE(ABORT, ''ward_audit is append-only (RFC-0001 §5.6)''); +END'"# + }; +} + +macro_rules! ward_audit_exact_legacy_predicate_sql { + () => { + concat!( + r#" +table_exists = 1 +AND unexpected_durable_namespace_object_count = 0 +AND temp_shadow_count = 0 +AND table_sql = "#, + ward_audit_exact_legacy_table_sql_sql!(), + r#" +AND column_fp = '0|id|INTEGER|0||1||1|event_type|TEXT|1||0||2|proposal_id|TEXT|0||0||3|familiar_id|TEXT|1||0||4|ward_version|TEXT|0||0||5|ward_hash|BLOB|1||0||6|tier|TEXT|0||0||7|decision|TEXT|1||0||8|approver|TEXT|0||0||9|diff_hash|BLOB|0||0||10|files_touched|TEXT|1||0||11|channel|TEXT|0||0||12|thread_id|TEXT|0||0||13|submitted_at|TEXT|1||0||14|decided_at|TEXT|1||0||15|recorded_at|TEXT|1|strftime(''%Y-%m-%dT%H:%M:%fZ'',''now'')|0' +AND index_fp = "#, + ward_audit_exact_explicit_index_fp_sql!(), + r#" +AND trigger_fp = "#, + ward_audit_exact_trigger_fp_sql!(), + r#" +"# + ) + }; +} + +macro_rules! ward_audit_exact_current_predicate_sql { + () => { + concat!( + r#" +table_exists = 1 +AND unexpected_durable_namespace_object_count = 0 +AND temp_shadow_count = 0 +AND table_sql IN ("#, + ward_audit_exact_current_fresh_table_sql_sql!(), + r#", "#, + ward_audit_exact_current_migrated_table_sql_sql!(), + r#") +AND column_fp = '0|id|INTEGER|0||1||1|event_type|TEXT|1||0||2|proposal_id|TEXT|0||0||3|familiar_id|TEXT|1||0||4|ward_version|TEXT|0||0||5|ward_hash|BLOB|1||0||6|tier|TEXT|0||0||7|decision|TEXT|1||0||8|approver|TEXT|0||0||9|diff_hash|BLOB|0||0||10|detail|TEXT|0||0||11|files_touched|TEXT|1||0||12|channel|TEXT|0||0||13|thread_id|TEXT|0||0||14|submitted_at|TEXT|1||0||15|decided_at|TEXT|1||0||16|recorded_at|TEXT|1|strftime(''%Y-%m-%dT%H:%M:%fZ'',''now'')|0' +AND index_fp = "#, + ward_audit_exact_explicit_index_fp_sql!(), + r#" +AND trigger_fp = "#, + ward_audit_exact_trigger_fp_sql!(), + r#" +"# + ) + }; +} + +macro_rules! ward_audit_schema_state_case_sql { + () => { + concat!( + r#"CASE + WHEN temp_shadow_count > 0 THEN 'unknown' + WHEN table_exists = 0 AND unexpected_durable_namespace_object_count = 0 THEN 'missing' + WHEN "#, + ward_audit_exact_legacy_predicate_sql!(), + r#" THEN 'legacy_v013' + WHEN "#, + ward_audit_exact_current_predicate_sql!(), + r#" THEN 'current_v020' + ELSE 'unknown' +END"# + ) + }; +} + +/// Table-local schema-state query for the durable `main.ward_audit` contract +/// inside `coven.sqlite3` (§3.4). +/// +/// Callers run this exact query and branch on the stable text result: +/// - [`WARD_AUDIT_SCHEMA_STATE_MISSING`] — `main.ward_audit` is absent, no +/// unexpected durable main-schema object named `ward_audit` or +/// `ward_audit_*` exists, and no temp shadow/reserved object exists; +/// initialize with [`WARD_AUDIT_SCHEMA_SQL`]; +/// - [`WARD_AUDIT_SCHEMA_STATE_LEGACY_V013`] — run +/// [`WARD_AUDIT_MIGRATION_V020_SQL`]; +/// - [`WARD_AUDIT_SCHEMA_STATE_CURRENT_V020`] — continue without schema work; +/// - [`WARD_AUDIT_SCHEMA_STATE_UNKNOWN`] — fail closed and investigate the +/// table manually. +/// +/// The fingerprint is strict: the exact `main.sqlite_master.sql` stored for +/// `main.ward_audit`, ordered column metadata from +/// `pragma_table_info('ward_audit', 'main')`, the exact explicit main-index SQL +/// set (discovered with `pragma_index_list('ward_audit', 'main')` and then read +/// from `main.sqlite_master`), and the exact append-only main-trigger SQL set +/// must all match. Full stored-table-SQL equality covers every declared +/// table-level constraint (`CHECK`, `UNIQUE`, foreign-key clauses, and the +/// `event_type` list), so any extra or missing column, constraint, index, or +/// trigger returns `unknown`. Across **all** durable states, the reserved +/// main-schema namespace is whitelisted to exactly these objects attached to +/// `main.ward_audit`: the `ward_audit` table, `ward_audit_event_idx`, +/// `ward_audit_familiar_idx`, `ward_audit_append_only_update`, and +/// `ward_audit_append_only_delete`. Any other main-schema table/view/index/ +/// trigger whose name is exactly `ward_audit` or begins with `ward_audit_` +/// returns `unknown`, including `ward_audit_new`, backup/shadow tables, or +/// reserved-name indexes/triggers attached elsewhere. Any temp-schema +/// table/view/index/trigger whose name is exactly `ward_audit` or begins with +/// `ward_audit_` also returns `unknown`, even when `main.ward_audit` itself is +/// exact current or legacy, so callers cannot treat a shadowed namespace as +/// healthy durable state. No whitespace-destroying normalization is applied: +/// the only accepted `current_v020` table SQL variants are the fresh +/// `CREATE TABLE ward_audit (...)` form and SQLite's quoted +/// `CREATE TABLE "ward_audit" (...)` form produced by the exact legacy +/// migration path, while the `legacy_v013` fingerprint intentionally includes +/// the inline comments preserved from the shipped v0.1.3 DDL. +pub const WARD_AUDIT_SCHEMA_STATE_SQL: &str = concat!( + ward_audit_schema_state_ctes_sql!(), + r#" +SELECT "#, + ward_audit_schema_state_case_sql!(), + r#" AS schema_state +FROM ward_audit_shape; +"#, +); + +/// DDL migration for the exact `main.ward_audit` `legacy_v013` fingerprint +/// inside `coven.sqlite3` (§3.4). /// -/// Idempotent (`IF NOT EXISTS` throughout) so the daemon can apply it at -/// startup. Append-only is enforced *in the store*: UPDATE and DELETE abort via +/// Append-only is enforced *in the store*: UPDATE and DELETE abort via /// triggers (RFC-0001 §5.6: entries MUST NOT be deleted or modified). -pub const WARD_AUDIT_SCHEMA_SQL: &str = r#" +/// SQLite cannot `ALTER` a CHECK constraint on an existing table, so this +/// transaction: +/// 1. reserves the main database up front with `BEGIN IMMEDIATE`, then +/// re-checks the exact legacy fingerprint in SQL before any mutation, +/// including exact stored `main.sqlite_master.sql` equality plus main +/// column/index/trigger fingerprints, the durable reserved-namespace +/// whitelist, and temp-shadow rejection; +/// 2. adds the legacy `detail` column on `main.ward_audit` so the old table +/// matches the copy shape; +/// 3. creates `main.ward_audit_new` with the updated CHECK; +/// 4. copies every existing row, preserving `detail`; +/// 5. swaps the tables; and +/// 6. re-creates the exact explicit main indexes and append-only main triggers; +/// and +/// 7. re-runs the shared schema-state expression in a distinct TEMP +/// postcondition guard and requires exact `current_v020` before `COMMIT`. +/// +/// Callers should still branch on [`WARD_AUDIT_SCHEMA_STATE_SQL`] first: +/// initialize when the state is `missing`, migrate only `legacy_v013`, +/// continue on `current_v020`, and fail closed on `unknown`. This migration +/// independently guards the same `legacy_v013` fingerprint, durable +/// reserved-namespace whitelist, temp-shadow rejection, and exact-current +/// postcondition so callers cannot mutate a partial, already-current, +/// shadowed, or rebuilt-but-drifted schema by skipping classification. Because +/// the transaction begins IMMEDIATELY, concurrent migrators serialize before +/// the guard read; after the winner commits, a second caller re-runs the guard +/// against the now-current schema and fails closed there rather than racing +/// into a `sqlite_master` lock. If the postcondition guard or a later step +/// fails after `ALTER TABLE main.ward_audit ADD COLUMN detail`, callers must +/// `ROLLBACK` the failed transaction before continuing so SQLite restores the +/// untouched legacy table. This SQL does not read or write database-wide +/// `PRAGMA user_version`. +pub const WARD_AUDIT_MIGRATION_V020_SQL: &str = concat!( + r#" +BEGIN IMMEDIATE; + +CREATE TEMP TABLE coven_threads_ward_audit_migration_guard ( + ok INTEGER NOT NULL CHECK (ok = 1) +); + +INSERT INTO coven_threads_ward_audit_migration_guard (ok) +"#, + ward_audit_schema_state_ctes_sql!(), + r#" +SELECT CASE + WHEN "#, + ward_audit_exact_legacy_predicate_sql!(), + r#" THEN 1 + ELSE 0 +END +FROM ward_audit_shape; + +DROP TABLE coven_threads_ward_audit_migration_guard; + +ALTER TABLE main.ward_audit ADD COLUMN detail TEXT; + +CREATE TABLE main.ward_audit_new ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + event_type TEXT NOT NULL CHECK (event_type IN ( + 'proposal_submitted','proposal_approved','proposal_rejected', + 'proposal_vetoed','ward_updated','validation_verdict', + 'compaction_ledger','apply_audit')), + proposal_id TEXT, + familiar_id TEXT NOT NULL, + ward_version TEXT, + ward_hash BLOB NOT NULL, + tier TEXT, + decision TEXT NOT NULL, + approver TEXT, + diff_hash BLOB, + detail TEXT, + files_touched TEXT NOT NULL, + channel TEXT, + thread_id TEXT, + submitted_at TEXT NOT NULL, + decided_at TEXT NOT NULL, + recorded_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')) +); + +INSERT INTO main.ward_audit_new ( + id, event_type, proposal_id, familiar_id, ward_version, ward_hash, + tier, decision, approver, diff_hash, detail, files_touched, channel, + thread_id, submitted_at, decided_at, recorded_at +) +SELECT + id, event_type, proposal_id, familiar_id, ward_version, ward_hash, + tier, decision, approver, diff_hash, detail, files_touched, channel, + thread_id, submitted_at, decided_at, recorded_at +FROM main.ward_audit; + +DROP TABLE main.ward_audit; +ALTER TABLE main.ward_audit_new RENAME TO ward_audit; + +CREATE INDEX IF NOT EXISTS main.ward_audit_familiar_idx ON ward_audit (familiar_id, recorded_at); +CREATE INDEX IF NOT EXISTS main.ward_audit_event_idx ON ward_audit (event_type, recorded_at); + +CREATE TRIGGER IF NOT EXISTS main.ward_audit_append_only_update +BEFORE UPDATE ON ward_audit +BEGIN + SELECT RAISE(ABORT, 'ward_audit is append-only (RFC-0001 §5.6)'); +END; + +CREATE TRIGGER IF NOT EXISTS main.ward_audit_append_only_delete +BEFORE DELETE ON ward_audit +BEGIN + SELECT RAISE(ABORT, 'ward_audit is append-only (RFC-0001 §5.6)'); +END; + +CREATE TEMP TABLE coven_threads_ward_audit_migration_post_guard ( + ok INTEGER NOT NULL CHECK (ok = 1) +); + +INSERT INTO coven_threads_ward_audit_migration_post_guard (ok) +"#, + ward_audit_schema_state_ctes_sql!(), + r#" +SELECT CASE + WHEN ("#, + ward_audit_schema_state_case_sql!(), + r#") = 'current_v020' THEN 1 + ELSE 0 +END +FROM ward_audit_shape; + +DROP TABLE coven_threads_ward_audit_migration_post_guard; + +COMMIT; +"#, +); + +// Shared durable-main current-v0.2.0 DDL body used by the guarded init SQL and +// drift tests. +macro_rules! ward_audit_current_objects_sql { + () => { + r#" +CREATE TABLE IF NOT EXISTS main.ward_audit ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + event_type TEXT NOT NULL CHECK (event_type IN ( + 'proposal_submitted','proposal_approved','proposal_rejected', + 'proposal_vetoed','ward_updated','validation_verdict', + 'compaction_ledger','apply_audit')), + proposal_id TEXT, + familiar_id TEXT NOT NULL, + ward_version TEXT, + ward_hash BLOB NOT NULL, + tier TEXT, + decision TEXT NOT NULL, + approver TEXT, + diff_hash BLOB, + detail TEXT, + files_touched TEXT NOT NULL, + channel TEXT, + thread_id TEXT, + submitted_at TEXT NOT NULL, + decided_at TEXT NOT NULL, + recorded_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')) +); + +CREATE INDEX IF NOT EXISTS main.ward_audit_familiar_idx ON ward_audit (familiar_id, recorded_at); +CREATE INDEX IF NOT EXISTS main.ward_audit_event_idx ON ward_audit (event_type, recorded_at); + +CREATE TRIGGER IF NOT EXISTS main.ward_audit_append_only_update +BEFORE UPDATE ON ward_audit +BEGIN + SELECT RAISE(ABORT, 'ward_audit is append-only (RFC-0001 §5.6)'); +END; + +CREATE TRIGGER IF NOT EXISTS main.ward_audit_append_only_delete +BEFORE DELETE ON ward_audit +BEGIN + SELECT RAISE(ABORT, 'ward_audit is append-only (RFC-0001 §5.6)'); +END; +"# + }; +} + +/// DDL for the durable `main.ward_audit` table inside `coven.sqlite3` (§3.4). +/// +/// See module docs for the full schema-state contract. This `BEGIN IMMEDIATE` +/// transaction is safe to run unconditionally on every store open: it permits +/// only exact `missing` or `current_v020` before any mutation, uses idempotent +/// `IF NOT EXISTS` DDL for daemon compatibility, then requires exact +/// `current_v020` before `COMMIT`. Exact `legacy_v013`, every drifted +/// `unknown` shape, every unexpected durable reserved-namespace object, and +/// every temp shadow/reserved temp object fail closed. The IMMEDIATE +/// reservation serializes concurrent initializers before any guard read so the +/// loser waits, re-runs the guard against committed state, and sees exact +/// `current_v020` rather than racing into `sqlite_master` lock errors. Durable +/// schema objects are explicitly created in `main`; the temp guard tables live +/// in `temp` under unique non-reserved names. If this SQL errors, callers must +/// explicitly `ROLLBACK` before continuing so SQLite discards any uncommitted +/// work. This DDL never mutates database-wide `PRAGMA user_version`. +pub const WARD_AUDIT_SCHEMA_SQL: &str = concat!( + r#" +BEGIN IMMEDIATE; + +CREATE TEMP TABLE coven_threads_ward_audit_schema_pre_guard ( + ok INTEGER NOT NULL CHECK (ok = 1) +); + +INSERT INTO coven_threads_ward_audit_schema_pre_guard (ok) +"#, + ward_audit_schema_state_ctes_sql!(), + r#" +SELECT CASE + WHEN ("#, + ward_audit_schema_state_case_sql!(), + r#") IN ('missing', 'current_v020') THEN 1 + ELSE 0 +END +FROM ward_audit_shape; + +DROP TABLE coven_threads_ward_audit_schema_pre_guard; +"#, + ward_audit_current_objects_sql!(), + r#" +CREATE TEMP TABLE coven_threads_ward_audit_schema_post_guard ( + ok INTEGER NOT NULL CHECK (ok = 1) +); + +INSERT INTO coven_threads_ward_audit_schema_post_guard (ok) +"#, + ward_audit_schema_state_ctes_sql!(), + r#" +SELECT CASE + WHEN ("#, + ward_audit_schema_state_case_sql!(), + r#") = 'current_v020' THEN 1 + ELSE 0 +END +FROM ward_audit_shape; + +DROP TABLE coven_threads_ward_audit_schema_post_guard; + +COMMIT; +"#, +); + +#[cfg(test)] +mod tests { + use super::*; + use crate::ids::{SurfaceId, WriterId}; + use rusqlite::{params, Connection}; + use std::{ + collections::BTreeSet, + fs, + io::ErrorKind, + path::{Path, PathBuf}, + sync::{Arc, Barrier}, + thread, + time::Duration, + }; + use uuid::Uuid; + + const FIXED_SUBMITTED_AT: &str = "2026-07-19T00:00:00.000Z"; + const FIXED_DECIDED_AT: &str = "2026-07-19T00:01:00.000Z"; + const FIXED_RECORDED_AT: &str = "2026-07-19T00:02:00.000Z"; + const FIXED_WARD_HASH: [u8; 32] = *b"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + const FIXED_DIFF_HASH: [u8; 32] = *b"bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; + const FIXED_PREV_DETAIL: &str = r#"{"prev_sha256":"cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc","bytes_written":42}"#; + const FIXED_FILES_TOUCHED: &str = r#"["SOUL.md"]"#; + const CONCURRENT_DB_RUNS: usize = 3; + + /// Exact shipped v0.1.3 `ward_audit` DDL from `origin/main` / the PR base. + /// Keep the inline comments: SQLite preserves them in `sqlite_master.sql`, + /// and the legacy fingerprint intentionally matches that stored text. + const LEGACY_WARD_AUDIT_SCHEMA_SQL: &str = r#" CREATE TABLE IF NOT EXISTS ward_audit ( id INTEGER PRIMARY KEY AUTOINCREMENT, event_type TEXT NOT NULL CHECK (event_type IN ( @@ -202,73 +1000,831 @@ BEGIN END; "#; -#[cfg(test)] -mod tests { - use super::*; - use crate::ids::{SurfaceId, WriterId}; + const EXPECTED_EXPLICIT_INDEX_NAMES: &[&str] = + &["ward_audit_event_idx", "ward_audit_familiar_idx"]; + const EXPECTED_TRIGGER_NAMES: &[&str] = &[ + "ward_audit_append_only_delete", + "ward_audit_append_only_update", + ]; - fn request() -> MutationRequest { - MutationRequest { - surface: SurfaceId::new("SOUL.md"), - writer: WriterId::new("principal:val"), - channel: Channel::Mutation, - } + #[derive(Debug, PartialEq, Eq)] + struct StoredAuditRow { + id: i64, + event_type: String, + proposal_id: Option, + familiar_id: String, + ward_version: Option, + ward_hash: Vec, + tier: Option, + decision: String, + approver: Option, + diff_hash: Option>, + detail: Option, + files_touched: String, + channel: Option, + thread_id: Option, + submitted_at: String, + decided_at: String, + recorded_at: String, } - #[test] - fn verdict_records_carry_rfc_5_6_fields() { - let now = OffsetDateTime::now_utc(); - let record = WardAuditRecord::for_verdict( - FamiliarId::new(), - &[0xab; 32], - &request(), - &Verdict::Reject { - reason: RejectReason::UnknownSurface { - surface: SurfaceId::new("SOUL.md"), - }, - }, - now, - now, - ); - assert_eq!(record.event_type, AuditEventType::ValidationVerdict); - assert_eq!(record.decision, "reject:unknown_surface"); - assert_eq!(record.ward_hash, vec![0xab; 32]); - assert_eq!(record.files_touched, vec![SurfaceId::new("SOUL.md")]); - assert_eq!(record.channel, Some(Channel::Mutation)); + fn user_version(conn: &Connection) -> i64 { + conn.pragma_query_value(None, "user_version", |row| row.get(0)) + .unwrap() } - #[test] - fn audit_records_roundtrip_json() { - let now = OffsetDateTime::now_utc(); - let record = WardAuditRecord::for_verdict( - FamiliarId::new(), - &[1; 32], - &request(), - &Verdict::Permit { - thread: ThreadId::new(), - }, - now, - now, - ); - let json = serde_json::to_string(&record).unwrap(); - let back: WardAuditRecord = serde_json::from_str(&json).unwrap(); - assert_eq!(record, back); + fn set_user_version(conn: &Connection, version: i64) { + conn.pragma_update(None, "user_version", version).unwrap(); } - #[test] - fn schema_names_all_event_tags() { - // The CHECK constraint and the enum must not drift. - for et in [ - AuditEventType::ProposalSubmitted, - AuditEventType::ProposalApproved, - AuditEventType::ProposalRejected, - AuditEventType::ProposalVetoed, - AuditEventType::WardUpdated, - AuditEventType::ValidationVerdict, - AuditEventType::CompactionLedger, - ] { + fn concurrent_db_dir() -> PathBuf { + PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../target/ward-audit-concurrency") + } + + fn sqlite_artifact_paths(path: &Path) -> Vec { + ["", "-journal", "-wal", "-shm"] + .into_iter() + .map(|suffix| { + let mut candidate = path.as_os_str().to_os_string(); + candidate.push(suffix); + PathBuf::from(candidate) + }) + .collect() + } + + fn cleanup_sqlite_artifacts(path: &Path) { + for candidate in sqlite_artifact_paths(path) { + match fs::remove_file(&candidate) { + Ok(()) => {} + Err(err) if err.kind() == ErrorKind::NotFound => {} + Err(err) => panic!("failed to remove {}: {err}", candidate.display()), + } + } + } + + fn assert_sqlite_artifacts_absent(path: &Path) { + for candidate in sqlite_artifact_paths(path) { assert!( - WARD_AUDIT_SCHEMA_SQL.contains(&format!("'{}'", et.tag())), + !candidate.exists(), + "expected {} to be cleaned up", + candidate.display() + ); + } + } + + struct ScratchDbPath { + path: PathBuf, + } + + impl ScratchDbPath { + fn new(prefix: &str) -> Self { + let dir = concurrent_db_dir(); + fs::create_dir_all(&dir).unwrap(); + let path = dir.join(format!("{prefix}-{}.sqlite3", Uuid::new_v4())); + cleanup_sqlite_artifacts(&path); + Self { path } + } + + fn path(&self) -> &Path { + &self.path + } + } + + impl Drop for ScratchDbPath { + fn drop(&mut self) { + for candidate in sqlite_artifact_paths(&self.path) { + let _ = fs::remove_file(candidate); + } + } + } + + fn open_file_backed_connection(path: &Path) -> Connection { + let conn = Connection::open(path).unwrap(); + conn.busy_timeout(Duration::from_secs(5)).unwrap(); + conn + } + + fn run_concurrent_sql(path: &Path, sql: &'static str) -> Vec> { + let barrier = Arc::new(Barrier::new(2)); + let handles: Vec<_> = (0..2) + .map(|_| { + let barrier = Arc::clone(&barrier); + let path = path.to_path_buf(); + thread::spawn(move || { + let conn = open_file_backed_connection(&path); + conn.commit_hook(Some(|| { + thread::sleep(Duration::from_millis(150)); + false + })); + barrier.wait(); + conn.execute_batch(sql).map_err(|err| err.to_string()) + }) + }) + .collect(); + + handles + .into_iter() + .map(|handle| handle.join().unwrap()) + .collect() + } + + fn ward_audit_schema_state(conn: &Connection) -> String { + conn.query_row(WARD_AUDIT_SCHEMA_STATE_SQL, [], |row| row.get(0)) + .unwrap() + } + + fn assert_schema_state(conn: &Connection, expected: &str) { + assert_eq!(ward_audit_schema_state(conn), expected); + } + + fn sql_literal_value(conn: &Connection, literal_sql: &str) -> String { + conn.query_row(&format!("SELECT {literal_sql};"), [], |row| row.get(0)) + .unwrap() + } + + fn schema_master_table(schema: &str) -> &'static str { + match schema { + "main" => "main.sqlite_master", + "temp" => "temp.sqlite_master", + _ => panic!("unexpected schema: {schema}"), + } + } + + fn ward_audit_column_names(conn: &Connection, schema: &str) -> Vec { + let sql = + format!("SELECT name FROM pragma_table_info('ward_audit', '{schema}') ORDER BY cid;"); + let mut stmt = conn.prepare(&sql).unwrap(); + stmt.query_map([], |row| row.get::<_, String>(0)) + .unwrap() + .collect::>() + .unwrap() + } + + fn stored_table_sql_in_schema(conn: &Connection, schema: &str) -> String { + conn.query_row( + &format!( + "SELECT sql FROM {} WHERE type = 'table' AND name = 'ward_audit';", + schema_master_table(schema) + ), + [], + |row| row.get(0), + ) + .unwrap() + } + + fn stored_table_sql(conn: &Connection) -> String { + stored_table_sql_in_schema(conn, "main") + } + + fn explicit_index_sql_fingerprint(conn: &Connection) -> String { + conn.query_row( + r#" + SELECT COALESCE(group_concat(item, '||'), '') + FROM ( + SELECT printf('%s|%s', il.name, sm.sql) AS item + FROM pragma_index_list('ward_audit', 'main') AS il + JOIN main.sqlite_master AS sm + ON sm.type = 'index' AND sm.name = il.name + WHERE il.origin = 'c' AND sm.sql IS NOT NULL + ORDER BY il.name + ); + "#, + [], + |row| row.get(0), + ) + .unwrap() + } + + fn trigger_sql_fingerprint(conn: &Connection) -> String { + conn.query_row( + r#" + SELECT COALESCE(group_concat(item, '||'), '') + FROM ( + SELECT printf('%s|%s', name, sql) AS item + FROM main.sqlite_master + WHERE type = 'trigger' AND tbl_name = 'ward_audit' + ORDER BY name + ); + "#, + [], + |row| row.get(0), + ) + .unwrap() + } + + fn schema_sql_with_extra_table_constraint( + base_schema_sql: &str, + constraint_sql: &str, + ) -> String { + let marker = "\n);\n\nCREATE INDEX"; + let replacement = format!(",\n {constraint_sql}{marker}"); + let schema_sql = base_schema_sql.replacen(marker, &replacement, 1); + assert_ne!( + schema_sql, base_schema_sql, + "expected to inject {constraint_sql}" + ); + schema_sql + } + + fn table_only_sql(base_schema_sql: &str) -> String { + let marker = "\n);\n\nCREATE INDEX"; + let (table_sql, _) = base_schema_sql + .split_once(marker) + .expect("ward_audit schema SQL must contain the table/index boundary marker"); + format!("{table_sql}\n);") + } + + fn temp_shadow_table_sql(base_schema_sql: &str) -> String { + let table_sql = table_only_sql(base_schema_sql); + let shadow_sql = table_sql + .replacen( + "CREATE TABLE IF NOT EXISTS main.ward_audit (", + "CREATE TEMP TABLE ward_audit (", + 1, + ) + .replacen( + "CREATE TABLE IF NOT EXISTS ward_audit (", + "CREATE TEMP TABLE ward_audit (", + 1, + ); + assert_ne!( + shadow_sql, table_sql, + "expected to rewrite ward_audit to TEMP" + ); + shadow_sql + } + + fn create_temp_shadow_table(conn: &Connection, base_schema_sql: &str) { + conn.execute_batch(&temp_shadow_table_sql(base_schema_sql)) + .unwrap(); + } + + fn inject_sql_before_anchor(base_sql: &str, anchor: &str, injection_sql: &str) -> String { + let replacement = format!("{injection_sql}\n\n{anchor}"); + let updated = base_sql.replacen(anchor, &replacement, 1); + assert_ne!( + updated, base_sql, + "expected to inject before anchor {anchor}" + ); + updated + } + + fn migration_sql_with_durable_drift_before_post_guard(drift_sql: &str) -> String { + inject_sql_before_anchor( + WARD_AUDIT_MIGRATION_V020_SQL, + "CREATE TEMP TABLE coven_threads_ward_audit_migration_post_guard (", + drift_sql, + ) + } + + fn legacy_schema_with_extra_table_constraint(constraint_sql: &str) -> String { + schema_sql_with_extra_table_constraint(LEGACY_WARD_AUDIT_SCHEMA_SQL, constraint_sql) + } + + fn current_schema_with_extra_table_constraint(constraint_sql: &str) -> String { + schema_sql_with_extra_table_constraint(ward_audit_current_objects_sql!(), constraint_sql) + } + + fn drift_event_type_literal( + schema_sql: &str, + exact_literal: &str, + drifted_literal: &str, + ) -> String { + let exact = format!("'{exact_literal}'"); + let drifted = format!("'{drifted_literal}'"); + let drifted_schema_sql = schema_sql.replacen(&exact, &drifted, 1); + assert_ne!( + drifted_schema_sql, schema_sql, + "expected to drift {exact_literal} to {drifted_literal}" + ); + drifted_schema_sql + } + + fn expected_explicit_index_names() -> BTreeSet { + EXPECTED_EXPLICIT_INDEX_NAMES + .iter() + .map(|name| (*name).to_string()) + .collect() + } + + fn expected_trigger_names() -> BTreeSet { + EXPECTED_TRIGGER_NAMES + .iter() + .map(|name| (*name).to_string()) + .collect() + } + + fn explicit_index_names_in_schema(conn: &Connection, schema: &str) -> BTreeSet { + let sql = format!( + "SELECT name FROM pragma_index_list('ward_audit', '{schema}') WHERE origin = 'c' AND name NOT LIKE 'sqlite_autoindex_%' ORDER BY name;" + ); + let mut stmt = conn.prepare(&sql).unwrap(); + stmt.query_map([], |row| row.get::<_, String>(0)) + .unwrap() + .collect::>() + .unwrap() + } + + fn explicit_index_names(conn: &Connection) -> BTreeSet { + explicit_index_names_in_schema(conn, "main") + } + + fn trigger_names_in_schema(conn: &Connection, schema: &str) -> BTreeSet { + let mut stmt = conn + .prepare(&format!( + "SELECT name FROM {} WHERE type = 'trigger' AND tbl_name = 'ward_audit' ORDER BY name;", + schema_master_table(schema) + )) + .unwrap(); + stmt.query_map([], |row| row.get::<_, String>(0)) + .unwrap() + .collect::>() + .unwrap() + } + + fn trigger_names(conn: &Connection) -> BTreeSet { + trigger_names_in_schema(conn, "main") + } + + fn has_column_in_schema(conn: &Connection, schema: &str, name: &str) -> bool { + conn.query_row( + &format!( + "SELECT EXISTS(SELECT 1 FROM pragma_table_info('ward_audit', '{schema}') WHERE name = ?1);" + ), + params![name], + |row| row.get::<_, i64>(0), + ) + .unwrap() + == 1 + } + + fn has_column(conn: &Connection, name: &str) -> bool { + has_column_in_schema(conn, "main", name) + } + + fn schema_object_exists( + conn: &Connection, + schema: &str, + object_type: &str, + name: &str, + ) -> bool { + conn.query_row( + &format!( + "SELECT EXISTS(SELECT 1 FROM {} WHERE type = ?1 AND name = ?2);", + schema_master_table(schema) + ), + params![object_type, name], + |row| row.get::<_, i64>(0), + ) + .unwrap() + == 1 + } + + fn main_schema_object_exists(conn: &Connection, object_type: &str, name: &str) -> bool { + schema_object_exists(conn, "main", object_type, name) + } + + fn temp_schema_object_exists(conn: &Connection, object_type: &str, name: &str) -> bool { + schema_object_exists(conn, "temp", object_type, name) + } + + fn reserved_main_namespace_object_key(object_type: &str, name: &str, tbl_name: &str) -> String { + format!("{object_type}|{name}|{tbl_name}") + } + + fn reserved_main_namespace_objects(conn: &Connection) -> BTreeSet { + let mut stmt = conn + .prepare( + r#" + SELECT printf('%s|%s|%s', type, name, COALESCE(tbl_name, '')) + FROM main.sqlite_master + WHERE type IN ('table', 'index', 'trigger', 'view') + AND (lower(name) = 'ward_audit' OR lower(name) GLOB 'ward_audit_*') + ORDER BY type, name; + "#, + ) + .unwrap(); + stmt.query_map([], |row| row.get::<_, String>(0)) + .unwrap() + .collect::>() + .unwrap() + } + + fn expected_reserved_main_namespace_objects() -> BTreeSet { + BTreeSet::from([ + reserved_main_namespace_object_key("index", "ward_audit_event_idx", "ward_audit"), + reserved_main_namespace_object_key("index", "ward_audit_familiar_idx", "ward_audit"), + reserved_main_namespace_object_key("table", "ward_audit", "ward_audit"), + reserved_main_namespace_object_key( + "trigger", + "ward_audit_append_only_delete", + "ward_audit", + ), + reserved_main_namespace_object_key( + "trigger", + "ward_audit_append_only_update", + "ward_audit", + ), + ]) + } + + fn ward_audit_row_count(conn: &Connection, schema: &str) -> i64 { + conn.query_row( + &format!("SELECT COUNT(*) FROM {schema}.ward_audit;"), + [], + |row| row.get(0), + ) + .unwrap() + } + + fn assert_fresh_schema_preserves_user_version(initial_version: i64) { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, initial_version); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + + assert_eq!(user_version(&conn), initial_version); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + + let row_id = insert_current_apply_audit_row(&conn); + let row = load_audit_row(&conn, row_id); + assert_eq!(row.event_type, "apply_audit"); + assert_eq!(row.detail.as_deref(), Some(FIXED_PREV_DETAIL)); + } + + fn assert_legacy_schema_guard_rollback_preserves_state( + schema_sql: &str, + after_rollback: impl FnOnce(&Connection), + ) { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(schema_sql).unwrap(); + let row_id = insert_legacy_ward_updated_row(&conn); + let before = load_legacy_audit_row(&conn, row_id); + let before_version = user_version(&conn); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_MIGRATION_V020_SQL) + .expect_err("drifted legacy schema must fail at the migration guard"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected migration error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_legacy_audit_row(&conn, row_id), before); + assert_eq!(user_version(&conn), before_version); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + assert!(!has_column(&conn, "detail")); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + + after_rollback(&conn); + } + + fn assert_legacy_drift_guard_rollback_preserves_state( + constraint_sql: &str, + after_rollback: impl FnOnce(&Connection), + ) { + let schema_sql = legacy_schema_with_extra_table_constraint(constraint_sql); + assert_legacy_schema_guard_rollback_preserves_state(&schema_sql, after_rollback); + } + + fn load_audit_row_from_schema(conn: &Connection, schema: &str, id: i64) -> StoredAuditRow { + conn.query_row( + &format!( + r#" + SELECT id, event_type, proposal_id, familiar_id, ward_version, ward_hash, + tier, decision, approver, diff_hash, detail, files_touched, + channel, thread_id, submitted_at, decided_at, recorded_at + FROM {schema}.ward_audit + WHERE id = ?1 + "# + ), + params![id], + |row| { + Ok(StoredAuditRow { + id: row.get(0)?, + event_type: row.get(1)?, + proposal_id: row.get(2)?, + familiar_id: row.get(3)?, + ward_version: row.get(4)?, + ward_hash: row.get(5)?, + tier: row.get(6)?, + decision: row.get(7)?, + approver: row.get(8)?, + diff_hash: row.get(9)?, + detail: row.get(10)?, + files_touched: row.get(11)?, + channel: row.get(12)?, + thread_id: row.get(13)?, + submitted_at: row.get(14)?, + decided_at: row.get(15)?, + recorded_at: row.get(16)?, + }) + }, + ) + .unwrap() + } + + fn load_audit_row(conn: &Connection, id: i64) -> StoredAuditRow { + load_audit_row_from_schema(conn, "main", id) + } + + fn load_legacy_audit_row_from_schema( + conn: &Connection, + schema: &str, + id: i64, + ) -> StoredAuditRow { + conn.query_row( + &format!( + r#" + SELECT id, event_type, proposal_id, familiar_id, ward_version, ward_hash, + tier, decision, approver, diff_hash, files_touched, channel, + thread_id, submitted_at, decided_at, recorded_at + FROM {schema}.ward_audit + WHERE id = ?1 + "# + ), + params![id], + |row| { + Ok(StoredAuditRow { + id: row.get(0)?, + event_type: row.get(1)?, + proposal_id: row.get(2)?, + familiar_id: row.get(3)?, + ward_version: row.get(4)?, + ward_hash: row.get(5)?, + tier: row.get(6)?, + decision: row.get(7)?, + approver: row.get(8)?, + diff_hash: row.get(9)?, + detail: None, + files_touched: row.get(10)?, + channel: row.get(11)?, + thread_id: row.get(12)?, + submitted_at: row.get(13)?, + decided_at: row.get(14)?, + recorded_at: row.get(15)?, + }) + }, + ) + .unwrap() + } + + fn load_legacy_audit_row(conn: &Connection, id: i64) -> StoredAuditRow { + load_legacy_audit_row_from_schema(conn, "main", id) + } + + fn load_legacy_extra_value(conn: &Connection, id: i64) -> Option { + conn.query_row( + "SELECT legacy_extra FROM main.ward_audit WHERE id = ?1;", + params![id], + |row| row.get(0), + ) + .unwrap() + } + + fn ward_audit_new_conflict_value(conn: &Connection) -> String { + conn.query_row( + "SELECT conflict FROM ward_audit_new ORDER BY rowid LIMIT 1;", + [], + |row| row.get(0), + ) + .unwrap() + } + + fn insert_current_apply_audit_row_into_schema(conn: &Connection, schema: &str) -> i64 { + conn.execute( + &format!( + r#" + INSERT INTO {schema}.ward_audit ( + event_type, proposal_id, familiar_id, ward_version, ward_hash, + tier, decision, approver, diff_hash, detail, files_touched, + channel, thread_id, submitted_at, decided_at, recorded_at + ) VALUES ( + ?1, ?2, ?3, ?4, ?5, + ?6, ?7, ?8, ?9, ?10, ?11, + ?12, ?13, ?14, ?15, ?16 + ) + "# + ), + params![ + "apply_audit", + Some("proposal-current"), + "familiar-current", + Some("0.2.0"), + FIXED_WARD_HASH.as_ref(), + Some("tier_2"), + "applied", + Option::::None, + Some(FIXED_DIFF_HASH.as_ref()), + FIXED_PREV_DETAIL, + FIXED_FILES_TOUCHED, + Some("mutation"), + Option::::None, + FIXED_SUBMITTED_AT, + FIXED_DECIDED_AT, + FIXED_RECORDED_AT, + ], + ) + .unwrap(); + conn.last_insert_rowid() + } + + fn insert_current_apply_audit_row(conn: &Connection) -> i64 { + insert_current_apply_audit_row_into_schema(conn, "main") + } + + fn insert_current_apply_audit_row_unqualified(conn: &Connection) -> i64 { + conn.execute( + r#" + INSERT INTO ward_audit ( + event_type, proposal_id, familiar_id, ward_version, ward_hash, + tier, decision, approver, diff_hash, detail, files_touched, + channel, thread_id, submitted_at, decided_at, recorded_at + ) VALUES ( + ?1, ?2, ?3, ?4, ?5, + ?6, ?7, ?8, ?9, ?10, ?11, + ?12, ?13, ?14, ?15, ?16 + ) + "#, + params![ + "apply_audit", + Some("proposal-current"), + "familiar-current", + Some("0.2.0"), + FIXED_WARD_HASH.as_ref(), + Some("tier_2"), + "applied", + Option::::None, + Some(FIXED_DIFF_HASH.as_ref()), + FIXED_PREV_DETAIL, + FIXED_FILES_TOUCHED, + Some("mutation"), + Option::::None, + FIXED_SUBMITTED_AT, + FIXED_DECIDED_AT, + FIXED_RECORDED_AT, + ], + ) + .unwrap(); + conn.last_insert_rowid() + } + + fn try_insert_legacy_ward_updated_row_into_schema( + conn: &Connection, + schema: &str, + decision: &str, + recorded_at: &str, + ) -> rusqlite::Result { + conn.execute( + &format!( + r#" + INSERT INTO {schema}.ward_audit ( + event_type, proposal_id, familiar_id, ward_version, ward_hash, + tier, decision, approver, diff_hash, files_touched, channel, + thread_id, submitted_at, decided_at, recorded_at + ) VALUES ( + ?1, ?2, ?3, ?4, ?5, + ?6, ?7, ?8, ?9, ?10, ?11, + ?12, ?13, ?14, ?15 + ) + "# + ), + params![ + "ward_updated", + Some("proposal-legacy"), + "familiar-legacy", + Some("0.1.3"), + FIXED_WARD_HASH.as_ref(), + Some("tier_1"), + decision, + Some("writer:legacy"), + Some(FIXED_DIFF_HASH.as_ref()), + FIXED_FILES_TOUCHED, + Some("mutation"), + Some("thread-legacy"), + FIXED_SUBMITTED_AT, + FIXED_DECIDED_AT, + recorded_at, + ], + )?; + Ok(conn.last_insert_rowid()) + } + + fn try_insert_legacy_ward_updated_row( + conn: &Connection, + decision: &str, + recorded_at: &str, + ) -> rusqlite::Result { + try_insert_legacy_ward_updated_row_into_schema(conn, "main", decision, recorded_at) + } + + fn insert_legacy_ward_updated_row(conn: &Connection) -> i64 { + try_insert_legacy_ward_updated_row(conn, "updated", FIXED_RECORDED_AT).unwrap() + } + + fn insert_legacy_ward_updated_row_with_extra(conn: &Connection, extra: &str) -> i64 { + conn.execute( + r#" + INSERT INTO main.ward_audit ( + event_type, proposal_id, familiar_id, ward_version, ward_hash, + tier, decision, approver, diff_hash, files_touched, channel, + thread_id, submitted_at, decided_at, recorded_at, legacy_extra + ) VALUES ( + ?1, ?2, ?3, ?4, ?5, + ?6, ?7, ?8, ?9, ?10, ?11, + ?12, ?13, ?14, ?15, ?16 + ) + "#, + params![ + "ward_updated", + Some("proposal-legacy"), + "familiar-legacy", + Some("0.1.3"), + FIXED_WARD_HASH.as_ref(), + Some("tier_1"), + "updated", + Some("writer:legacy"), + Some(FIXED_DIFF_HASH.as_ref()), + FIXED_FILES_TOUCHED, + Some("mutation"), + Some("thread-legacy"), + FIXED_SUBMITTED_AT, + FIXED_DECIDED_AT, + FIXED_RECORDED_AT, + extra, + ], + ) + .unwrap(); + conn.last_insert_rowid() + } + + fn request() -> MutationRequest { + MutationRequest { + surface: SurfaceId::new("SOUL.md"), + writer: WriterId::new("principal:val"), + channel: Channel::Mutation, + } + } + + #[test] + fn verdict_records_carry_rfc_5_6_fields() { + let now = OffsetDateTime::now_utc(); + let record = WardAuditRecord::for_verdict( + FamiliarId::new(), + &[0xab; 32], + &request(), + &Verdict::Reject { + reason: RejectReason::UnknownSurface { + surface: SurfaceId::new("SOUL.md"), + }, + }, + now, + now, + ); + assert_eq!(record.event_type, AuditEventType::ValidationVerdict); + assert_eq!(record.decision, "reject:unknown_surface"); + assert_eq!(record.ward_hash, vec![0xab; 32]); + assert_eq!(record.files_touched, vec![SurfaceId::new("SOUL.md")]); + assert_eq!(record.channel, Some(Channel::Mutation)); + } + + #[test] + fn audit_records_roundtrip_json() { + let now = OffsetDateTime::now_utc(); + let record = WardAuditRecord::for_verdict( + FamiliarId::new(), + &[1; 32], + &request(), + &Verdict::Permit { + thread: ThreadId::new(), + }, + now, + now, + ); + let json = serde_json::to_string(&record).unwrap(); + let back: WardAuditRecord = serde_json::from_str(&json).unwrap(); + assert_eq!(record, back); + } + + #[test] + fn schema_names_all_event_tags() { + // The CHECK constraint and the enum must not drift. + for et in [ + AuditEventType::ProposalSubmitted, + AuditEventType::ProposalApproved, + AuditEventType::ProposalRejected, + AuditEventType::ProposalVetoed, + AuditEventType::WardUpdated, + AuditEventType::ValidationVerdict, + AuditEventType::CompactionLedger, + AuditEventType::ApplyAudit, + ] { + assert!( + WARD_AUDIT_SCHEMA_SQL.contains(&format!("'{}'", et.tag())), "schema CHECK is missing event tag {}", et.tag() ); @@ -284,4 +1840,1331 @@ mod tests { assert!(WARD_AUDIT_SCHEMA_SQL.contains("ward_audit_append_only_delete")); assert!(WARD_AUDIT_SCHEMA_SQL.contains("append-only (RFC-0001 §5.6)")); } + + #[test] + fn schema_and_migration_sql_use_begin_immediate() { + for (label, sql) in [ + ("schema", WARD_AUDIT_SCHEMA_SQL), + ("migration", WARD_AUDIT_MIGRATION_V020_SQL), + ] { + assert!( + sql.trim_start().starts_with("BEGIN IMMEDIATE;"), + "{label} SQL must reserve the main database before guard reads" + ); + } + } + + #[test] + fn migration_sql_uses_distinct_pre_and_post_guards_before_commit() { + let pre_guard_offset = WARD_AUDIT_MIGRATION_V020_SQL + .find("CREATE TEMP TABLE coven_threads_ward_audit_migration_guard (") + .expect("migration SQL must define a precondition guard"); + let post_guard_offset = WARD_AUDIT_MIGRATION_V020_SQL + .find("CREATE TEMP TABLE coven_threads_ward_audit_migration_post_guard (") + .expect("migration SQL must define a postcondition guard"); + let post_guard_drop_offset = WARD_AUDIT_MIGRATION_V020_SQL + .find("DROP TABLE coven_threads_ward_audit_migration_post_guard;") + .expect("migration SQL must drop the postcondition guard before commit"); + let commit_offset = WARD_AUDIT_MIGRATION_V020_SQL + .rfind("COMMIT;") + .expect("migration SQL must commit on success"); + + assert!( + pre_guard_offset < post_guard_offset, + "postcondition guard must run after the legacy precondition guard" + ); + assert!( + post_guard_offset < post_guard_drop_offset, + "postcondition guard must be dropped on the success path" + ); + assert!( + post_guard_drop_offset < commit_offset, + "postcondition guard must run before COMMIT" + ); + } + + #[test] + fn for_apply_produces_correct_shape() { + let now = OffsetDateTime::now_utc(); + let prev = [0xaa_u8; 32]; + let next = [0xbb_u8; 32]; + let record = WardAuditRecord::for_apply( + FamiliarId::new(), + &[0xcc; 32], + SurfaceId::new("SOUL.md"), + "tier_2", + Some(&prev), + Some(&next), + 42, + Some(Channel::Mutation), + now, + now, + ); + assert_eq!(record.event_type, AuditEventType::ApplyAudit); + assert_eq!(record.decision, "applied"); + assert_eq!(record.tier.as_deref(), Some("tier_2")); + assert_eq!(record.diff_hash, Some(vec![0xbb; 32])); + assert_eq!(record.files_touched, vec![SurfaceId::new("SOUL.md")]); + + // detail decoding helpers + let prev_hex = record.apply_prev_sha256_hex().unwrap(); + assert_eq!(prev_hex.len(), 64); + assert!(prev_hex.chars().all(|c| c.is_ascii_hexdigit())); + assert_eq!(record.apply_bytes_written(), Some(42)); + } + + #[test] + fn for_apply_roundtrips_json() { + let now = OffsetDateTime::now_utc(); + let record = WardAuditRecord::for_apply( + FamiliarId::new(), + &[1; 32], + SurfaceId::new("SOUL.md"), + "tier_2", + None, + None, + 0, + None, + now, + now, + ); + let json = serde_json::to_string(&record).unwrap(); + let back: WardAuditRecord = serde_json::from_str(&json).unwrap(); + assert_eq!(record, back); + } + + #[test] + fn schema_state_query_returns_missing_on_empty_db() { + let conn = Connection::open_in_memory().unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_MISSING); + } + + #[test] + fn schema_qualified_table_valued_pragmas_resolve_main_and_temp_separately() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + conn.execute_batch( + "CREATE TEMP TABLE ward_audit ( + id INTEGER PRIMARY KEY, + shadow TEXT NOT NULL + ); + CREATE INDEX temp.temp_shadow_idx ON ward_audit (shadow);", + ) + .unwrap(); + + assert_eq!( + ward_audit_column_names(&conn, "main"), + vec![ + "id".to_string(), + "event_type".to_string(), + "proposal_id".to_string(), + "familiar_id".to_string(), + "ward_version".to_string(), + "ward_hash".to_string(), + "tier".to_string(), + "decision".to_string(), + "approver".to_string(), + "diff_hash".to_string(), + "detail".to_string(), + "files_touched".to_string(), + "channel".to_string(), + "thread_id".to_string(), + "submitted_at".to_string(), + "decided_at".to_string(), + "recorded_at".to_string(), + ] + ); + assert_eq!( + ward_audit_column_names(&conn, "temp"), + vec!["id".to_string(), "shadow".to_string()] + ); + assert_eq!( + explicit_index_names_in_schema(&conn, "main"), + expected_explicit_index_names() + ); + assert_eq!( + explicit_index_names_in_schema(&conn, "temp"), + BTreeSet::from(["temp_shadow_idx".to_string()]) + ); + } + + #[test] + fn current_main_with_temp_shadow_is_unknown_and_guards_preserve_main_and_temp_rows() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + let main_row_id = insert_current_apply_audit_row(&conn); + let main_before = load_audit_row(&conn, main_row_id); + + create_temp_shadow_table(&conn, ward_audit_current_objects_sql!()); + let temp_row_id = insert_current_apply_audit_row_into_schema(&conn, "temp"); + let temp_before = load_audit_row_from_schema(&conn, "temp", temp_row_id); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let init_err = conn + .execute_batch(WARD_AUDIT_SCHEMA_SQL) + .expect_err("temp shadow must make schema init fail closed"); + assert!( + init_err.to_string().contains("CHECK constraint failed"), + "unexpected schema init error: {init_err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_audit_row(&conn, main_row_id), main_before); + assert_eq!( + load_audit_row_from_schema(&conn, "temp", temp_row_id), + temp_before + ); + assert_eq!(ward_audit_row_count(&conn, "main"), 1); + assert_eq!(ward_audit_row_count(&conn, "temp"), 1); + assert!(temp_schema_object_exists(&conn, "table", "ward_audit")); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let migration_err = conn + .execute_batch(WARD_AUDIT_MIGRATION_V020_SQL) + .expect_err("temp shadow must make migration fail closed"); + assert!( + migration_err + .to_string() + .contains("CHECK constraint failed"), + "unexpected migration error: {migration_err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_audit_row(&conn, main_row_id), main_before); + assert_eq!( + load_audit_row_from_schema(&conn, "temp", temp_row_id), + temp_before + ); + assert_eq!(ward_audit_row_count(&conn, "main"), 1); + assert_eq!(ward_audit_row_count(&conn, "temp"), 1); + assert!(temp_schema_object_exists(&conn, "table", "ward_audit")); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + conn.execute_batch("DROP TABLE temp.ward_audit;").unwrap(); + assert!(!temp_schema_object_exists(&conn, "table", "ward_audit")); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!(load_audit_row(&conn, main_row_id), main_before); + } + + #[test] + fn missing_main_with_temp_ward_audit_shadow_is_unknown_and_schema_sql_rejects_without_creating_main( + ) { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 23); + create_temp_shadow_table(&conn, ward_audit_current_objects_sql!()); + let temp_row_id = insert_current_apply_audit_row_into_schema(&conn, "temp"); + let temp_before = load_audit_row_from_schema(&conn, "temp", temp_row_id); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_SCHEMA_SQL) + .expect_err("temp ward_audit shadow must block schema init before main creation"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected schema init error: {err}" + ); + assert!(!main_schema_object_exists(&conn, "table", "ward_audit")); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert!(!main_schema_object_exists(&conn, "table", "ward_audit")); + assert_eq!( + load_audit_row_from_schema(&conn, "temp", temp_row_id), + temp_before + ); + assert_eq!(ward_audit_row_count(&conn, "temp"), 1); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + assert_eq!(user_version(&conn), 23); + + conn.execute_batch("DROP TABLE temp.ward_audit;").unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_MISSING); + } + + #[test] + fn missing_main_with_reserved_temp_objects_is_unknown_and_schema_sql_preserves_temp_objects() { + for (label, setup_sql, object_type, object_name) in [ + ( + "view", + "CREATE TEMP VIEW ward_audit_shadow_view AS SELECT 1 AS sentinel;", + "view", + "ward_audit_shadow_view", + ), + ( + "index", + "CREATE TEMP TABLE other (id INTEGER PRIMARY KEY, note TEXT NOT NULL); + CREATE INDEX temp.ward_audit_shadow_idx ON other (note);", + "index", + "ward_audit_shadow_idx", + ), + ( + "trigger", + "CREATE TEMP TABLE other (id INTEGER PRIMARY KEY, note TEXT NOT NULL); + CREATE TRIGGER temp.ward_audit_shadow_trigger + BEFORE UPDATE ON other + BEGIN + SELECT RAISE(ABORT, 'other is append-only'); + END;", + "trigger", + "ward_audit_shadow_trigger", + ), + ] { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(setup_sql).unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_SCHEMA_SQL) + .expect_err("reserved temp object must block schema init"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected schema init error for {label}: {err}" + ); + assert!( + !main_schema_object_exists(&conn, "table", "ward_audit"), + "schema init for {label} must not create main.ward_audit before rollback" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert!( + temp_schema_object_exists(&conn, object_type, object_name), + "{label} temp object should survive rollback" + ); + assert!(!main_schema_object_exists(&conn, "table", "ward_audit")); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + } + + #[test] + fn fresh_schema_sql_initializes_current_schema_atomically_and_enforces_append_only() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 11); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_MISSING); + + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!(user_version(&conn), 11); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + + let row_id = insert_current_apply_audit_row(&conn); + let update_err = conn + .execute( + "UPDATE ward_audit SET decision = 'changed' WHERE id = ?1;", + params![row_id], + ) + .unwrap_err(); + assert!(update_err.to_string().contains("append-only")); + + let delete_err = conn + .execute("DELETE FROM ward_audit WHERE id = ?1;", params![row_id]) + .unwrap_err(); + assert!(delete_err.to_string().contains("append-only")); + } + + #[test] + fn exact_legacy_fixture_returns_legacy_v013() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + assert_eq!( + reserved_main_namespace_objects(&conn), + expected_reserved_main_namespace_objects() + ); + assert_eq!( + stored_table_sql(&conn), + sql_literal_value(&conn, ward_audit_exact_legacy_table_sql_sql!()) + ); + assert_eq!( + explicit_index_sql_fingerprint(&conn), + sql_literal_value(&conn, ward_audit_exact_explicit_index_fp_sql!()) + ); + assert_eq!( + trigger_sql_fingerprint(&conn), + sql_literal_value(&conn, ward_audit_exact_trigger_fp_sql!()) + ); + } + + #[test] + fn exact_current_schema_returns_current_v020() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!( + reserved_main_namespace_objects(&conn), + expected_reserved_main_namespace_objects() + ); + assert_eq!( + stored_table_sql(&conn), + sql_literal_value(&conn, ward_audit_exact_current_fresh_table_sql_sql!()) + ); + assert_eq!( + explicit_index_sql_fingerprint(&conn), + sql_literal_value(&conn, ward_audit_exact_explicit_index_fp_sql!()) + ); + assert_eq!( + trigger_sql_fingerprint(&conn), + sql_literal_value(&conn, ward_audit_exact_trigger_fp_sql!()) + ); + } + + #[test] + fn current_schema_sql_reruns_idempotently_and_preserves_rows_and_objects() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 29); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_current_apply_audit_row(&conn); + let before = load_audit_row(&conn, row_id); + let before_table_sql = stored_table_sql(&conn); + let before_indexes = explicit_index_names(&conn); + let before_triggers = trigger_names(&conn); + + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + + assert_eq!(load_audit_row(&conn, row_id), before); + assert_eq!(stored_table_sql(&conn), before_table_sql); + assert_eq!(explicit_index_names(&conn), before_indexes); + assert_eq!(trigger_names(&conn), before_triggers); + assert_eq!(user_version(&conn), 29); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + } + + #[test] + fn fresh_and_migrated_current_schemas_use_controlled_exact_sql_variants() { + let fresh = Connection::open_in_memory().unwrap(); + fresh.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + + let migrated = Connection::open_in_memory().unwrap(); + migrated + .execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL) + .unwrap(); + migrated + .execute_batch(WARD_AUDIT_MIGRATION_V020_SQL) + .unwrap(); + + assert_schema_state(&fresh, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_schema_state(&migrated, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!( + stored_table_sql(&fresh), + sql_literal_value(&fresh, ward_audit_exact_current_fresh_table_sql_sql!()) + ); + assert_eq!( + stored_table_sql(&migrated), + sql_literal_value( + &migrated, + ward_audit_exact_current_migrated_table_sql_sql!() + ) + ); + assert_ne!(stored_table_sql(&fresh), stored_table_sql(&migrated)); + assert_eq!( + explicit_index_sql_fingerprint(&fresh), + sql_literal_value(&fresh, ward_audit_exact_explicit_index_fp_sql!()) + ); + assert_eq!( + explicit_index_sql_fingerprint(&migrated), + sql_literal_value(&migrated, ward_audit_exact_explicit_index_fp_sql!()) + ); + assert_eq!( + trigger_sql_fingerprint(&fresh), + sql_literal_value(&fresh, ward_audit_exact_trigger_fp_sql!()) + ); + assert_eq!( + trigger_sql_fingerprint(&migrated), + sql_literal_value(&migrated, ward_audit_exact_trigger_fp_sql!()) + ); + } + + #[test] + fn current_schema_with_spaced_event_type_literal_is_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(&drift_event_type_literal( + ward_audit_current_objects_sql!(), + "apply_audit", + "apply_ audit", + )) + .unwrap(); + + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + assert!(stored_table_sql(&conn).contains("'apply_ audit'")); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn legacy_main_with_temp_shadow_is_unknown_and_migration_rejects_before_mutating_either_schema() + { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + let main_row_id = insert_legacy_ward_updated_row(&conn); + let main_before = load_legacy_audit_row(&conn, main_row_id); + + create_temp_shadow_table(&conn, LEGACY_WARD_AUDIT_SCHEMA_SQL); + let temp_row_id = try_insert_legacy_ward_updated_row_into_schema( + &conn, + "temp", + "updated", + FIXED_RECORDED_AT, + ) + .unwrap(); + let temp_before = load_legacy_audit_row_from_schema(&conn, "temp", temp_row_id); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_MIGRATION_V020_SQL) + .expect_err("temp shadow must make legacy migration fail before mutation"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected migration error: {err}" + ); + assert!(!has_column_in_schema(&conn, "main", "detail")); + assert!(!has_column_in_schema(&conn, "temp", "detail")); + assert!(!main_schema_object_exists(&conn, "table", "ward_audit_new")); + assert!(temp_schema_object_exists(&conn, "table", "ward_audit")); + + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_legacy_audit_row(&conn, main_row_id), main_before); + assert_eq!( + load_legacy_audit_row_from_schema(&conn, "temp", temp_row_id), + temp_before + ); + assert_eq!(user_version(&conn), 37); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + conn.execute_batch("DROP TABLE temp.ward_audit;").unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + } + + #[test] + fn legacy_schema_with_spaced_event_type_literal_is_unknown_and_guard_preserves_state() { + let schema_sql = drift_event_type_literal( + LEGACY_WARD_AUDIT_SCHEMA_SQL, + "compaction_ledger", + "compaction_ ledger", + ); + assert_legacy_schema_guard_rollback_preserves_state(&schema_sql, |conn| { + assert!(stored_table_sql(conn).contains("'compaction_ ledger'")); + assert!(!stored_table_sql(conn).contains("'compaction_ledger'")); + }); + } + + #[test] + fn legacy_schema_sql_rejects_and_rollback_preserves_state() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_legacy_ward_updated_row(&conn); + let before = load_legacy_audit_row(&conn, row_id); + let before_version = user_version(&conn); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + + let err = conn + .execute_batch(WARD_AUDIT_SCHEMA_SQL) + .expect_err("exact legacy schema must fail closed at the schema-init guard"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected schema init error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_legacy_audit_row(&conn, row_id), before); + assert_eq!(user_version(&conn), before_version); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + assert!(!has_column(&conn, "detail")); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + } + + #[test] + fn fresh_schema_preserves_user_version_zero_and_creates_current_shape() { + assert_fresh_schema_preserves_user_version(0); + } + + #[test] + fn fresh_schema_preserves_user_version_ninety_nine_and_creates_current_shape() { + assert_fresh_schema_preserves_user_version(99); + } + + #[test] + fn legacy_plus_extra_column_and_data_is_unknown_and_guard_preserves_state() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + conn.execute_batch("ALTER TABLE ward_audit ADD COLUMN legacy_extra TEXT;") + .unwrap(); + let row_id = insert_legacy_ward_updated_row_with_extra(&conn, "survivor"); + let before = load_legacy_audit_row(&conn, row_id); + let before_extra = load_legacy_extra_value(&conn, row_id); + let before_version = user_version(&conn); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_MIGRATION_V020_SQL) + .expect_err("partial legacy schema must fail at the migration guard"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected migration error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_legacy_audit_row(&conn, row_id), before); + assert_eq!(load_legacy_extra_value(&conn, row_id), before_extra); + assert_eq!(user_version(&conn), before_version); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + assert!(has_column(&conn, "legacy_extra")); + assert!(!has_column(&conn, "detail")); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + } + + #[test] + fn legacy_schema_with_extra_table_check_is_unknown_and_guard_preserves_constraint() { + assert_legacy_drift_guard_rollback_preserves_state( + "CHECK (length(decision) > 0)", + |conn| { + let err = try_insert_legacy_ward_updated_row(conn, "", FIXED_RECORDED_AT) + .expect_err( + "legacy CHECK drift must still reject empty decisions after rollback", + ); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected post-rollback CHECK error: {err}" + ); + }, + ); + } + + #[test] + fn legacy_schema_with_extra_unique_is_unknown_and_guard_preserves_constraint() { + assert_legacy_drift_guard_rollback_preserves_state( + "UNIQUE (decision, recorded_at)", + |conn| { + let err = try_insert_legacy_ward_updated_row(conn, "updated", FIXED_RECORDED_AT) + .expect_err( + "legacy UNIQUE drift must still reject duplicate decision/recorded_at rows after rollback", + ); + assert!( + err.to_string().contains("UNIQUE constraint failed"), + "unexpected post-rollback UNIQUE error: {err}" + ); + }, + ); + } + + #[test] + fn current_schema_with_extra_table_check_is_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(¤t_schema_with_extra_table_constraint( + "CHECK (length(decision) > 0)", + )) + .unwrap(); + + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn current_schema_with_extra_unique_is_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(¤t_schema_with_extra_table_constraint( + "UNIQUE (decision, recorded_at)", + )) + .unwrap(); + + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn current_schema_missing_append_only_trigger_is_unknown_and_update_succeeds() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + let row_id = insert_current_apply_audit_row(&conn); + + conn.execute_batch("DROP TRIGGER ward_audit_append_only_update;") + .unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + conn.execute( + "UPDATE ward_audit SET decision = 'mutated' WHERE id = ?1;", + params![row_id], + ) + .unwrap(); + assert_eq!(load_audit_row(&conn, row_id).decision, "mutated"); + } + + #[test] + fn current_schema_with_extra_index_is_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + + conn.execute_batch("CREATE INDEX ward_audit_decision_idx ON ward_audit (decision);") + .unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn current_schema_with_desc_index_is_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + + conn.execute_batch( + "DROP INDEX ward_audit_event_idx; + CREATE INDEX ward_audit_event_idx ON ward_audit (event_type, recorded_at DESC);", + ) + .unwrap(); + + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert!( + explicit_index_sql_fingerprint(&conn).contains("recorded_at DESC"), + "expected DESC drift in index SQL" + ); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn current_schema_with_collated_index_is_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + + conn.execute_batch( + "DROP INDEX ward_audit_event_idx; + CREATE INDEX ward_audit_event_idx ON ward_audit (event_type COLLATE NOCASE, recorded_at);", + ) + .unwrap(); + + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert!( + explicit_index_sql_fingerprint(&conn).contains("COLLATE NOCASE"), + "expected collation drift in index SQL" + ); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn current_schema_with_extra_reserved_main_view_or_table_is_unknown_and_schema_sql_preserves_state( + ) { + for (label, setup_sql, object_type, object_name, object_row_value) in [ + ( + "view", + "CREATE VIEW ward_audit_history AS SELECT id, decision FROM main.ward_audit;", + "view", + "ward_audit_history", + None, + ), + ( + "table", + "CREATE TABLE ward_audit_backup (id INTEGER PRIMARY KEY, note TEXT NOT NULL); + INSERT INTO ward_audit_backup (id, note) VALUES (1, 'sentinel');", + "table", + "ward_audit_backup", + Some("sentinel"), + ), + ] { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 41); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_current_apply_audit_row(&conn); + let before = load_audit_row(&conn, row_id); + + conn.execute_batch(setup_sql).unwrap(); + + let reserved_objects = reserved_main_namespace_objects(&conn); + assert!( + expected_reserved_main_namespace_objects().is_subset(&reserved_objects), + "{label} drift must preserve the expected durable whitelist objects" + ); + assert!( + reserved_objects.contains(&reserved_main_namespace_object_key( + object_type, + object_name, + object_name + )), + "{label} drift should register the extra durable namespace object" + ); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_SCHEMA_SQL) + .expect_err("extra reserved durable object must make schema init fail closed"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected schema init error for {label}: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_audit_row(&conn, row_id), before); + assert_eq!(user_version(&conn), 41); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + assert!(main_schema_object_exists(&conn, object_type, object_name)); + if let Some(expected_note) = object_row_value { + assert_eq!( + conn.query_row( + "SELECT note FROM ward_audit_backup WHERE id = 1;", + [], + |row| row.get::<_, String>(0) + ) + .unwrap(), + expected_note + ); + } + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + } + + #[test] + fn current_schema_with_altered_trigger_error_literal_is_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + + conn.execute_batch( + "DROP TRIGGER ward_audit_append_only_update; + CREATE TRIGGER ward_audit_append_only_update + BEFORE UPDATE ON ward_audit + BEGIN + SELECT RAISE(ABORT, 'ward_audit is append-only (drifted)'); + END;", + ) + .unwrap(); + + assert_eq!(trigger_names(&conn), expected_trigger_names()); + assert!( + trigger_sql_fingerprint(&conn).contains("drifted"), + "expected trigger-literal drift in trigger SQL" + ); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn current_schema_with_altered_trigger_body_is_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + + conn.execute_batch( + "DROP TRIGGER ward_audit_append_only_delete; + CREATE TRIGGER ward_audit_append_only_delete + BEFORE DELETE ON ward_audit + BEGIN + SELECT 1; + SELECT RAISE(ABORT, 'ward_audit is append-only (RFC-0001 §5.6)'); + END;", + ) + .unwrap(); + + assert_eq!(trigger_names(&conn), expected_trigger_names()); + assert!( + trigger_sql_fingerprint(&conn).contains("SELECT 1;"), + "expected trigger-body drift in trigger SQL" + ); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn absent_ward_audit_with_reserved_index_collision_is_unknown_and_schema_sql_preserves_other_objects( + ) { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 23); + conn.execute_batch( + "CREATE TABLE other (id INTEGER PRIMARY KEY, note TEXT NOT NULL); + INSERT INTO other (id, note) VALUES (1, 'sentinel'); + CREATE INDEX ward_audit_event_idx ON other (note);", + ) + .unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_SCHEMA_SQL) + .expect_err("reserved index collision must fail closed before creating ward_audit"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected schema init error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + assert!(!main_schema_object_exists(&conn, "table", "ward_audit")); + assert!(main_schema_object_exists(&conn, "table", "other")); + assert!(main_schema_object_exists( + &conn, + "index", + "ward_audit_event_idx" + )); + assert_eq!( + conn.query_row("SELECT note FROM other WHERE id = 1;", [], |row| row + .get::<_, String>(0)) + .unwrap(), + "sentinel" + ); + assert_eq!(user_version(&conn), 23); + } + + #[test] + fn absent_ward_audit_with_reserved_trigger_collision_is_unknown_and_schema_sql_preserves_other_objects( + ) { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 31); + conn.execute_batch( + "CREATE TABLE other (id INTEGER PRIMARY KEY, note TEXT NOT NULL); + INSERT INTO other (id, note) VALUES (1, 'sentinel'); + CREATE TRIGGER ward_audit_append_only_update + BEFORE UPDATE ON other + BEGIN + SELECT RAISE(ABORT, 'other is append-only'); + END;", + ) + .unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_SCHEMA_SQL) + .expect_err("reserved trigger collision must fail closed before creating ward_audit"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected schema init error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + assert!(!main_schema_object_exists(&conn, "table", "ward_audit")); + assert!(main_schema_object_exists(&conn, "table", "other")); + assert!(main_schema_object_exists( + &conn, + "trigger", + "ward_audit_append_only_update" + )); + let trigger_err = conn + .execute("UPDATE other SET note = 'changed' WHERE id = 1;", []) + .unwrap_err(); + assert!(trigger_err.to_string().contains("other is append-only")); + assert_eq!( + conn.query_row("SELECT note FROM other WHERE id = 1;", [], |row| row + .get::<_, String>(0)) + .unwrap(), + "sentinel" + ); + assert_eq!(user_version(&conn), 31); + } + + #[test] + fn unknown_partial_current_schema_rejects_schema_sql_and_preserves_state() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_current_apply_audit_row(&conn); + let before = load_audit_row(&conn, row_id); + + conn.execute_batch("DROP TRIGGER ward_audit_append_only_update;") + .unwrap(); + let before_triggers = trigger_names(&conn); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_SCHEMA_SQL) + .expect_err("partial current schema must fail closed at the schema-init guard"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected schema init error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_audit_row(&conn, row_id), before); + assert_eq!(trigger_names(&conn), before_triggers); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(user_version(&conn), 37); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn migration_rejects_current_schema_rows_with_detail_and_preserves_state() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_current_apply_audit_row(&conn); + let before = load_audit_row(&conn, row_id); + let before_version = user_version(&conn); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + + let err = conn + .execute_batch(WARD_AUDIT_MIGRATION_V020_SQL) + .expect_err("current schema migration must fail at the legacy guard"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected migration error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_audit_row(&conn, row_id), before); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!(user_version(&conn), before_version); + } + + #[test] + fn legacy_schema_with_preexisting_main_ward_audit_new_is_unknown_and_guard_rejects_before_alter( + ) { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_legacy_ward_updated_row(&conn); + let before = load_legacy_audit_row(&conn, row_id); + conn.execute_batch( + "CREATE TABLE main.ward_audit_new (conflict TEXT NOT NULL); + INSERT INTO main.ward_audit_new (conflict) VALUES ('sentinel');", + ) + .unwrap(); + + let reserved_objects = reserved_main_namespace_objects(&conn); + assert!( + expected_reserved_main_namespace_objects().is_subset(&reserved_objects), + "legacy fixture should still include the expected durable whitelist objects" + ); + assert!( + reserved_objects.contains(&reserved_main_namespace_object_key( + "table", + "ward_audit_new", + "ward_audit_new" + )), + "preexisting ward_audit_new should register as an unexpected durable object" + ); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + let err = conn + .execute_batch(WARD_AUDIT_MIGRATION_V020_SQL) + .expect_err("preexisting ward_audit_new must make migration fail before ALTER"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected migration error: {err}" + ); + assert!(!has_column(&conn, "detail")); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_legacy_audit_row(&conn, row_id), before); + assert_eq!(user_version(&conn), 37); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + assert!(!has_column(&conn, "detail")); + assert_eq!(ward_audit_new_conflict_value(&conn), "sentinel"); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + } + + #[test] + fn legacy_schema_upgrade_passes_post_guard_and_preserves_append_only_behavior() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_legacy_ward_updated_row(&conn); + let before = load_legacy_audit_row(&conn, row_id); + assert_eq!(user_version(&conn), 37); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + + conn.execute_batch(WARD_AUDIT_MIGRATION_V020_SQL).unwrap(); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!(user_version(&conn), 37); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + + let after = load_audit_row(&conn, row_id); + assert_eq!(after.id, row_id); + assert_eq!(after.event_type, before.event_type); + assert_eq!(after.proposal_id, before.proposal_id); + assert_eq!(after.familiar_id, before.familiar_id); + assert_eq!(after.ward_version, before.ward_version); + assert_eq!(after.ward_hash, before.ward_hash); + assert_eq!(after.tier, before.tier); + assert_eq!(after.decision, before.decision); + assert_eq!(after.approver, before.approver); + assert_eq!(after.diff_hash, before.diff_hash); + assert_eq!(after.detail, None); + assert_eq!(after.files_touched, before.files_touched); + assert_eq!(after.channel, before.channel); + assert_eq!(after.thread_id, before.thread_id); + assert_eq!(after.submitted_at, before.submitted_at); + assert_eq!(after.decided_at, before.decided_at); + assert_eq!(after.recorded_at, before.recorded_at); + + let apply_row_id = insert_current_apply_audit_row(&conn); + let apply_row = load_audit_row(&conn, apply_row_id); + assert_eq!(apply_row.event_type, "apply_audit"); + assert_eq!(apply_row.detail.as_deref(), Some(FIXED_PREV_DETAIL)); + + let update_err = conn + .execute( + "UPDATE ward_audit SET decision = 'changed' WHERE id = ?1", + params![row_id], + ) + .unwrap_err(); + assert!(update_err.to_string().contains("append-only")); + + let delete_err = conn + .execute("DELETE FROM ward_audit WHERE id = ?1", params![row_id]) + .unwrap_err(); + assert!(delete_err.to_string().contains("append-only")); + } + + #[test] + fn post_guard_rejects_durable_drift_and_rollback_restores_exact_legacy_state() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_legacy_ward_updated_row(&conn); + let before = load_legacy_audit_row(&conn, row_id); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + + let drift_sql = migration_sql_with_durable_drift_before_post_guard( + "CREATE VIEW main.ward_audit_history AS SELECT id, decision FROM main.ward_audit;", + ); + let err = conn + .execute_batch(&drift_sql) + .expect_err("postcondition guard must reject durable drift before commit"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected migration error: {err}" + ); + assert!(has_column(&conn, "detail")); + assert!(main_schema_object_exists( + &conn, + "view", + "ward_audit_history" + )); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_legacy_audit_row(&conn, row_id), before); + assert_eq!(user_version(&conn), 37); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + assert!(!has_column(&conn, "detail")); + assert!(!main_schema_object_exists(&conn, "table", "ward_audit_new")); + assert!(!main_schema_object_exists( + &conn, + "view", + "ward_audit_history" + )); + assert_eq!( + stored_table_sql(&conn), + sql_literal_value(&conn, ward_audit_exact_legacy_table_sql_sql!()) + ); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + } + + #[test] + fn rerunning_migration_after_legacy_upgrade_errors_and_preserves_rows() { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + let legacy_row_id = insert_legacy_ward_updated_row(&conn); + + conn.execute_batch(WARD_AUDIT_MIGRATION_V020_SQL).unwrap(); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!(user_version(&conn), 37); + + let apply_row_id = insert_current_apply_audit_row(&conn); + let legacy_before = load_audit_row(&conn, legacy_row_id); + let apply_before = load_audit_row(&conn, apply_row_id); + + let err = conn + .execute_batch(WARD_AUDIT_MIGRATION_V020_SQL) + .expect_err("rerunning the migration must fail at the legacy guard"); + assert!( + err.to_string().contains("CHECK constraint failed"), + "unexpected migration error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_audit_row(&conn, legacy_row_id), legacy_before); + assert_eq!(load_audit_row(&conn, apply_row_id), apply_before); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!(user_version(&conn), 37); + } + + #[test] + fn concurrent_schema_initialization_serializes_without_locked_errors() { + for attempt in 0..CONCURRENT_DB_RUNS { + let path = { + let db = ScratchDbPath::new("concurrent-schema-init"); + let path = db.path().to_path_buf(); + + { + let setup = open_file_backed_connection(db.path()); + set_user_version(&setup, 41); + } + + let outcomes = run_concurrent_sql(db.path(), WARD_AUDIT_SCHEMA_SQL); + assert!( + outcomes.iter().all(|result| result.is_ok()), + "attempt {attempt} concurrent init outcomes: {outcomes:?}" + ); + + { + let final_conn = open_file_backed_connection(db.path()); + assert_eq!(user_version(&final_conn), 41); + assert_schema_state(&final_conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!( + explicit_index_names(&final_conn), + expected_explicit_index_names() + ); + assert_eq!(trigger_names(&final_conn), expected_trigger_names()); + } + + path + }; + + assert_sqlite_artifacts_absent(&path); + } + } + + #[test] + fn concurrent_legacy_migration_waits_then_rejects_current_at_guard() { + for attempt in 0..CONCURRENT_DB_RUNS { + let path = { + let db = ScratchDbPath::new("concurrent-legacy-migration"); + let path = db.path().to_path_buf(); + + let (row_id, before) = { + let setup = open_file_backed_connection(db.path()); + set_user_version(&setup, 37); + setup.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_legacy_ward_updated_row(&setup); + let before = load_legacy_audit_row(&setup, row_id); + (row_id, before) + }; + + let outcomes = run_concurrent_sql(db.path(), WARD_AUDIT_MIGRATION_V020_SQL); + let success_count = outcomes.iter().filter(|result| result.is_ok()).count(); + let errors: Vec<_> = outcomes + .iter() + .filter_map(|result| result.as_ref().err()) + .collect(); + assert_eq!( + success_count, 1, + "attempt {attempt} concurrent migration outcomes: {outcomes:?}" + ); + assert_eq!( + errors.len(), + 1, + "attempt {attempt} concurrent migration outcomes: {outcomes:?}" + ); + let error = errors[0].to_lowercase(); + assert!( + error.contains("check constraint failed"), + "attempt {attempt} unexpected migration error: {}", + errors[0] + ); + assert!( + !error.contains("locked"), + "attempt {attempt} migration must wait then fail at the guard, not lock: {}", + errors[0] + ); + + { + let final_conn = open_file_backed_connection(db.path()); + assert_eq!(user_version(&final_conn), 37); + assert_schema_state(&final_conn, WARD_AUDIT_SCHEMA_STATE_CURRENT_V020); + assert_eq!(ward_audit_row_count(&final_conn, "main"), 1); + assert_eq!(load_audit_row(&final_conn, row_id), before); + assert_eq!( + explicit_index_names(&final_conn), + expected_explicit_index_names() + ); + assert_eq!(trigger_names(&final_conn), expected_trigger_names()); + } + + path + }; + + assert_sqlite_artifacts_absent(&path); + } + } + + #[test] + fn unqualified_insert_targets_temp_shadow_while_schema_state_stays_unknown() { + let conn = Connection::open_in_memory().unwrap(); + conn.execute_batch(WARD_AUDIT_SCHEMA_SQL).unwrap(); + let main_row_id = insert_current_apply_audit_row(&conn); + let main_before = load_audit_row(&conn, main_row_id); + + create_temp_shadow_table(&conn, ward_audit_current_objects_sql!()); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + assert_eq!(ward_audit_row_count(&conn, "main"), 1); + assert_eq!(ward_audit_row_count(&conn, "temp"), 0); + + let temp_row_id = insert_current_apply_audit_row_unqualified(&conn); + + assert_eq!(load_audit_row(&conn, main_row_id), main_before); + assert_eq!(ward_audit_row_count(&conn, "main"), 1); + assert_eq!(ward_audit_row_count(&conn, "temp"), 1); + assert_eq!( + load_audit_row_from_schema(&conn, "temp", temp_row_id).event_type, + "apply_audit" + ); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_UNKNOWN); + } + + #[test] + fn sqlite_post_alter_failure_rollback_restores_legacy_schema_for_production_migration_contract() + { + let conn = Connection::open_in_memory().unwrap(); + set_user_version(&conn, 37); + conn.execute_batch(LEGACY_WARD_AUDIT_SCHEMA_SQL).unwrap(); + let row_id = insert_legacy_ward_updated_row(&conn); + let before = load_legacy_audit_row(&conn, row_id); + + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + + // Validate SQLite rollback semantics for the production migration's + // post-ALTER failure contract without relying on a preexisting reserved + // durable namespace object. + let err = conn + .execute_batch( + r#" + BEGIN; + ALTER TABLE main.ward_audit ADD COLUMN detail TEXT; + CREATE TABLE main.ward_audit_new (conflict TEXT NOT NULL); + CREATE TABLE main.ward_audit_new (conflict TEXT NOT NULL); + "#, + ) + .expect_err("controlled post-ALTER SQL failure must abort the transaction"); + assert!( + err.to_string() + .contains("table ward_audit_new already exists"), + "unexpected migration error: {err}" + ); + conn.execute_batch("ROLLBACK;").unwrap(); + + assert_eq!(load_legacy_audit_row(&conn, row_id), before); + assert_eq!(user_version(&conn), 37); + assert_schema_state(&conn, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013); + assert!(!has_column(&conn, "detail")); + assert!(!main_schema_object_exists(&conn, "table", "ward_audit_new")); + assert_eq!( + stored_table_sql(&conn), + sql_literal_value(&conn, ward_audit_exact_legacy_table_sql_sql!()) + ); + assert_eq!(explicit_index_names(&conn), expected_explicit_index_names()); + assert_eq!(trigger_names(&conn), expected_trigger_names()); + } } diff --git a/crates/coven-threads-core/src/lib.rs b/crates/coven-threads-core/src/lib.rs index 235a840..712d9a2 100644 --- a/crates/coven-threads-core/src/lib.rs +++ b/crates/coven-threads-core/src/lib.rs @@ -66,7 +66,11 @@ pub mod thread; pub mod validate; pub mod weave; -pub use audit::{AuditEventType, WardAuditRecord, WARD_AUDIT_SCHEMA_SQL}; +pub use audit::{ + AuditEventType, WardAuditRecord, WARD_AUDIT_MIGRATION_V020_SQL, WARD_AUDIT_SCHEMA_SQL, + WARD_AUDIT_SCHEMA_STATE_CURRENT_V020, WARD_AUDIT_SCHEMA_STATE_LEGACY_V013, + WARD_AUDIT_SCHEMA_STATE_MISSING, WARD_AUDIT_SCHEMA_STATE_SQL, WARD_AUDIT_SCHEMA_STATE_UNKNOWN, +}; pub use channel::Channel; pub use fray::{FrayOrSnap, FrayReason, SnapReason}; pub use ids::{ diff --git a/docs/architecture.md b/docs/architecture.md index 9148191..be64efc 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -62,7 +62,7 @@ The rationale stacks three facts: - RFC-0001 §5.6 defines the audit-log entry shape, including `ward_hash`, and requires append-only behavior: entries MUST NOT be deleted or modified. - The daemon already owns `coven.sqlite3`, so putting the table there inherits the existing ownership and access boundary for free. -Every gate verdict is auditable, and WARD-C6's compaction ledger rides the same table rather than a second store. The implemented contract (`audit.rs`, `[IMPLEMENTED, NOT ENFORCING]`) mirrors RFC-0001 §5.6's event vocabulary (`proposal_submitted`, `proposal_approved`, `proposal_rejected`, `proposal_vetoed`, `ward_updated`) and adds the coven-threads extensions (gate verdicts, compaction ledger entries). One implementation note: the SQL table is spelled `ward_audit`, because a literal dot in the name would collide with SQLite's attached-database syntax — and an attached `ward.*` database would *be* the forbidden sidecar. +Every gate verdict is auditable, and WARD-C6's compaction ledger rides the same table rather than a second store. The implemented contract (`audit.rs`, `[IMPLEMENTED, NOT ENFORCING]`) mirrors RFC-0001 §5.6's event vocabulary (`proposal_submitted`, `proposal_approved`, `proposal_rejected`, `proposal_vetoed`, `ward_updated`) and adds the coven-threads extensions (gate verdicts, compaction ledger entries). One implementation note: the SQL table is spelled `ward_audit`, because a literal dot in the name would collide with SQLite's attached-database syntax — and an attached `ward.*` database would *be* the forbidden sidecar. Fresh init and the legacy→v0.2.0 migration both run exact pre- and postcondition guards inside `BEGIN IMMEDIATE`, so callers only commit exact `current_v020` and must `ROLLBACK` on any drift. ## Compatibility contract diff --git a/docs/superpowers/plans/2026-07-19-apply-audit-migration-repair.md b/docs/superpowers/plans/2026-07-19-apply-audit-migration-repair.md new file mode 100644 index 0000000..e5877b4 --- /dev/null +++ b/docs/superpowers/plans/2026-07-19-apply-audit-migration-repair.md @@ -0,0 +1,221 @@ +# ApplyAudit Migration Repair Implementation Plan + +**Goal:** Replace the old two-boolean `ward_audit` migration gate with a full +schema fingerprint/state contract built around exact durable +`main.sqlite_master.sql` fingerprints for the table/indexes/triggers plus +ordered column metadata, reserve the durable `ward_audit` / +`ward_audit_*` namespace to an exact five-object whitelist in every state, fail +closed on any TEMP shadow/reserved temp object, make schema initialization fail +closed and atomic, preserve evidence and unrelated `user_version`, and prove +rollback behavior with executable rusqlite tests. + +Because PR #7 also adds the public `AuditEventType::ApplyAudit` enum variant +and the public `WardAuditRecord::detail` field, the release must bump to v0.2.0 +instead of claiming `0.1.x` compatibility under Cargo `^0.1`. + +**Architecture:** Keep ownership in `crates/coven-threads-core/src/audit.rs`. +Expose `WARD_AUDIT_SCHEMA_STATE_SQL` plus stable tags so callers can branch on +`missing` / `legacy_v013` / `current_v020` / `unknown`, reserve `missing` for +an absent `main.ward_audit` with no unexpected durable reserved-name object and +no temp shadow/reserved temp object, require the same exact durable whitelist +for `legacy_v013` and `current_v020`, wrap both `WARD_AUDIT_SCHEMA_SQL` and +`WARD_AUDIT_MIGRATION_V020_SQL` in `BEGIN IMMEDIATE` transactions so guard +reads happen under the main-database write reservation, let concurrent init +serialize cleanly to `current_v020`, and make a concurrent second migration +wait, then reclassify current when the legacy guard re-runs. Durable DDL/DML +stays explicitly qualified to `main` wherever SQLite permits it, and the +init/migration paths never write database-wide `user_version`. + +**Tech Stack:** Rust 2021, SQLite, Cargo, bundled `rusqlite` (hooks-enabled in +dev-dependencies) for in-memory and file-backed concurrency tests. + +--- + +## Final File Map + +For this semver-corrected follow-up, the repair delta relative to +`origin/cody/apply-audit-v014` stays scoped to these eight tracked files: + +1. `CHANGELOG.md` +2. `Cargo.lock` +3. `Cargo.toml` +4. `crates/coven-threads-core/src/audit.rs` +5. `crates/coven-threads-core/src/lib.rs` +6. `docs/architecture.md` +7. `docs/superpowers/plans/2026-07-19-apply-audit-migration-repair.md` +8. `docs/superpowers/specs/2026-07-19-apply-audit-migration-repair-design.md` + +--- + +## Task 1: Lock the behavior down with tests + +- Add schema-state tests for: + - empty DB → `missing`; + - exact legacy fixture → `legacy_v013`; + - exact current schema → `current_v020`; + - exact legacy/current with only the durable whitelist objects still + classifying correctly; + - absent-table reserved-name collisions (`ward_audit_event_idx`, + `ward_audit_append_only_update`) → `unknown`; + - exact current + extra durable reserved-name view/table → `unknown`; + - exact legacy + preexisting `main.ward_audit_new` → `unknown`; + - current/legacy/missing durable states plus a TEMP `ward_audit` shadow or + reserved temp object → `unknown`. +- Add syntax-verification tests for bundled SQLite: + - `pragma_table_info('ward_audit', 'main')` sees durable columns even when a + TEMP `ward_audit` exists; and + - `pragma_index_list('ward_audit', 'main')` sees durable explicit indexes. +- Add init-safety tests for: + - clean empty DB → `WARD_AUDIT_SCHEMA_SQL` succeeds atomically, lands on + `current_v020`, and append-only UPDATE/DELETE still abort; + - exact current schema → rerunning `WARD_AUDIT_SCHEMA_SQL` is idempotent and + preserves rows/objects; + - file-backed two-connection concurrent init with `busy_timeout` and repeated + synchronized starts → both callers complete successfully because + `BEGIN IMMEDIATE` serializes before guard reads, with no locked/schema-locked + errors and exact final `current_v020`; + - exact legacy schema → `WARD_AUDIT_SCHEMA_SQL` rejects, requires explicit + `ROLLBACK`, and preserves state/data; + - exact current `main.ward_audit` + TEMP shadow clone → init rejects, + explicit rollback preserves both main and temp rows, and dropping the TEMP + shadow restores `current_v020`; + - missing main + TEMP `ward_audit` / reserved `ward_audit_*` temp object → + init rejects and never creates `main.ward_audit`; + - unknown partial current schema → `WARD_AUDIT_SCHEMA_SQL` rejects and + preserves the drifted state. +- Add drift tests for: + - legacy + extra column/data → `unknown`, migration guard failure, explicit + `ROLLBACK`, and preservation of row data, extra column, indexes, triggers, + and unrelated `user_version`; + - legacy/current `event_type` CHECK drift where a quoted literal gains an + internal space (for example `apply_ audit`) → `unknown`, with the legacy + guard failing before mutation and rollback preserving data; + - legacy/current with an extra table-level `CHECK (length(decision) > 0)` → + `unknown`, with the legacy guard failing before mutation and rollback + preserving the original CHECK behavior; + - legacy/current with an extra table-level `UNIQUE (decision, recorded_at)` + → `unknown`, with the legacy guard failing before mutation and rollback + preserving the original UNIQUE behavior; + - current missing one append-only trigger → `unknown`, plus an `UPDATE` + succeeding to prove why it is unknown; + - current with an extra explicit object (index), `recorded_at DESC`, or + `COLLATE NOCASE` on the explicit index SQL → `unknown`; + - current with altered append-only trigger error literal/body → `unknown`. +- Update existing migration failure tests so exact current/rerun cases fail at + the new guard and always `ROLLBACK` with `unwrap()`. +- Add legacy TEMP-shadow migration coverage proving the guard rejects before any + main/temp mutation and rollback preserves the exact legacy durable table. +- Add file-backed two-connection concurrent legacy migration coverage with + `busy_timeout` and repeated synchronized starts proving one caller upgrades, + the other waits, then fails only at the legacy guard after seeing exact + current (never with a lock error), while preserving the legacy row. +- Add a post-rebuild drift coverage path that reuses + `WARD_AUDIT_MIGRATION_V020_SQL`, injects an unexpected durable + `main.ward_audit_*` object immediately before the real postcondition guard, + requires a guard failure, and proves explicit rollback restores exact legacy + row/schema/constraints/user_version while removing the injected object. +- Add reason-demo coverage proving an unqualified `INSERT INTO ward_audit ...` + lands in TEMP while the durable contract remains `unknown`. +- Add a controlled post-`ALTER` rollback coverage path that starts from exact + legacy, performs the same `ALTER TABLE main.ward_audit ADD COLUMN detail + TEXT`, then forces a later SQL error by creating `main.ward_audit_new` twice + inside the same transaction; require the error, explicit rollback success, + full row/schema/user_version restoration, and `legacy_v013` classification. +- Add exact-stored-SQL tests proving fresh current and successfully migrated + current both classify `current_v020` while matching only their controlled + SQLite-emitted table-SQL variants, and that the shipped legacy fixture + matches `legacy_v013`. + +## Task 2: Implement the schema fingerprint/state contract + +- Add stable state tag constants. +- Add public `WARD_AUDIT_SCHEMA_STATE_SQL` that fingerprints: + - exact stored `main.sqlite_master.sql` for `ward_audit`, covering all + table-level constraints; + - ordered durable column metadata from `pragma_table_info('ward_audit', + 'main')` (including `recorded_at` default and PK metadata); + - explicit durable index discovery from `pragma_index_list('ward_audit', + 'main')`, with exact index SQL then read from `main.sqlite_master`; + - exact append-only trigger SQL from `main.sqlite_master`; + - at every durable state, an exact reserved main-schema whitelist consisting + only of table `main.ward_audit`, indexes `ward_audit_event_idx` and + `ward_audit_familiar_idx` attached to it, and triggers + `ward_audit_append_only_update` / `ward_audit_append_only_delete` attached + to it, with every other `ward_audit` / `ward_audit_*` main object rejected; + - at every durable state, rejection when any temp-schema table/view/index/ + trigger is named `ward_audit` or begins with `ward_audit_`; and + - only the controlled fresh/migrated current table-SQL variants plus the + shipped legacy table-SQL variant, including the presence of `apply_audit` + and any preserved inline comments. +- Re-export the state query and tags from `crates/coven-threads-core/src/lib.rs` + for root-API callers. +- Update `WARD_AUDIT_SCHEMA_SQL` so it: + - starts with `BEGIN IMMEDIATE` so the main-database write reservation is + acquired before any guard read/classification; + - creates a uniquely named TEMP pre-install guard with `CHECK (ok = 1)`; + - inserts `1` only when the shared schema-state expression returns + `missing` or `current_v020`, otherwise inserts `0` and aborts before any + mutation; + - keeps `CREATE TABLE/INDEX/TRIGGER IF NOT EXISTS` for current daemon + compatibility while targeting `main` unambiguously wherever SQLite syntax + allows it; + - creates a uniquely named TEMP post-install guard that requires exact + `current_v020`, aborting on any silent no-op/collision or malformed result; + - commits on success; and + - requires callers to `ROLLBACK` after any init error. +- Update `WARD_AUDIT_MIGRATION_V020_SQL` so it: + - starts with `BEGIN IMMEDIATE` so concurrent migrators serialize before the + legacy guard read; + - creates a uniquely named TEMP precondition guard table with `CHECK (ok = 1)`; + - inserts `1` only when the full exact legacy durable predicate holds and no + unexpected durable reserved-name object or temp shadow/reserved temp object + exists, otherwise inserts `0` and aborts; + - after the rebuild, creates a distinct TEMP postcondition guard that reruns + the shared schema-state CTE/predicates and requires exact `current_v020` + before `COMMIT`; + - drops both guards on the success path; + - keeps `ALTER ADD detail`, strict replacement-table creation, detail copy, + and explicit main index/trigger recreation, all qualified to `main` + wherever SQLite syntax permits; + - never writes `PRAGMA user_version`. + +## Task 3: Update the written contract and validate + +- Bump `[workspace.package].version` in the root `Cargo.toml` to `0.2.0`, let + Cargo refresh the `coven-threads-core` lock entry, and verify + `cargo metadata --no-deps --format-version 1` reports `0.2.0`. +- Update `audit.rs` docs, the design doc, the plan doc, and `CHANGELOG.md` to + say that the durable audit contract is `main.ward_audit`, exact stored + `main.sqlite_master.sql` equality covers every declared table-level + constraint, schema-qualified PRAGMAs are required for durable column/index + inspection, the exact durable namespace whitelist applies in every state, + temp shadows/reserved temp objects fail closed, guard reads happen under an + IMMEDIATE transaction, concurrent init serializes, concurrent second + migration callers must reclassify after the legacy guard rejects current, and + no whitespace-destroying normalization is allowed. +- Document the caller contract explicitly: + - query `WARD_AUDIT_SCHEMA_STATE_SQL`; + - initialize on `missing` only; + - migrate only `legacy_v013`; + - continue on `current_v020`; + - fail closed on `unknown`; + - `ROLLBACK` after any init or migration error; and + - rely on the init/migration independent guards as second lines of defense. +- Run: + - `cargo fmt` + - `cargo clippy --workspace --all-targets -- -D warnings` + - focused audit tests + - `cargo metadata --no-deps --format-version 1` + - `cargo test --workspace` if focused tests pass + - `git diff --check` + - stale-wording searches for the old partial-gate terminology. + +## Commit + +Create a new commit (no amend) with: + +```text +fix(audit): release ApplyAudit as v0.2.0 + +Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> +``` diff --git a/docs/superpowers/specs/2026-07-19-apply-audit-migration-repair-design.md b/docs/superpowers/specs/2026-07-19-apply-audit-migration-repair-design.md new file mode 100644 index 0000000..07a9e31 --- /dev/null +++ b/docs/superpowers/specs/2026-07-19-apply-audit-migration-repair-design.md @@ -0,0 +1,202 @@ +# ApplyAudit Migration Repair + +## Goal + +Repair PR #7 so `ward_audit` upgrades are gated by a full table-local schema +fingerprint/state contract, not a two-boolean proxy. The upgrade must preserve +immutable evidence, preserve unrelated shared-store `user_version`, reject +partial or drifted schemas before mutation, and remain scoped to the audit +schema contract. + +Because PR #7 adds the public `AuditEventType::ApplyAudit` enum variant and the +public `WardAuditRecord::detail` field, the corrected release line is v0.2.0. +Cargo `^0.1` consumers can break on exhaustive `match` arms or struct literals, +so the current schema/state contract is named `current_v020`. + +## Schema-state contract + +`crates/coven-threads-core::WARD_AUDIT_SCHEMA_STATE_SQL` is the reusable query +callers run before touching the durable `main.ward_audit` table. It returns one +stable tag: + +- `missing` — `main.ward_audit` does not exist, no unexpected durable + main-schema object named `ward_audit` or `ward_audit_*` exists, and no temp + shadow/reserved temp object exists; initialize with `WARD_AUDIT_SCHEMA_SQL`. +- `legacy_v013` — exact v0.1.3 legacy fingerprint plus the durable namespace + whitelist; run + `WARD_AUDIT_MIGRATION_V020_SQL`. +- `current_v020` — exact v0.2.0 current fingerprint plus the durable namespace + whitelist; continue without schema work. +- `unknown` — every other shape; fail closed and investigate manually. + +The fingerprint is exact and table-local to `main.ward_audit`: + +- exact stored `main.sqlite_master.sql` for the `ward_audit` table; +- ordered durable-column metadata from `pragma_table_info('ward_audit', + 'main')`, including name, type, `NOT NULL`, `DEFAULT`, and primary-key + metadata; +- durable explicit index discovery from `pragma_index_list('ward_audit', + 'main')`, with exact index SQL then read from `main.sqlite_master` and + fingerprinted as `name|sql` (excluding SQLite internal autoindexes); and +- exact stored `main.sqlite_master.sql` for each append-only trigger (ordered + by name and fingerprinted as `name|sql`). + +The full stored table definition covers every declared table-level constraint — +the `event_type` CHECK list, extra `CHECK` clauses, `UNIQUE` clauses, and +foreign-key clauses — so any extra or missing constraint, column, index, or +trigger classifies as `unknown`. Across **all** durable states, the reserved +main-schema namespace is whitelisted to exactly these `main.ward_audit` +objects: the table itself, indexes `ward_audit_event_idx` and +`ward_audit_familiar_idx`, and triggers +`ward_audit_append_only_update` / `ward_audit_append_only_delete`, each attached +to `main.ward_audit`. Every other main-schema table/view/index/trigger whose +name is exactly `ward_audit` or begins with `ward_audit_` also classifies as +`unknown`, including `ward_audit_new`, backup/shadow tables, and reserved-name +indexes/triggers attached elsewhere. Any temp-schema table/view/index/trigger +whose name is exactly `ward_audit` or begins with `ward_audit_` likewise +classifies as `unknown`, even when `main.ward_audit` is otherwise exact current +or legacy; that fail-closed rule stops TEMP shadows from being mistaken for +healthy durable state and blocks unqualified daemon writes from proceeding under +a false `current_v020`. No whitespace-destroying normalization is applied: only +the exact stored SQL variants SQLite emits for the shipped schemas are +accepted. For `current_v020`, that means the fresh `CREATE TABLE ward_audit +(...)` form and the quoted `CREATE TABLE "ward_audit" (...)` form produced by +the legacy rename path. For `legacy_v013`, the fingerprint intentionally +includes the inline comments preserved from the shipped v0.1.3 DDL. + +## Initialization design + +`WARD_AUDIT_SCHEMA_SQL` must stay compatible with the current daemon behavior: +the daemon executes it unconditionally on every store open. The safe contract is +therefore: + +1. `BEGIN IMMEDIATE;` to reserve the main-database write slot before any guard + read/classification; +2. run a uniquely named TEMP pre-install guard that reuses the exact + schema-state CTEs and permits only `missing` or exact `current_v020`; +3. create the durable `main.ward_audit` table, explicit main indexes, and + append-only main triggers with `IF NOT EXISTS`; +4. run a uniquely named TEMP post-install guard that reuses the same + schema-state expression and requires exact `current_v020`; and +5. `COMMIT;` + +This makes fresh installs atomic, makes exact current reruns idempotent, and +serializes concurrent initializers before they read `main.sqlite_master`: the +winner reserves the main-database write slot first, and the second caller waits +until commit, re-runs the guard against the committed schema, and then +idempotently sees exact `current_v020`. The path still fails closed for +`legacy_v013`, `unknown`, any unexpected durable reserved-name object, any temp +shadow/reserved temp object, or any malformed/silently skipped install result. +If either guard errors, callers must explicitly `ROLLBACK` before continuing so +SQLite removes any uncommitted `main.ward_audit` table created on the failed +path while preserving unrelated durable/temp objects that caused the collision. + +## Migration design + +The v0.2.0 repair remains a table-local migration. The SQL still: + +1. `ALTER TABLE main.ward_audit ADD COLUMN detail TEXT`; +2. creates a strict replacement `main.ward_audit_new`; +3. copies every row, preserving `detail`; +4. swaps tables; and +5. re-creates the required explicit main indexes and append-only main triggers; + and +6. runs a distinct TEMP post-migration guard before `COMMIT`. + +The change is the guardrail: immediately after `BEGIN IMMEDIATE;`, the +migration creates a uniquely named TEMP precondition guard table with `CHECK (ok = 1)` and +inserts `1` only if the exact `legacy_v013` durable fingerprint holds **and** +no unexpected durable reserved-name object or temp shadow/reserved temp object +exists. Any missing/current/partial/shadowed schema inserts `0` instead, +aborting before `ALTER TABLE`. That IMMEDIATE reservation also serializes +concurrent migrators before the guard read: the winner upgrades first, and a +second caller waits, re-runs the guard against the now-current durable table, +and fails closed there instead of racing into a `sqlite_master` lock. This +makes the migration fail closed even if a caller skips the classification query, +and keeps initialization and migration aligned on the same exact state +contract. + +Before `COMMIT`, the migration now creates a second uniquely named TEMP guard +table that reruns the same shared schema-state CTE/predicates and requires exact +`current_v020`. That postcondition guard catches rebuilt tables that have become +self-unknown inside the still-open `BEGIN IMMEDIATE` transaction — for example, +if an unexpected durable `main.ward_audit_*` view or index appears after the +swap but before commit. On that failure path, the caller's explicit `ROLLBACK` +restores the untouched exact legacy table, its rows, and the pre-migration +namespace while removing the injected drift object. + +If a later step fails after `ALTER TABLE`, callers must explicitly `ROLLBACK` +the failed transaction before continuing so SQLite restores the untouched +legacy table. A preexisting `main.ward_audit_new` is now itself durable drift +and must fail at the guard before `ALTER TABLE`; rollback semantics for the +production migration's post-`ALTER` failure contract are instead validated with +a controlled duplicate `CREATE TABLE main.ward_audit_new` later in the same +transaction. Ward SQL never reads or writes database-wide `PRAGMA user_version`, +so unrelated shared-store version state is preserved. + +## Tests + +Executable rusqlite tests cover: + +1. schema-state classification for `missing`, exact `legacy_v013`, and exact + `current_v020`, including absent-table reserved durable-namespace collisions, + exact current/legacy shapes with extra durable reserved-name view/table + objects, reserved temp objects, and TEMP `ward_audit` shadows classifying as + `unknown`; +2. fresh empty-store initialization classifying `missing`, running + `WARD_AUDIT_SCHEMA_SQL` atomically to `current_v020`, and enforcing + append-only UPDATE/DELETE rejection; +3. exact current reruns of `WARD_AUDIT_SCHEMA_SQL` staying idempotent while + preserving rows, indexes, triggers, and exact stored SQL fingerprints; +4. exact legacy/current/unknown init-guard failures requiring explicit + `ROLLBACK` and preserving preexisting data/object state; +5. reserved-name collision repros (`ward_audit_event_idx` and + `ward_audit_append_only_update` on another table) failing closed before any + `ward_audit` table is created; and +6. partial legacy drift (`legacy` + extra column/data) classifying as + `unknown`, rejecting at the guard, and rolling back cleanly without losing + row data, extra columns, indexes, triggers, or unrelated `user_version`; +7. legacy/current drift caused by extra table-level `CHECK` or `UNIQUE` + constraints classifying as `unknown`, with legacy guard failures requiring + explicit `ROLLBACK` and preserving the original constraint behavior; +8. current/legacy `event_type` CHECK drift where a quoted literal gains an + internal space (for example `apply_ audit`) classifying as `unknown`, with + the legacy guard rejecting before mutation and rollback preserving data; +9. current drift cases such as a missing append-only trigger, an extra index, a + `recorded_at DESC` index, a `COLLATE NOCASE` index, or altered append-only + trigger SQL classifying as `unknown`; +10. successful exact-legacy upgrade landing in `current_v020`, with fresh and + migrated current schemas both classifying `current_v020` while matching only + their controlled exact stored SQL variants; +11. a test-only migration variant that injects durable reserved-name drift after + the rebuild but before the real postcondition guard, proving the guard fails + closed and explicit rollback restores the exact legacy row/schema/ + constraints/user_version while removing the injected object; +12. a controlled post-`ALTER` SQL failure inside the transaction validating + SQLite rollback semantics for the production migration contract and + restoring the full legacy table state; and +13. schema-qualified PRAGMA syntax on bundled SQLite plus the reason for the + fail-closed contract: unqualified inserts hit TEMP first while the contract + remains `unknown`. +14. file-backed multi-connection initialization with two simultaneous callers, + `busy_timeout`, and repeated runs proving both `WARD_AUDIT_SCHEMA_SQL` + executions complete without locked/schema-locked errors because + `BEGIN IMMEDIATE` serializes them before guard reads; and +15. file-backed multi-connection legacy migration with two simultaneous callers, + `busy_timeout`, and repeated runs proving one migration succeeds while the + second waits, re-runs the guard against exact `current_v020`, and fails only + at the legacy guard (never with a lock error), while preserving the legacy + row data. + +## Scope + +This repair does not add daemon migration orchestration or change the +`WardAuditRecord` wire shape. The daemon remains responsible for calling +`WARD_AUDIT_SCHEMA_STATE_SQL`, choosing fresh initialization vs legacy +migration, treating `main.ward_audit` as the only durable audit contract, +accepting only the exact durable whitelist of `main.ward_audit` plus its two +indexes and two append-only triggers, running `WARD_AUDIT_SCHEMA_SQL` only +through the allowed `missing`/`current_v020` contract, and failing closed on +`unknown`. Concurrent callers must also treat a migration guard rejection after +waiting as a signal to reclassify: another writer may already have serialized +the schema to exact `current_v020`. diff --git a/specs/PHASE-4-CAVE-SURFACES.md b/specs/PHASE-4-CAVE-SURFACES.md index e4432f3..5c47191 100644 --- a/specs/PHASE-4-CAVE-SURFACES.md +++ b/specs/PHASE-4-CAVE-SURFACES.md @@ -258,7 +258,7 @@ never apply edits, never touch `pending/`, never write sqlite. | # | Route | Method | Returns | |---|---|---|---| -| 1 | `/api/weaves` | GET | `WeaveSummary[]` — optional `?familiar=` filter; degraded familiars surface per §2.7 + §4.R12 | +| 1 | `/api/weaves` | GET | `WeaveListView` — `{ "weaves": WeaveSummary[], "degraded": DegradedFamiliarView[] }` (§2.7); optional `?familiar=` filter; degraded familiars render per §4.R12 | | 2 | `/api/weaves/[id]` | GET | `WeaveDetail` | | 3 | `/api/threads/[id]` | GET | `ThreadView` | | 4 | `/api/threads/[id]/strands` | GET | `StrandView[]` (with `fray` diff blocks where blamed) | diff --git a/specs/PHASE-5-APPROVAL-SEMANTICS.md b/specs/PHASE-5-APPROVAL-SEMANTICS.md new file mode 100644 index 0000000..5f5fdb0 --- /dev/null +++ b/specs/PHASE-5-APPROVAL-SEMANTICS.md @@ -0,0 +1,392 @@ +# PHASE-5 PROPOSAL — Approval semantics over the Phase-2 Ward + +**Status:** PROPOSAL (decisions recorded — awaiting PR merge to open Phase 5) +**Date:** 2026-07-18 +**Decision date:** 2026-07-18 +**Bead:** `threads-uqx` +**Authors:** Sage lane (drafted by coordinator session agent) +**Decided by:** Val Alexander + Nova (main session) + +--- + +## 0. Scope and sources + +This is a proposal document only. It does not start Phase 5, amend frozen +Phase 0, or change the daemon. Val and Nova decide whether Phase 5 opens. + +Sources read and cited: + +- `specs/PHASE-0-DESIGN.md`: frozen v0.2; Thread (authority relationship: + surface -> writer), Weave (enforced pattern of threads), Strand (fiber inside + a thread), Channel (axis of load), predicate-authoritative / descriptor- + derived rule, Gate-4 fail-closed posture, and WARD-C1..C7 lineage. +- `../familiar-contract/rfcs/RFC-0001-familiar-contract.md`: §4.2 protected + invariants, §4.3 editable surface, §5.3 approval tiers, §5.4 gates, §5.5 + probes. RFC amendments for closure precondition and provenance predicate are + pending on issues #3/#4 and are treated as upstream-in-flight. +- `../coven-grimoire/articles/drafts/2026-06-24-ward-layer-spec-brief.md` §9: + canonical home of WARD-C1..C7. +- `specs/PHASE-4-CAVE-SURFACES.md`: approval flow §3.7, fail-closed rendering + rules §4, and non-goals §6. +- `../coven/crates/coven-cli/src/ward.rs`, `threads_gate.rs`, `api.rs`: current + daemon reality — path tiers, Gate 1-4 comments, protected-surface weave + construction, and `decide_threads_proposal` as the only tier-0 commit point. +- `~/.coven/workspaces/familiars/nova/ward.toml.v01.bak`: read-only example of + the retired dialect. This draft paraphrases its invariant shape and does not + reproduce Val's exact invariant strings. + +--- + +## 1. What v0.1 expressed that Phase 2 cannot + +Phase 2 reduced Ward configuration to path-tier mapping: tier 0 protected, +tier 1 reviewed, tier 2 logged, tier 3 free. In the daemon, `WardConfig` stores +surface globs and `protected_surface`; `Ward::apply` refuses blocked proposals +as a unit, holds Tier 1 and Gate-1-authorized Tier 0 for Gate 3, and writes only +Tier 2/3. `apply_after_threads_approval` still reruns Gates 1 and 2, and +`api.rs::decide_threads_proposal` revalidates a pending proposal, calls the +threads gate, appends audit rows, advances baselines, removes the pending file, +and is the only current tier-0 approval commit point. + +The retired v0.1 dialect carried three semantic groups Phase 2 cannot express. + +### 1.1 Approval tiers as named governance lanes + +v0.1 distinguished four proposal lanes: automatic regression-gated promotion +with a human veto window; familiar-coherence review with a shorter veto window; +human pre-approval; and human approval with rationale plus audit. Each lane +bound named semantic blocks to a gate. + +Phase 2 can say "logged" or "reviewed." It cannot say "this semantic region is +auto-promoted after regression but vetoable for 48 hours," nor distinguish +human approval from human approval with rationale. RFC-0001 §5.3 still names +these approval tiers and forbids auto-promotion of protected-surface proposals. + +### 1.2 Editable harness blocks as semantic regions + +v0.1 named editable blocks such as execution and recovery prompt regions, tool +defaults, skill configuration, subagent templates, output formats, heartbeat +behavior, tool grants, skill activations, memory conventions, session +introduction, and protected-surface adjacency. These are not file paths. One +block may cut across files; one file may contain several blocks. + +Phase 2 only classifies materialized paths. It cannot answer which semantic +region a diff changed unless that region aligns with a glob. That loses the +RFC-0001 §4.3 editable-surface shape. + +### 1.3 Protected invariants as cross-file identity predicates + +v0.1 carried textual identity assertions over familiar metadata and purpose. +The real backup shows the shape: compact predicate-like statements about name, +person binding, pronouns, purpose, and Coven membership. This document does not +copy them. + +Phase 2 protects files and globs. It does not protect an identity fact that can +be invalidated through an allowed path, prompt block, derived context, or future +portability envelope. RFC-0001 §4.2 says path-only protection is insufficient: +identity-probe inconsistency must reject a proposal regardless of target path. + +--- + +## 2. Carry forward vs deliberately dead + +### 2.1 Approval tiers: carry forward, old encoding dead + +Carry forward the governance distinction; do not revive the v0.1 table shape. +The useful semantic is four approval paths: deterministic low-risk gate, +familiar-coherence gate, human approval, and human approval with rationale. +That matches RFC-0001 §5.3 and explains why Phase 4 already models +`proposal_approved`, `proposal_rejected`, and `proposal_vetoed` audit events. + +What stays dead is stringly coupling among `blocks = [...]`, `gate = "..."`, +and ad hoc veto fields. If Phase 5 opens, policy should be typed and +daemon-owned. The Cave may render it; it must not become a policy engine. + +Recommended name: `ApprovalPath`, not `Tier`. "Tier" is already the path-trust +axis in Phase 2. Reusing it would collapse where a write lands with which +ceremony promotes it. + +### 2.2 Harness blocks: carry forward as regions, not authority + +Carry forward the region abstraction; kill any claim that labels are +authoritative by themselves. A block label is a descriptor unless a +parser/extractor maps bytes to that region deterministically and the daemon can +replay that mapping at apply time. Phase 0 §2.2 binds the rule: predicate +authoritative, descriptor derived. + +Recommended name: `SurfaceRegion` above `SurfaceId`. A `SurfaceRegion` is a +typed semantic region with an extractor predicate over materialized content. +Its descriptor may say "output formats" for the Cave, but the daemon gates on +extractor evidence and final materialized diff. + +### 2.3 Protected invariants: carry forward as predicates, strings dead + +Carry forward strongly. RFC-0001 §4.2 still requires semantic invariants, and +Phase 0 already gives them a home: `PatternPredicate::coherent` is the +authoritative gate; `describe()` yields a derived descriptor. Identity +invariants should become predicate implementations in the weave, not strings +interpreted by clients. + +The v0.1 invariant syntax should stay dead. It was legible but underspecified: +parser semantics, evidence collection, normalization, provenance, and failure +handling were not defined. Pending RFC issues #3/#4 should land, or be waived +by Val/Nova, before Phase 5 freezes invariant interfaces. + +--- + +## 3. Mapping sketches inside the frozen metaphor + +### 3.1 Approval paths: gate policy over existing channels + +Do not model approval paths as new `Channel` variants by default. Channel (axis +of load) says why a thread is stressed: deliberate, forced, serialization, or +mutation. Approval path says which promotion ceremony is required. A mutation +that needs human rationale is still `Channel::Mutation`. + +Recommended default: + +```rust +pub enum ApprovalPath { + AutoRegression { veto: Option }, + FamiliarCoherence { veto: Option }, + HumanApproval, + HumanApprovalWithRationale, +} + +pub struct ProposalClassification { + pub channel: Channel, + pub affected_regions: Vec, + pub path_tier_floor: Tier, + pub approval_path: ApprovalPath, +} +``` + +Path tier remains a floor. A Tier 0 target still requires principal authority +and the threads gate. A Tier 2 path touching a high-risk semantic region may +still require human review. Highest ceremony wins for the proposal as a unit, +matching current all-or-nothing Ward behavior. + +Only add a new channel if Val/Nova decide approval-policy edits themselves need +a distinct load axis. That should not be the default. + +### 3.2 Invariants: PatternPredicate implementations + +Identity invariants should compile into `PatternPredicate` implementations. +The predicate takes relevant threads, strands, and verified proposal evidence; +its result is authoritative. Its descriptor is Cave-readable, labeled derived, +and never enforced on. + +Concrete referent mapping: + +- Thread (authority relationship: surface -> writer): protected identity + surfaces keep principal-writer threads. +- Strand (fiber inside a thread): invariant evidence may use hashes, manifest + entries, audit anchors, serialization markers, and only later a new strand if + evidence forces one. +- Weave (enforced pattern of threads): identity predicates are part of the weave + pattern, not side-table rows. +- Channel (axis of load): invariants must hold under mutation and serialization + at minimum; forced compaction connects through WARD-C1..C6. + +Recommended default: predicate-first; no new strand kind until implementation +evidence requires it. + +### 3.3 Harness blocks: SurfaceRegion above SurfaceId + +A `SurfaceRegion` should be daemon-replayable over a materialized diff: + +```rust +pub trait SurfaceRegionPredicate { + fn materialize(&self, proposal: &MaterializedDiff) -> RegionEvidence; + fn describe(&self) -> SurfaceRegionDescriptor; +} +``` + +The descriptor is for humans. The predicate output feeds classification and +Gate 4 replay. It cannot depend on Cave state, agent self-report, or stale +metadata. + +Thread options: + +- one thread per block: inspectable, but risks threads detached from + source-authoritative surfaces; +- one thread per source surface, with region evidence on the mutation request: + closer to Phase 2 and safer by default. + +Recommended default: keep threads source-bound; attach region evidence to +proposal classification. Promote a region to thread status only when it has a +stable source-authoritative projection. + +--- + +## 4. Veto windows vs synchronous gates + +A synchronous gate answers before write: permit, stage, or reject. Phase 2's +protected-target flow stages on fray, waits for principal approval, revalidates, +and applies. Nothing writes before authority clears. + +A veto window has two possible meanings: + +1. delayed apply: gates pass, the proposal becomes pending-visible, and the + daemon applies only after the window expires with no veto; +2. provisional apply: gates pass, the daemon applies immediately, and a later + veto triggers rollback. + +Recommended default: delayed apply. Provisional apply should be forbidden until +Val/Nova explicitly accept rollback semantics. Gate-4 fail-closed and Phase 4's +"no optimistic UI, no queued decisions" posture both prefer visible pending +state over speculative mutation. + +Delayed apply needs audit moments: + +- `proposal_submitted`: classification, diff hash, affected paths/regions, + predicate evidence, deadline; +- `proposal_window_opened`: gate evidence and veto deadline recorded; +- final `proposal_approved`, `proposal_vetoed`, `proposal_expired`, or + `proposal_rejected`: daemon revalidation result immediately before apply or + refusal. + +WARD-C6 matters by analogy: a time-separated, lossy or delayed boundary needs a +ledger. A proposal whose evidence cannot be replayed at deadline fails closed. +Future Cave surfaces should extend `ProposalView` only after daemon evidence +exists; until then unknown or stale veto state renders blocked under Phase 4 §4. + +--- + +## 5. Explicit non-goals + +- No general policy engine; README anti-goals and Phase 4 §6 bind. +- No client-side approval authority; Cave forwards, daemon decides. +- No revival of v0.1 TOML as normative format. +- No Phase-5 start and no changes to frozen Phase 0. +- No weakening of Gate 4; every path ends in live daemon re-materialization. +- No `.af` compatibility work; Phase 3 already decided `.weave` canonical plus + lossy one-way `.af` exporter. +- No broad runtime capability system; tool grants may classify proposals, but + runtime chamber authority remains separate. + +--- + +## 6. Decisions (Val + Nova, 2026-07-18) + +All eight questions resolved on Sage's recommended defaults. RFC dependency +blocker (issues #3/#4) resolved: both landed in OpenCoven/familiar-contract@2a5bb98 +and issues closed before this decision record was written. + +1. **`ApprovalPath` separate from `Channel`? → YES.** + Channel remains the load axis (why a thread is stressed). ApprovalPath is + the promotion ceremony. Conflating them would make Phase 0 load-axis + semantics ambiguous. `ApprovalPath` proceeds as a distinct type. + *Follows the source-authoritative-enforcement rule: ApprovalPath is + authoritative; Channel is a derived load descriptor. Never gate on Channel. + If this rule shifts, revisit all eight decisions.* (Echo, Sage 2026-07-18) + +2. **Veto windows: delayed-apply or provisional-apply? → DELAYED APPLY ONLY.** + Matches Gate-4 fail-closed posture and Phase 4 "no optimistic UI, no queued + decisions." Provisional apply + rollback is a distinct threat model; not + opening that door in Phase 5. + *Audit implication: the veto window close event needs an explicit reason + field — `applied | vetoed | expired | superseded`. Without it, + delayed-apply is a post-hoc audit black hole. Confirm `ApplyAudit` (issue + #5) captures the close event, not just the apply event. (Echo 2026-07-18)* + +3. **Harness regions: classify ceremony or become threads? → CLASSIFY FIRST.** + Region → thread promotion only when a stable source-authoritative projection + exists. Default: threads source-bound, region evidence on proposal + classification. No premature thread proliferation. + *Forward-only promotion: if a region reclassifies mid-session, promotion + applies forward only. Retroactive projection would corrupt the authority + trail with apparently-authored writes from before the promotion decision. + (Echo 2026-07-18)* + +4. **Invariants: deterministic extraction, identity probes, or both? + → DETERMINISTIC WHERE POSSIBLE; probes as Gate-3 evidence, not sole authority.** + Consistent with RFC-0001 rule that LLM-judge-only is forbidden for auto-tier. + Same logic applies to invariant checking. + *When deterministic extraction fails or is ambiguous, default is + fail-closed (no promotion) — not silent fallback to LLM judgment. + Ambiguity is an explicit ignored/blocked state. (Echo 2026-07-18)* + +5. **Must RFC issues #3/#4 land before freeze? → MOOT; both landed.** + OpenCoven/coven-threads#3 and #4 closed 2026-07-18. Phase 5 freeze has no + RFC amendment blocker. See familiar-contract@2a5bb98 for the resolved shape; + issues #3/#4 preserved in git history for provenance. + +6. **Cave `ProposalView` extension: daemon contract first or simultaneous? + → DAEMON CONTRACT FIRST.** + Cave extension only after daemon read models exist. Consistent with Phase 4 + pattern. Cave ProposalView for Phase 5 starts as `[DESIGNED, NOT SHIPPED]` + until release evidence exists. (Echo 2026-07-18) + +7. **Preserve display labels `auto`, `familiar_review`, `human_review`, + `human_required`? → YES — display names preserved, typed variants internal.** + Daemon owns the typed `ApprovalPath` enum; Cave renders the human-readable + labels. No mental-model breakage. + *Label-variant round-trip must be a daemon wire contract, not Cave + convention. Daemon emits `{variant, label, veto_deadline}`; Cave has zero + policy freedom over label strings. Daemon should reject at load if a variant + has no corresponding display label or vice versa. (Sage 2026-07-18)* + +8. **Add `proposal_window_opened` audit event? → YES, when delayed apply ships.** + Minimal ledger entry for auditable veto windows. Connects to issue #5 + (`ward_audit` / `ApplyAudit`); that lane (Cody) can consume both in the same + schema pass. + *Also requires a corresponding close event with reason field (see decision + #2 audit note). The window is a first-class audit interval, not a gap.* + +--- + +## 7. Phase shape — Phase 5 is open + +Phase 5 opened 2026-07-18 (Val + Nova decision). Beads are live. + +- `threads-uqx` — [EPIC] approval semantics over the Phase-2 Ward. +- `threads-uqx.1` — ✅ RESOLVED: `ApprovalPath` separate from `Channel`; delayed + apply; classify-first for regions; deterministic+probe for invariants. +- `threads-uqx.2` — ✅ RESOLVED: RFC #3/#4 landed (familiar-contract@2a5bb98). +- `threads-uqx.3` — Core type sketch: `ApprovalPath`, `VetoWindow`, + `ProposalClassification`, region evidence, audit event shape. + *Design constraints from Sage/Echo (2026-07-18) to incorporate:* + *(a) `evidence_replay_hash` on `ProposalClassification` — delayed-apply* + *revalidation at deadline must prove it replays the same evidence that* + *opened the window (WARD-C7 generalizes: evidence must survive the time gap).* + *(b) `VetoWindow.min_visible: Duration` — veto window is only fail-closed if* + *pending state was actually visible long enough for a human to act on it* + *(same shape as two-compaction contract).* + *(c) Label mapping as daemon wire contract with load-time reject on drift.* +- `threads-uqx.4` — Predicate design: identity invariant + `PatternPredicate`; descriptor-derived proof. +- `threads-uqx.5` — Surface-region design: extractor predicates over + materialized diffs and Gate-4 replay. +- `threads-uqx.6` — Daemon integration design: classification, delayed-apply + scheduler, revalidation at deadline, append-only audit in `coven.sqlite3`. +- `threads-uqx.7` — Cave contract amendment after daemon read models exist; + add blocked fixtures for veto states. +- `threads-uqx.8` — Cody implementation lane: core crate and daemon tests. +- `threads-uqx.9` — Nova sign-off: RFC round-trip, Gate-4 fail-closed proof, + descriptor-not-authority review. +- `threads-uqx.10` — Val freeze: Phase-5 design frozen or rejected. + +Suggested gates: proposal review; Nova design gate; Val authority gate; RFC +amendment gate; implementation gate proving no apply bypass; Cave fail-closed +rendering gate; final Val freeze after Nova sign-off. + +--- + +## 8. Recommendation — executed + +Phase 5 opened. Val and Nova confirmed: we want to recover semantic approval +behavior, not just document why it was lost. + +Carry forward: approval paths as typed promotion ceremony over existing +channels; harness blocks as daemon-replayable surface-region classifiers; +protected invariants as predicate-authoritative identity checks. + +Leave dead: v0.1 stringly gate/block coupling; invariant strings as enforcement +objects; client-side or descriptor-based authority; provisional apply during +veto windows unless explicitly accepted. + +The conservative design is not to make Ward more general. It is to restore the +minimum semantic distinctions RFC-0001 still names while preserving the Phase-0 +spine: source-authoritative surfaces, predicate-authoritative patterns, +daemon-owned audit, and fail-closed Gate 4.