feat(engine): Phase 11.2 logical clock + active-tx registry (SQLR-22)

docs/_index.md

@@ -54,7 +54,7 @@ As of May 2026, SQLRite has:
 - Full-text search + hybrid retrieval (Phase 8 complete): FTS5-style inverted index with BM25 ranking + `fts_match` / `bm25_score` scalar functions + `try_fts_probe` optimizer hook + on-disk persistence with on-demand v4 → v5 file-format bump (8a-8c), a worked hybrid-retrieval example combining BM25 with vector cosine via raw arithmetic (8d), and a `bm25_search` MCP tool symmetric with `vector_search` (8e). See [`docs/fts.md`](fts.md).
 - SQL surface + DX follow-ups (Phase 9 complete, v0.2.0 → v0.9.1): DDL completeness — `DEFAULT`, `DROP TABLE` / `DROP INDEX`, `ALTER TABLE` (9a); free-list + manual `VACUUM` (9b) + auto-VACUUM (9c); `IS NULL` / `IS NOT NULL` (9d); `GROUP BY` + aggregates + `DISTINCT` + `LIKE` + `IN` (9e); four flavors of `JOIN` — INNER, LEFT, RIGHT, FULL OUTER (9f); prepared statements + `?` parameter binding with a per-connection LRU plan cache (9g); HNSW probe widened to cosine + dot via `WITH (metric = …)` (9h); `PRAGMA` dispatcher with the `auto_vacuum` knob (9i)
 - Benchmarks against SQLite + DuckDB (Phase 10 complete, SQLR-4 / SQLR-16): twelve-workload bench harness with a pluggable `Driver` trait, criterion-driven, pinned-host runs published. See [`docs/benchmarks.md`](benchmarks.md).
-- Phase 11 (concurrent writes via MVCC + `BEGIN CONCURRENT`, SQLR-22) is in flight. **11.1 multi-connection foundation: shipped on this branch.** `Connection` is now `Send + Sync` and `Connection::connect()` mints a sibling handle that shares the same backing `Database`. Snapshot-isolation reads + `BEGIN CONCURRENT` writes follow in 11.3 / 11.4. Plan: [`docs/concurrent-writes-plan.md`](concurrent-writes-plan.md).
+- Phase 11 (concurrent writes via MVCC + `BEGIN CONCURRENT`, SQLR-22) is in flight. **11.1 multi-connection foundation: shipped.** `Connection` is now `Send + Sync` and `Connection::connect()` mints a sibling handle that shares the same backing `Database`. **11.2 logical clock + active-tx registry: shipped on this branch.** New `sqlrite::mvcc` module exposes `MvccClock` (AtomicU64-backed) and `ActiveTxRegistry` (`min_active_begin_ts` for GC). The WAL header bumps v1 → v2 to persist the clock high-water mark; v1 WALs upgrade transparently. Snapshot-isolation reads + `BEGIN CONCURRENT` writes follow in 11.3 / 11.4. Plan: [`docs/concurrent-writes-plan.md`](concurrent-writes-plan.md).
 - A fully-automated release pipeline that ships every product to its registry on every release with one human action — Rust engine + `sqlrite-ask` + `sqlrite-mcp` to crates.io, Python wheels to PyPI (`sqlrite`), Node.js + WASM to npm (`@joaoh82/sqlrite` + `@joaoh82/sqlrite-wasm`), Go module via `sdk/go/v*` git tag, plus C FFI tarballs, MCP binary tarballs, and unsigned desktop installers as GitHub Release assets (Phase 6 complete)
 
 See the [Roadmap](roadmap.md) for the full phase plan.

docs/design-decisions.md

@@ -156,6 +156,22 @@ Decisions are grouped by the engine layer they concern: parser, storage, concurr
 
 ---
 
+### 12b. MVCC logical clock persisted in the WAL header (Phase 11.2)
+
+**Decision.** A new `sqlrite::mvcc` module ships [`MvccClock`](../src/mvcc/clock.rs) (an `AtomicU64`-backed process-wide counter) and [`ActiveTxRegistry`](../src/mvcc/registry.rs) (a `Mutex<BTreeMap>`-backed set of in-flight `TxId → begin_ts` mappings). The clock's high-water mark is persisted in the WAL header — bytes 24..32, previously reserved-zero — and the WAL format version bumps from 1 to 2.
+
+**Why.** MVCC visibility (`begin_ts <= ts < end_ts`) requires that timestamps never repeat — including across reopens. Restarting the engine and resuming the clock from zero would silently break that invariant the moment two transactions on either side of a restart picked the same value. Persisting the high-water mark on each checkpoint means reopen seeds the in-memory clock past the last value the previous run handed out. Putting it in the WAL header (rather than the main file) is right because the WAL is the durability boundary already; commits already fsync the WAL, and the existing checkpoint code rewrites the WAL header on truncate.
+
+**Cost.** A WAL format bump is real but is mitigated by the v1 layout: bytes 24..32 were reserved-zero, so a v1 WAL parses cleanly under v2 rules with `clock_high_water = 0` — exactly what a never-ticked clock would carry. The next checkpoint rewrites the header at v2; no offline upgrade step. The reader still rejects forward versions (e.g. v3) with a typed error rather than silently misinterpreting bytes.
+
+**Why an `AtomicU64`, not a plain `u64` behind a `Mutex`.** Each transaction calls the clock twice (begin + commit). Under contention, a `Mutex` would funnel every BEGIN through one critical section. Atomic CAS is wait-free and cheaper. The `observe()` helper (used at WAL replay to bring the clock up to a persisted value) is a CAS loop rather than a `store` so two racing observers can't move the clock backwards.
+
+**Why `Mutex<BTreeMap>` for the active-tx registry rather than a sharded skip list.** The registry is touched twice per transaction (begin + commit/rollback). Phase 11.6's GC reads `min_active_begin_ts` once per sweep. That's roughly 2N + 1 mutex calls per N transactions — well below contention thresholds for any realistic workload. When the GC profile actually shows the registry on the hot path, a sharded structure is the obvious upgrade; until then, simple wins.
+
+**Plan-doc reference.** [`concurrent-writes-plan.md`](concurrent-writes-plan.md) §4.1 (clock) and §4.2 (version index — the registry is an extracted slice of MvStore that's standalone in 11.2). Phase 11.3 is the first in-tree consumer.
+
+---
+
 ## Query execution
 
 ### 13. `NULL`-as-false in `WHERE` clauses

docs/file-format.md

@@ -326,14 +326,27 @@ A second file alongside the `.sqlrite`, named `<stem>.sqlrite-wal`, records page
 │ offset │ length │ content                                         │
 ├────────┼────────┼─────────────────────────────────────────────────┤
 │     0  │    8   │ magic:  "SQLRWAL\0"                             │
-│     8  │    4   │ format version (u32 LE) = 1                     │
+│     8  │    4   │ format version (u32 LE)                         │
+│        │        │   1 = pre-Phase-11                              │
+│        │        │   2 = Phase 11.2 — adds clock_high_water        │
 │    12  │    4   │ page size      (u32 LE) = 4096                  │
 │    16  │    4   │ salt (u32 LE) — rolled each checkpoint          │
 │    20  │    4   │ checkpoint seq (u32 LE) — increments per ckpt   │
-│    24  │    8   │ reserved / zero                                 │
+│    24  │    8   │ clock_high_water (u64 LE) — Phase 11.2 MVCC     │
+│        │        │   logical-clock high-water mark; rewritten on   │
+│        │        │   each checkpoint. v1 left these bytes as zero  │
+│        │        │   (read identically as "no timestamps issued"). │
 └────────┴────────┴─────────────────────────────────────────────────┘
 ```
 
+**v1 → v2 compatibility.** The bytes that now hold `clock_high_water`
+were reserved-zero in v1, so a pre-Phase-11 WAL opens cleanly: the
+parser interprets the zeros as `clock_high_water = 0`, which is
+indistinguishable from "fresh checkpoint, clock has never advanced."
+The next checkpoint rewrites the header at v2 — there's no offline
+upgrade step. Forward versions we don't recognize (e.g. v3) error
+out with a clean diagnostic rather than misinterpreting the bytes.
+
 ### Frames
 
 Each frame is `FRAME_HEADER_SIZE + PAGE_SIZE` = **4112 bytes**:

docs/roadmap.md

@@ -585,13 +585,19 @@ End-to-end SQLR-4 / SQLR-16 bench harness with twelve workloads across three gro
 
 Lift SQLRite past SQLite's single-writer ceiling with multi-version concurrency control and a `BEGIN CONCURRENT` transaction mode, modelled on Turso's experimental MVCC. The plan doc internally numbers sub-phases as "Phase 10.x" (its working title before the roadmap renumbering); they're listed under Phase 11 here because Phase 10 already shipped.
 
-### 🚧 Phase 11.1 — Multi-connection foundation *(in progress, plan-doc "Phase 10.1")*
+### ✅ Phase 11.1 — Multi-connection foundation *(plan-doc "Phase 10.1")*
 
-`Connection` is now a thin handle backed by `Arc<Mutex<Database>>`. Call [`Connection::connect`] to mint a sibling that shares the same engine state — typically one per worker thread. The headline contract: `Connection` is `Send + Sync`, and the engine no longer requires the caller to wrap the public API in their own `Mutex`. Today every operation still serializes through the per-database mutex (and the pager's existing process-level flock), so the behaviour change is *capability*, not throughput; concurrent throughput arrives with `BEGIN CONCURRENT` in 11.4.
+`Connection` is a thin handle backed by `Arc<Mutex<Database>>`. Call [`Connection::connect`] to mint a sibling that shares the same engine state — typically one per worker thread. The headline contract: `Connection` is `Send + Sync`, and the engine no longer requires the caller to wrap the public API in their own `Mutex`. Today every operation still serializes through the per-database mutex (and the pager's existing process-level flock), so the behaviour change is *capability*, not throughput; concurrent throughput arrives with `BEGIN CONCURRENT` in 11.4.
 
-### Phase 11.2 — Logical clock + active-tx registry *(planned)*
+### 🚧 Phase 11.2 — Logical clock + active-tx registry *(in progress, plan-doc "Phase 10.2")*
 
-`MvccClock` (`AtomicU64`) hands out begin / commit timestamps; `ActiveTxRegistry` exposes `min_active_begin_ts()` for GC. WAL header bumps from v1 → v2 to persist the high-water mark.
+New [`sqlrite::mvcc`](../src/mvcc/) module:
+
+- `MvccClock` — process-wide monotonic `u64` over `AtomicU64`. `tick()` hands out begin- / commit-timestamps; `now()` reads the high-water without advancing it; `observe(value)` advances the clock to `value` if greater (used at WAL replay).
+- `ActiveTxRegistry` — `Mutex<BTreeMap>` over in-flight transactions. `register(&clock)` allocates a `TxId`, snapshots `begin_ts`, and returns a RAII `TxHandle`; `min_active_begin_ts()` answers Phase 11.6 GC's "what's still possibly visible" question.
+- `TxId` newtype + `TxTimestampOrId` tagged union — defined now so 11.4 can plug in without re-litigating the type shape.
+
+WAL format bumps **v1 → v2**: bytes 24..32 of the WAL header (previously reserved-zero) now carry the persisted `clock_high_water` `u64`. v1 WALs open cleanly — those zero bytes read as "clock never advanced" — and the next checkpoint rewrites the header at v2. No offline upgrade step. `Wal::set_clock_high_water` / `Wal::clock_high_water` accessors expose the field; the setter rejects regressions with a typed error. The clock isn't wired into the executor yet (that's 11.3); the persistence + restore plumbing is in place so 11.3 just reads the high-water at open and seeds the in-memory clock.
 
 ### Phase 11.3 — `MvStore` skeleton + snapshot-isolation reads *(planned)*
 

src/lib.rs

@@ -61,6 +61,12 @@ extern crate prettytable;
 pub mod ask;
 pub mod connection;
 pub mod error;
+// Phase 11.2 — multi-version concurrency control primitives. The
+// module is public because Phase 11.3 / 11.4 grow externally-visible
+// types (timestamps, busy errors) on top of these foundations; today
+// it just exposes the standalone clock + active-tx registry. See
+// `docs/concurrent-writes-plan.md`.
+pub mod mvcc;
 pub mod sql;
 
 // Phase 5a public API.

src/mvcc/clock.rs

@@ -0,0 +1,182 @@
+//! [`MvccClock`] — the logical-clock primitive that hands out
+//! begin- and commit-timestamps for MVCC transactions (Phase 11.2).
+//!
+//! Per [`docs/concurrent-writes-plan.md`](../../../docs/concurrent-writes-plan.md):
+//!
+//! > A monotonic `u64` counter, per-`Database`. Hands out `begin_ts`
+//! > at `BEGIN CONCURRENT` and `commit_ts` at the start of validation.
+//! > Wrapped in `AtomicU64`; no contention because each transaction
+//! > calls it twice.
+//!
+//! The clock is persisted to the WAL header on each checkpoint so
+//! reopens resume past the highest committed timestamp — see
+//! [`crate::sql::pager::wal::WalHeader::clock_high_water`]. Without
+//! persistence, two transactions on either side of a reopen could
+//! receive the same timestamp and the snapshot-isolation visibility
+//! rule (`begin <= ts < end`) would mis-classify one of them.
+
+use std::sync::atomic::{AtomicU64, Ordering};
+
+/// Process-wide logical clock. Cheap to clone — internally an `Arc`
+/// over an [`AtomicU64`] in the [`Database`](crate::Database) wiring
+/// (added in Phase 11.3). Standalone today.
+#[derive(Debug, Default)]
+pub struct MvccClock {
+    counter: AtomicU64,
+}
+
+impl MvccClock {
+    /// Builds a clock seeded at `initial`. The next [`MvccClock::tick`]
+    /// returns `initial + 1`.
+    ///
+    /// Use this with the value persisted in the WAL header at open
+    /// time so the clock resumes past the last-checkpointed
+    /// high-water mark.
+    pub fn new(initial: u64) -> Self {
+        Self {
+            counter: AtomicU64::new(initial),
+        }
+    }
+
+    /// Returns the current high-water timestamp without advancing it.
+    /// Phase 11.6's GC reads this alongside
+    /// [`super::ActiveTxRegistry::min_active_begin_ts`] to decide
+    /// which row-version chains are reclaimable.
+    pub fn now(&self) -> u64 {
+        self.counter.load(Ordering::Acquire)
+    }
+
+    /// Bumps the clock by one and returns the new value. Strictly
+    /// monotonic: every call observes a distinct `u64`.
+    pub fn tick(&self) -> u64 {
+        // `fetch_add` returns the *previous* value — adjust to "after"
+        // semantics so callers see "the timestamp this call hands out".
+        // Wrap-around is impossible in practice (a billion ticks/s for
+        // 600 years still fits in `u64`), so saturating-add isn't
+        // needed.
+        self.counter.fetch_add(1, Ordering::AcqRel) + 1
+    }
+
+    /// Promotes the clock to at least `value`. No-op if `value` is at
+    /// or below the current high-water mark. Used at WAL replay to
+    /// bring the in-memory clock up to the persisted high-water
+    /// without an extra `tick()`.
+    pub fn observe(&self, value: u64) {
+        let mut current = self.counter.load(Ordering::Acquire);
+        while value > current {
+            // CAS rather than `store` — racing observers shouldn't
+            // step on each other and shouldn't move the clock
+            // backwards if a faster `tick` already overtook them.
+            match self.counter.compare_exchange_weak(
+                current,
+                value,
+                Ordering::AcqRel,
+                Ordering::Acquire,
+            ) {
+                Ok(_) => return,
+                Err(actual) => current = actual,
+            }
+        }
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+    use std::sync::Arc;
+    use std::thread;
+
+    #[test]
+    fn new_seeds_the_counter() {
+        let c = MvccClock::new(42);
+        assert_eq!(c.now(), 42);
+        assert_eq!(c.tick(), 43);
+        assert_eq!(c.now(), 43);
+    }
+
+    #[test]
+    fn default_starts_at_zero() {
+        let c = MvccClock::default();
+        assert_eq!(c.now(), 0);
+        assert_eq!(c.tick(), 1);
+    }
+
+    #[test]
+    fn tick_is_strictly_monotonic_within_a_thread() {
+        let c = MvccClock::new(0);
+        let mut last = 0;
+        for _ in 0..1_000 {
+            let t = c.tick();
+            assert!(t > last, "tick went backwards: {t} after {last}");
+            last = t;
+        }
+    }
+
+    #[test]
+    fn observe_only_moves_forward() {
+        let c = MvccClock::new(100);
+        c.observe(50); // ignored — below current
+        assert_eq!(c.now(), 100);
+        c.observe(200);
+        assert_eq!(c.now(), 200);
+        c.observe(150); // ignored — below current
+        assert_eq!(c.now(), 200);
+    }
+
+    /// Concurrent ticks across N threads must hand out N × M distinct
+    /// values (no duplicates, no skipped values). This is the property
+    /// MVCC visibility relies on.
+    #[test]
+    fn ticks_are_unique_under_contention() {
+        const THREADS: usize = 8;
+        const PER_THREAD: usize = 250;
+        let clock = Arc::new(MvccClock::new(0));
+
+        let handles: Vec<_> = (0..THREADS)
+            .map(|_| {
+                let c = Arc::clone(&clock);
+                thread::spawn(move || {
+                    let mut out = Vec::with_capacity(PER_THREAD);
+                    for _ in 0..PER_THREAD {
+                        out.push(c.tick());
+                    }
+                    out
+                })
+            })
+            .collect();
+
+        let mut all = Vec::with_capacity(THREADS * PER_THREAD);
+        for h in handles {
+            all.extend(h.join().unwrap());
+        }
+        all.sort_unstable();
+        // No duplicates.
+        for w in all.windows(2) {
+            assert_ne!(w[0], w[1], "duplicate timestamp {}", w[0]);
+        }
+        // Range is contiguous 1..=THREADS*PER_THREAD (clock seeded at 0).
+        assert_eq!(all.first().copied(), Some(1));
+        assert_eq!(all.last().copied(), Some((THREADS * PER_THREAD) as u64));
+    }
+
+    /// Concurrent `observe`s must not move the clock backwards.
+    #[test]
+    fn observe_under_contention_never_regresses() {
+        const THREADS: usize = 8;
+        let clock = Arc::new(MvccClock::new(0));
+        let handles: Vec<_> = (0..THREADS)
+            .map(|tid| {
+                let c = Arc::clone(&clock);
+                thread::spawn(move || {
+                    // Each thread observes a deterministic distinct
+                    // value; the clock should end at the max.
+                    c.observe((tid as u64 + 1) * 1_000);
+                })
+            })
+            .collect();
+        for h in handles {
+            h.join().unwrap();
+        }
+        assert_eq!(clock.now(), THREADS as u64 * 1_000);
+    }
+}

src/mvcc/mod.rs

@@ -0,0 +1,30 @@
+//! Multi-version concurrency control primitives (Phase 11).
+//!
+//! This module is the foundation for SQLRite's `BEGIN CONCURRENT` story
+//! — see [`docs/concurrent-writes-plan.md`](../../docs/concurrent-writes-plan.md)
+//! for the full sequenced design. As of **Phase 11.2** it carries the
+//! standalone primitives that the rest of the work hangs off:
+//!
+//! - [`MvccClock`] — a process-wide monotonic `u64` counter that hands
+//!   out begin- and commit-timestamps. Persisted to the WAL header so
+//!   timestamps don't reuse the same value across reopens.
+//! - [`ActiveTxRegistry`] — tracks the begin-timestamps of in-flight
+//!   transactions. Garbage collection (Phase 11.6) needs
+//!   [`ActiveTxRegistry::min_active_begin_ts`] to know which versions
+//!   are still possibly visible to a live reader.
+//! - [`TxId`] — opaque newtype around a `u64`, allocated by the clock
+//!   while a transaction is in flight. After commit the same value is
+//!   reused as the row version's `begin` timestamp; the discriminator
+//!   between "in-flight transaction id" and "committed timestamp"
+//!   lives in [`TxTimestampOrId`].
+//!
+//! Nothing in the executor reads from these yet — Phase 11.3 wires
+//! them into a new `MvStore` in front of the pager. Keeping the
+//! plumbing standalone in 11.2 means the Phase 11.4 `BEGIN CONCURRENT`
+//! work can pull them in without re-litigating the foundation.
+
+pub mod clock;
+pub mod registry;
+
+pub use clock::MvccClock;
+pub use registry::{ActiveTxRegistry, TxHandle, TxId, TxTimestampOrId};