diff --git a/docs/_index.md b/docs/_index.md index 0b5e7b7..9e5fadb 100644 --- a/docs/_index.md +++ b/docs/_index.md @@ -54,6 +54,7 @@ As of May 2026, SQLRite has: - Full-text search + hybrid retrieval (Phase 8 complete): FTS5-style inverted index with BM25 ranking + `fts_match` / `bm25_score` scalar functions + `try_fts_probe` optimizer hook + on-disk persistence with on-demand v4 → v5 file-format bump (8a-8c), a worked hybrid-retrieval example combining BM25 with vector cosine via raw arithmetic (8d), and a `bm25_search` MCP tool symmetric with `vector_search` (8e). See [`docs/fts.md`](fts.md). - SQL surface + DX follow-ups (Phase 9 complete, v0.2.0 → v0.9.1): DDL completeness — `DEFAULT`, `DROP TABLE` / `DROP INDEX`, `ALTER TABLE` (9a); free-list + manual `VACUUM` (9b) + auto-VACUUM (9c); `IS NULL` / `IS NOT NULL` (9d); `GROUP BY` + aggregates + `DISTINCT` + `LIKE` + `IN` (9e); four flavors of `JOIN` — INNER, LEFT, RIGHT, FULL OUTER (9f); prepared statements + `?` parameter binding with a per-connection LRU plan cache (9g); HNSW probe widened to cosine + dot via `WITH (metric = …)` (9h); `PRAGMA` dispatcher with the `auto_vacuum` knob (9i) - Benchmarks against SQLite + DuckDB (Phase 10 complete, SQLR-4 / SQLR-16): twelve-workload bench harness with a pluggable `Driver` trait, criterion-driven, pinned-host runs published. See [`docs/benchmarks.md`](benchmarks.md). +- Phase 11 (concurrent writes via MVCC + `BEGIN CONCURRENT`, SQLR-22) is in flight. **11.1 multi-connection foundation: shipped on this branch.** `Connection` is now `Send + Sync` and `Connection::connect()` mints a sibling handle that shares the same backing `Database`. Snapshot-isolation reads + `BEGIN CONCURRENT` writes follow in 11.3 / 11.4. Plan: [`docs/concurrent-writes-plan.md`](concurrent-writes-plan.md). - A fully-automated release pipeline that ships every product to its registry on every release with one human action — Rust engine + `sqlrite-ask` + `sqlrite-mcp` to crates.io, Python wheels to PyPI (`sqlrite`), Node.js + WASM to npm (`@joaoh82/sqlrite` + `@joaoh82/sqlrite-wasm`), Go module via `sdk/go/v*` git tag, plus C FFI tarballs, MCP binary tarballs, and unsigned desktop installers as GitHub Release assets (Phase 6 complete) See the [Roadmap](roadmap.md) for the full phase plan. diff --git a/docs/design-decisions.md b/docs/design-decisions.md index 2c01221..6dbe652 100644 --- a/docs/design-decisions.md +++ b/docs/design-decisions.md @@ -144,6 +144,18 @@ Decisions are grouped by the engine layer they concern: parser, storage, concurr --- +### 12a. `Connection` as a thin handle over `Arc>` (Phase 11.1) + +**Decision.** `Connection` no longer owns a `Database` by value; it holds `Arc>` plus a per-handle prepared-statement LRU. A new `Connection::connect()` mints a sibling handle that shares the same backing engine state. The mutex is acquired transparently at the entry of every public method (`execute`, `prepare`, `database()`, accessors); statements release it between calls. `Connection: Send + Sync`. + +**Why.** Phase 11 (concurrent writes via MVCC + `BEGIN CONCURRENT`) needs more than one connection to address the same in-memory tables and pager from inside the same process. The previous shape (`Connection { db: Database, … }`) made callers wrap the whole connection in their own `Mutex`, which works for single-writer workloads but collapses if the public API needs to grow a "this transaction is concurrent" mode that other handles can observe. Lifting the mutex into the engine itself is the minimum change that lets `BEGIN CONCURRENT` and snapshot-isolated reads hook in cleanly later. + +**Cost.** Every public-API call now takes one extra atomic CAS (`Mutex::lock`). Negligible against the work each call does (parser pass + executor + optional fsync). The internal `&mut Database` plumbing is unchanged — over a hundred call sites across the executor, parser, and pager keep the same signatures because they run under the lock the public method already acquired. `Connection::database()` now returns a `MutexGuard<'_, Database>` instead of `&Database`; this is `#[doc(hidden)]` and explicitly unstable, but downstream callers (mainly the MCP server tools and SDK shims) had to bind it to a local first, which they were mostly already doing. The prepared-statement cache deliberately stays per-handle so each thread's hot SQL doesn't fight an extra mutex on every `prepare_cached`. + +**What this doesn't do (yet).** Phase 11.1 is *capability*, not throughput. Two writers from sibling handles still serialize through the per-database mutex (and the existing pager `flock` between processes). The `BEGIN CONCURRENT` SQL surface, the `MvStore` version index, and snapshot-isolation reads land in 11.3–11.4. See [`concurrent-writes-plan.md`](concurrent-writes-plan.md) for the sequenced plan. + +--- + ## Query execution ### 13. `NULL`-as-false in `WHERE` clauses diff --git a/docs/embedding.md b/docs/embedding.md index 93b3b6e..726368a 100644 --- a/docs/embedding.md +++ b/docs/embedding.md @@ -69,6 +69,31 @@ if looks_good { See [Phase 4f notes in roadmap.md](roadmap.md) for the snapshot semantics and the auto-rollback-on-failed-COMMIT guarantee. +### Sharing one database across threads + +*Phase 11.1.* `Connection` is a thin handle over the engine state. Call `Connection::connect()` to mint a sibling handle that shares the same in-memory tables and persistent pager — typically one handle per worker thread. `Connection: Send + Sync`, so the handles can be moved across threads without an outer `Mutex`. + +```rust +use sqlrite::Connection; + +let mut primary = Connection::open("foo.sqlrite")?; +primary.execute("CREATE TABLE log (id INTEGER PRIMARY KEY, who TEXT);")?; + +let mut writers = Vec::new(); +for tid in 0..4 { + let mut conn = primary.connect(); // sibling handle, same backing DB + writers.push(std::thread::spawn(move || { + conn.execute(&format!("INSERT INTO log (who) VALUES ('t{tid}');")) + })); +} +for h in writers { h.join().unwrap()?; } +# Ok::<(), sqlrite::SQLRiteError>(()) +``` + +Today every commit still serializes through the per-database mutex (and the pager's existing process-level `flock`); the goal of 11.1 is *capability*, not throughput. True multi-writer throughput on disjoint rows arrives with `BEGIN CONCURRENT` in 11.4 — see [`concurrent-writes-plan.md`](concurrent-writes-plan.md). + +Per-handle state — the prepared-statement cache (LRU populated by `prepare_cached`), the cache capacity setter — stays on each handle, by design (no extra mutex traffic for a per-thread accelerator). The shared state is the `Database` (tables, pager, transaction snapshot, auto-VACUUM threshold). + ### What's deferred - **Parameter binding.** `stmt.query(&[&"alice"])` is the intended shape but the current implementation takes no arguments — use string interpolation for now. Parameter binding lands with the cursor refactor. diff --git a/docs/roadmap.md b/docs/roadmap.md index f47bc93..35b51a2 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -2,7 +2,7 @@ The project is staged in phases. Each phase is shippable on its own, ends with a working build + full test suite + a commit on `main`, and can be paused between. The README's roadmap section is a summary of this doc. -> **Active frontier (May 2026):** Phases 0–10 shipped end-to-end. After Phase 8 closed the v0.1.x cycle, the v0.2.0 → v0.9.1 wave (Phase 9, sub-phases 9a–9i) landed the SQL surface that had been parked under "possible extras": DDL completeness (DEFAULT, DROP TABLE/INDEX, ALTER TABLE), free-list + auto-VACUUM, IS NULL, GROUP BY + aggregates + DISTINCT + LIKE + IN, four flavors of JOIN, prepared statements with parameter binding, HNSW metric extension, and the PRAGMA dispatcher. Phase 10 published the SQLR-4 / SQLR-16 benchmarks against SQLite + DuckDB. **Current head: v0.9.1.** The roadmap from here is the smaller "possible extras" list at the bottom of this doc plus the 5a.2 cursor refactor. +> **Active frontier (May 2026):** Phases 0–10 shipped end-to-end. After Phase 8 closed the v0.1.x cycle, the v0.2.0 → v0.9.1 wave (Phase 9, sub-phases 9a–9i) landed the SQL surface that had been parked under "possible extras": DDL completeness (DEFAULT, DROP TABLE/INDEX, ALTER TABLE), free-list + auto-VACUUM, IS NULL, GROUP BY + aggregates + DISTINCT + LIKE + IN, four flavors of JOIN, prepared statements with parameter binding, HNSW metric extension, and the PRAGMA dispatcher. Phase 10 published the SQLR-4 / SQLR-16 benchmarks against SQLite + DuckDB. **Current head: v0.9.1.** Phase 11 (concurrent writes via MVCC + `BEGIN CONCURRENT`, SQLR-22) is now in flight — the multi-connection foundation (11.1) is the first slice; see [`concurrent-writes-plan.md`](concurrent-writes-plan.md) for the full design. ## ✅ Phase 0 — Modernization @@ -581,6 +581,46 @@ Every executable statement accepts `?` placeholders anywhere a value literal is End-to-end SQLR-4 / SQLR-16 bench harness with twelve workloads across three groups (read-by-PK, transactional CRUD, analytical slices, vector / FTS retrieval). Pluggable `Driver` trait + bundled SQLite + DuckDB drivers; criterion-based; pinned-host runs published at [`docs/benchmarks.md`](benchmarks.md). Excluded from CI (criterion is too noisy on shared runners; `rusqlite-bundled` is heavy). See [`docs/benchmarks-plan.md`](benchmarks-plan.md) for the design and PRs #102–#114 for the staged rollout. +## Phase 11 — Concurrent writes via MVCC + `BEGIN CONCURRENT` *(SQLR-22; in flight — see [`concurrent-writes-plan.md`](concurrent-writes-plan.md))* + +Lift SQLRite past SQLite's single-writer ceiling with multi-version concurrency control and a `BEGIN CONCURRENT` transaction mode, modelled on Turso's experimental MVCC. The plan doc internally numbers sub-phases as "Phase 10.x" (its working title before the roadmap renumbering); they're listed under Phase 11 here because Phase 10 already shipped. + +### 🚧 Phase 11.1 — Multi-connection foundation *(in progress, plan-doc "Phase 10.1")* + +`Connection` is now a thin handle backed by `Arc>`. Call [`Connection::connect`] to mint a sibling that shares the same engine state — typically one per worker thread. The headline contract: `Connection` is `Send + Sync`, and the engine no longer requires the caller to wrap the public API in their own `Mutex`. Today every operation still serializes through the per-database mutex (and the pager's existing process-level flock), so the behaviour change is *capability*, not throughput; concurrent throughput arrives with `BEGIN CONCURRENT` in 11.4. + +### Phase 11.2 — Logical clock + active-tx registry *(planned)* + +`MvccClock` (`AtomicU64`) hands out begin / commit timestamps; `ActiveTxRegistry` exposes `min_active_begin_ts()` for GC. WAL header bumps from v1 → v2 to persist the high-water mark. + +### Phase 11.3 — `MvStore` skeleton + snapshot-isolation reads *(planned)* + +In-memory version index + `PRAGMA journal_mode = mvcc` opt-in. Lazy-loads versions from the pager on first touch. Writes still go through the legacy path — only reads change. + +### Phase 11.4 — `BEGIN CONCURRENT` writes + commit-time validation *(planned, the meat)* + +Parser maps `BEGIN CONCURRENT` to `TxKind::Concurrent`; writes land in the MvStore's write-set; commit walks the write-set checking for newer versions. New `SQLRiteError::Busy` / `BusySnapshot` variants. New WAL log-record frame kind. + +### Phase 11.5 — Checkpoint integration + crash recovery *(planned)* + +Drains MVCC log-records into the existing bottom-up B-tree rebuild path. Replay on reopen rebuilds the in-memory index. + +### Phase 11.6 — Garbage collection *(planned)* + +Per-commit sweep over the write-set's chains, plus a background sweep behind `PRAGMA mvcc_gc_interval_ms`. + +### Phase 11.7 — Indexes under MVCC *(deferred-by-design, separate later phase)* + +Each secondary-index entry becomes its own `RowVersion`. Turso explicitly punted on this; SQLRite's v0 will reject `CREATE INDEX` while `journal_mode = mvcc`. + +### Phase 11.8 — SDK + REPL propagation *(planned)* + +Surface `Busy` / `BusySnapshot` through the FFI shim and each language SDK. New REPL `.spawn` meta-command + new "N concurrent writers" benchmark workload. + +### Phase 11.9 — Docs *(planned)* + +Promote the plan to `docs/concurrent-writes.md` and update the cross-references. + ## "Possible extras" not pinned to a phase The remaining items — actually open, not retroactively rewritten: @@ -592,7 +632,6 @@ The remaining items — actually open, not retroactively rewritten: - Multi-column / expression `ORDER BY`, `OFFSET`, `NULLS FIRST/LAST` - `UNION` / `INTERSECT` / `EXCEPT`, `INSERT ... SELECT` - Composite + expression indexes -- Concurrent writes via MVCC + `BEGIN CONCURRENT` — design sketch in [`docs/concurrent-writes-plan.md`](concurrent-writes-plan.md) - `CREATE VIEW`, `CREATE TRIGGER`, `FOREIGN KEY`, `CHECK`, table-level / composite constraints - Savepoints + isolation-level control (`BEGIN IMMEDIATE` / `BEGIN EXCLUSIVE`) - Built-in scalar functions (`LENGTH`, `UPPER`, `LOWER`, `COALESCE`, `IFNULL`, date/time, `printf`, …) diff --git a/sdk/nodejs/src/lib.rs b/sdk/nodejs/src/lib.rs index ad0a224..1d64f23 100644 --- a/sdk/nodejs/src/lib.rs +++ b/sdk/nodejs/src/lib.rs @@ -261,7 +261,8 @@ impl Database { let conn = borrow .as_ref() .ok_or_else(|| napi::Error::from_reason("cannot ask: database is closed"))?; - let resp = ask_with_database(conn.database(), &question, &resolved).map_err(map_err)?; + let db = conn.database(); + let resp = ask_with_database(&db, &question, &resolved).map_err(map_err)?; Ok(AskResponse { sql: resp.sql, explanation: resp.explanation, diff --git a/sdk/python/src/lib.rs b/sdk/python/src/lib.rs index b098af7..892b1a9 100644 --- a/sdk/python/src/lib.rs +++ b/sdk/python/src/lib.rs @@ -321,7 +321,10 @@ impl Connection { .lock() .map_err(|_| SQLRiteError::new_err("connection mutex poisoned"))?; let resp = py - .allow_threads(|| ask_with_database(locked.database(), question, &resolved)) + .allow_threads(|| { + let db = locked.database(); + ask_with_database(&db, question, &resolved) + }) .map_err(map_err)?; Ok(AskResponse::from_rust(resp)) } diff --git a/sdk/wasm/src/lib.rs b/sdk/wasm/src/lib.rs index 8a60117..805622f 100644 --- a/sdk/wasm/src/lib.rs +++ b/sdk/wasm/src/lib.rs @@ -255,7 +255,9 @@ impl Database { } }; - let schema = dump_schema_for_database(self.inner.database()); + let db = self.inner.database(); + let schema = dump_schema_for_database(&db); + drop(db); let system_blocks = build_system(&schema, cache_marker); let messages = vec![PromptUserMessage { role: "user", diff --git a/sqlrite-mcp/src/tools/ask.rs b/sqlrite-mcp/src/tools/ask.rs index 528aa3b..e4d1bd2 100644 --- a/sqlrite-mcp/src/tools/ask.rs +++ b/sqlrite-mcp/src/tools/ask.rs @@ -118,9 +118,14 @@ pub fn handle(args: Value, state: &mut ServerState) -> Result }; } - // Run the LLM call. - let resp: AskResponse = ask_with_database(state.conn.database(), &args.question, &cfg) - .map_err(|e| ToolError::new(format!("ask failed: {e}")))?; + // Run the LLM call. Hold the database guard across the call but + // drop it before any later `state.conn.execute` / `prepare` — + // those re-acquire the same lock. + let resp: AskResponse = { + let db = state.conn.database(); + ask_with_database(&db, &args.question, &cfg) + .map_err(|e| ToolError::new(format!("ask failed: {e}")))? + }; let mut result = json!({ "sql": resp.sql, diff --git a/sqlrite-mcp/src/tools/schema_dump.rs b/sqlrite-mcp/src/tools/schema_dump.rs index 4021e08..5134ca5 100644 --- a/sqlrite-mcp/src/tools/schema_dump.rs +++ b/sqlrite-mcp/src/tools/schema_dump.rs @@ -28,6 +28,7 @@ pub fn metadata() -> Value { } pub fn handle(_args: Value, state: &mut ServerState) -> Result { - let dump = sqlrite::ask::schema::dump_schema_for_database(state.conn.database()); + let db = state.conn.database(); + let dump = sqlrite::ask::schema::dump_schema_for_database(&db); Ok(dump) } diff --git a/src/ask/mod.rs b/src/ask/mod.rs index 6753d1a..767f075 100644 --- a/src/ask/mod.rs +++ b/src/ask/mod.rs @@ -104,7 +104,8 @@ impl ConnectionAskExt for Connection { /// pick whichever shape reads better at the call site. #[cfg(feature = "ask")] pub fn ask(conn: &Connection, question: &str, config: &AskConfig) -> Result { - ask_with_database(conn.database(), question, config) + let db = conn.database(); + ask_with_database(&db, question, config) } /// Same as [`ask`], but takes the engine's `&Database` directly. @@ -131,7 +132,8 @@ pub fn ask_with_provider( config: &AskConfig, provider: &P, ) -> Result { - ask_with_database_and_provider(conn.database(), question, config, provider) + let db = conn.database(); + ask_with_database_and_provider(&db, question, config, provider) } /// Lower-level entry taking `&Database` and a provider. Canonical diff --git a/src/ask/schema.rs b/src/ask/schema.rs index d91b243..0c07730 100644 --- a/src/ask/schema.rs +++ b/src/ask/schema.rs @@ -31,7 +31,8 @@ use crate::sql::db::table::{DataType, Table}; /// sequence of `CREATE TABLE … (…);` statements, sorted /// alphabetically by table name. pub fn dump_schema_for_connection(conn: &Connection) -> String { - dump_schema_for_database(conn.database()) + let db = conn.database(); + dump_schema_for_database(&db) } /// Same as [`dump_schema_for_connection`], but takes a `Database` diff --git a/src/connection.rs b/src/connection.rs index 8d72c5a..1cc94a0 100644 --- a/src/connection.rs +++ b/src/connection.rs @@ -52,7 +52,7 @@ use std::collections::VecDeque; use std::path::Path; -use std::sync::Arc; +use std::sync::{Arc, Mutex, MutexGuard}; use crate::sql::dialect::SqlriteDialect; use sqlparser::ast::Statement as AstStatement; @@ -87,16 +87,36 @@ const DEFAULT_PREP_CACHE_CAP: usize = 16; /// # Ok::<(), sqlrite::SQLRiteError>(()) /// ``` /// -/// `Connection` is `Send` but not `Sync` — clone it (it's currently -/// unclonable) or share via a `Mutex` if you need -/// multi-threaded access. +/// ## Multiple connections (Phase 10.1) +/// +/// `Connection` is a thin handle over an `Arc>`. Call +/// [`Connection::connect`] to mint a sibling handle that shares the +/// same backing `Database` — typically one per worker thread. Today +/// every operation still serializes through the single mutex (and the +/// pager's exclusive flock between processes), so the headline +/// behaviour change is that callers can hold and address the same DB +/// from more than one thread without wrapping the whole `Connection` +/// in a `Mutex` themselves. `BEGIN CONCURRENT` and snapshot-isolated +/// reads land in subsequent Phase 10 sub-phases. +/// +/// `Connection` is `Send + Sync`. The recommended pattern is one +/// connection per thread (clone via `connect()`); statements still +/// borrow `&mut Connection`, so a single connection isn't suitable +/// for true concurrent statement execution. pub struct Connection { - db: Database, + /// Shared engine state. Mints sibling connections via + /// [`Connection::connect`] without copying the in-memory tables + /// or the long-lived pager. + inner: Arc>, /// SQLR-23 — small SQL→cached-plan LRU. Keyed by the verbatim SQL /// string the caller passed to `prepare_cached`. Stored as a /// `VecDeque` rather than a HashMap+linked-list because the /// expected capacity is small (default 16) — linear scan is fine /// and the implementation stays dependency-free. + /// + /// Per-connection (not shared with sibling handles) — each thread + /// gets its own LRU so cache-mutation never crosses a thread + /// boundary. prep_cache: VecDeque<(String, Arc)>, prep_cache_cap: usize, } @@ -156,12 +176,67 @@ impl Connection { fn wrap(db: Database) -> Self { Self { - db, + inner: Arc::new(Mutex::new(db)), prep_cache: VecDeque::new(), prep_cache_cap: DEFAULT_PREP_CACHE_CAP, } } + /// Phase 10.1 — mints another `Connection` sharing the same + /// backing `Database`. Hand the returned handle to a separate + /// thread to address the same in-memory tables and persistent + /// pager from there. + /// + /// The new handle starts with an empty prepared-statement cache + /// (caches are per-handle, by design). Inherits the parent's + /// `prepare_cached` capacity. Concurrent operations still + /// serialize through the engine's internal lock and the pager's + /// existing single-writer rule — a true multi-writer story + /// arrives with `BEGIN CONCURRENT` in Phase 10.4. + /// + /// ```no_run + /// # use sqlrite::Connection; + /// let mut primary = Connection::open("foo.sqlrite")?; + /// let secondary = primary.connect(); + /// std::thread::spawn(move || { + /// let mut conn = secondary; + /// conn.execute("INSERT INTO t (x) VALUES (1)").unwrap(); + /// }) + /// .join() + /// .unwrap(); + /// # Ok::<(), sqlrite::SQLRiteError>(()) + /// ``` + pub fn connect(&self) -> Self { + Self { + inner: Arc::clone(&self.inner), + prep_cache: VecDeque::new(), + prep_cache_cap: self.prep_cache_cap, + } + } + + /// Phase 10.1 — number of `Connection` handles currently sharing + /// this database (this handle plus every live `connect()` + /// descendant). Useful for diagnostics and tests; no semantic + /// guarantee beyond that. + pub fn handle_count(&self) -> usize { + Arc::strong_count(&self.inner) + } + + /// Locks the shared `Database` and returns the guard. Internal + /// helper — every public method that needs `&mut Database` calls + /// this. The lock is released when the guard drops, so callers + /// must keep the guard alive for the duration of the engine call + /// (typically by binding it to a local). + fn lock(&self) -> MutexGuard<'_, Database> { + // `unwrap` propagates a panic from another thread that held + // the lock — there's no engine-level recovery story for a + // poisoned `Database` (the in-memory tables would be in an + // unknown state), so failing fast is the right behaviour. + self.inner + .lock() + .unwrap_or_else(|e| panic!("sqlrite: database mutex poisoned: {e}")) + } + /// Parses and executes one SQL statement. For DDL (`CREATE TABLE`, /// `CREATE INDEX`), DML (`INSERT`, `UPDATE`, `DELETE`) and /// transaction control (`BEGIN`, `COMMIT`, `ROLLBACK`). Returns @@ -172,7 +247,8 @@ impl Connection { /// just returns the rendered status — use [`Connection::prepare`] /// and [`Statement::query`] to iterate typed rows. pub fn execute(&mut self, sql: &str) -> Result { - crate::sql::process_command(sql, &mut self.db) + let mut db = self.lock(); + crate::sql::process_command(sql, &mut db) } /// Prepares a statement for repeated execution or row iteration. @@ -238,7 +314,7 @@ impl Connection { /// Returns `true` while a `BEGIN … COMMIT/ROLLBACK` block is open /// against this connection. pub fn in_transaction(&self) -> bool { - self.db.in_transaction() + self.lock().in_transaction() } /// Returns the current auto-VACUUM threshold (SQLR-10). After a @@ -248,50 +324,64 @@ impl Connection { /// default to `Some(0.25)` (SQLite parity); `None` means the /// trigger is disabled. See [`Connection::set_auto_vacuum_threshold`]. pub fn auto_vacuum_threshold(&self) -> Option { - self.db.auto_vacuum_threshold() + self.lock().auto_vacuum_threshold() } /// Sets the auto-VACUUM threshold (SQLR-10). `Some(t)` with `t` in /// `0.0..=1.0` arms the trigger; `None` disables it. Values outside /// `0.0..=1.0` (or NaN / infinite) return a typed error rather than - /// silently saturating. The setting is per-connection runtime - /// state — closing the connection drops it; new connections start - /// at the default `Some(0.25)`. + /// silently saturating. The setting is per-database runtime state — + /// closing the last connection to a database drops it; new + /// connections start at the default `Some(0.25)`. /// /// Calling this on an in-memory or read-only database is allowed /// (it just won't fire — there's nothing to compact / no writes /// will reach the trigger). pub fn set_auto_vacuum_threshold(&mut self, threshold: Option) -> Result<()> { - self.db.set_auto_vacuum_threshold(threshold) + self.lock().set_auto_vacuum_threshold(threshold) } /// Returns `true` if the connection was opened read-only. Mutating /// statements on a read-only connection return a typed error. pub fn is_read_only(&self) -> bool { - self.db.is_read_only() + self.lock().is_read_only() } - /// Escape hatch for advanced callers — the internal `Database` - /// backing this connection. Not part of the stable API; will move - /// or change as Phase 5's cursor abstraction lands. + /// Escape hatch for advanced callers — locks the shared `Database` + /// and hands back the guard. Not part of the stable API; will move + /// or change as Phase 10's MVCC sub-phases land. + /// + /// Bind the guard to a local before calling functions that take + /// `&Database`: + /// + /// ```no_run + /// # use sqlrite::Connection; + /// # fn use_db(_d: &sqlrite::Database) {} + /// let conn = Connection::open_in_memory()?; + /// let db = conn.database(); + /// use_db(&db); + /// # Ok::<(), sqlrite::SQLRiteError>(()) + /// ``` #[doc(hidden)] - pub fn database(&self) -> &Database { - &self.db + pub fn database(&self) -> MutexGuard<'_, Database> { + self.lock() } #[doc(hidden)] - pub fn database_mut(&mut self) -> &mut Database { - &mut self.db + pub fn database_mut(&mut self) -> MutexGuard<'_, Database> { + self.lock() } } impl std::fmt::Debug for Connection { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let db = self.lock(); f.debug_struct("Connection") - .field("in_transaction", &self.db.in_transaction()) - .field("read_only", &self.db.is_read_only()) - .field("tables", &self.db.tables.len()) + .field("in_transaction", &db.in_transaction()) + .field("read_only", &db.is_read_only()) + .field("tables", &db.tables.len()) .field("prep_cache_len", &self.prep_cache.len()) + .field("handles", &Arc::strong_count(&self.inner)) .finish() } } @@ -395,7 +485,8 @@ impl<'c> Statement<'c> { ))); } let ast = self.plan.ast.clone(); - process_ast_with_render(ast, &mut self.conn.db).map(|o| o.status) + let mut db = self.conn.lock(); + process_ast_with_render(ast, &mut db).map(|o| o.status) } /// SQLR-23 — executes a prepared non-SELECT statement after binding @@ -411,7 +502,8 @@ impl<'c> Statement<'c> { if !params.is_empty() { substitute_params(&mut ast, params)?; } - process_ast_with_render(ast, &mut self.conn.db).map(|o| o.status) + let mut db = self.conn.lock(); + process_ast_with_render(ast, &mut db).map(|o| o.status) } /// Runs a SELECT and returns a [`Rows`] iterator over typed rows. @@ -433,7 +525,8 @@ impl<'c> Statement<'c> { "query() only works on SELECT statements; use run() for DDL/DML".to_string(), )); }; - let result = execute_select_rows(sq.clone(), &self.conn.db)?; + let db = self.conn.lock(); + let result = execute_select_rows(sq.clone(), &db)?; Ok(Rows { columns: result.columns, rows: result.rows.into_iter(), @@ -466,7 +559,8 @@ impl<'c> Statement<'c> { substitute_params(&mut ast, params)?; } let sq = SelectQuery::new(&ast)?; - let result = execute_select_rows(sq, &self.conn.db)?; + let db = self.conn.lock(); + let result = execute_select_rows(sq, &db)?; Ok(Rows { columns: result.columns, rows: result.rows.into_iter(), @@ -1267,6 +1361,157 @@ mod tests { assert!(msg.contains("doesn't support any options"), "got: {msg}"); } + // ----------------------------------------------------------------- + // Phase 10.1 — multi-connection foundation + // ----------------------------------------------------------------- + + /// `connect()` mints a sibling handle that shares the backing + /// `Database`. Writes through one are visible through the other — + /// the headline behavioural change for Phase 10.1. + #[test] + fn connect_shares_underlying_database() { + let mut a = Connection::open_in_memory().unwrap(); + let mut b = a.connect(); + assert_eq!(a.handle_count(), 2); + + a.execute("CREATE TABLE shared (id INTEGER PRIMARY KEY, label TEXT);") + .unwrap(); + a.execute("INSERT INTO shared (label) VALUES ('via-a');") + .unwrap(); + b.execute("INSERT INTO shared (label) VALUES ('via-b');") + .unwrap(); + + let stmt = b.prepare("SELECT label FROM shared;").unwrap(); + let mut labels: Vec = stmt + .query() + .unwrap() + .collect_all() + .unwrap() + .into_iter() + .map(|r| r.get::(0).unwrap()) + .collect(); + labels.sort(); + assert_eq!(labels, vec!["via-a".to_string(), "via-b".to_string()]); + } + + /// Dropping a sibling decrements the handle count without + /// disturbing the surviving connections. + #[test] + fn handle_count_reflects_live_handles() { + let primary = Connection::open_in_memory().unwrap(); + assert_eq!(primary.handle_count(), 1); + let s1 = primary.connect(); + let s2 = primary.connect(); + assert_eq!(primary.handle_count(), 3); + drop(s1); + assert_eq!(primary.handle_count(), 2); + drop(s2); + assert_eq!(primary.handle_count(), 1); + } + + /// Multi-thread INSERT/COMMIT against the same in-memory DB. Today + /// the per-`Database` mutex serializes commits — this test proves + /// the locking holds without panics or data loss when N threads + /// race for the writer. Phase 10.4's `BEGIN CONCURRENT` will lift + /// the serialization for disjoint-row workloads; until then the + /// guarantee is "no panic, every commit lands." + #[test] + fn threaded_writers_serialize_cleanly() { + use std::thread; + + let primary = Connection::open_in_memory().unwrap(); + // Set up the shared schema before spawning so every worker + // sees the table. + { + let mut p = primary.connect(); + p.execute("CREATE TABLE log (id INTEGER PRIMARY KEY, who TEXT, n INTEGER);") + .unwrap(); + } + + const THREADS: usize = 8; + const PER_THREAD: usize = 25; + + let handles: Vec<_> = (0..THREADS) + .map(|tid| { + let mut conn = primary.connect(); + thread::spawn(move || { + for n in 0..PER_THREAD { + let sql = format!("INSERT INTO log (who, n) VALUES ('t{tid}', {n});"); + conn.execute(&sql).expect("insert under contention"); + } + }) + }) + .collect(); + + for h in handles { + h.join().expect("worker panicked"); + } + + // Every write must have landed exactly once — count rows by + // probing the table directly so we don't depend on a SELECT + // COUNT(*) implementation. + let db = primary.database(); + let table = db.get_table("log".to_string()).unwrap(); + assert_eq!( + table.rowids().len(), + THREADS * PER_THREAD, + "expected every threaded INSERT to commit", + ); + } + + /// `connect()` over a file-backed database produces sibling + /// handles that hit the same on-disk pager. Auto-save through one + /// must be visible through the other without a re-open. + #[test] + fn connect_shares_file_backed_database() { + let path = tmp_path("connect_file"); + let mut primary = Connection::open(&path).unwrap(); + primary + .execute("CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT);") + .unwrap(); + + let mut sibling = primary.connect(); + sibling.execute("INSERT INTO t (v) VALUES ('hi');").unwrap(); + + let stmt = primary.prepare("SELECT v FROM t;").unwrap(); + let rows = stmt.query().unwrap().collect_all().unwrap(); + assert_eq!(rows.len(), 1); + assert_eq!(rows[0].get::(0).unwrap(), "hi"); + + drop(sibling); + drop(primary); + cleanup(&path); + } + + /// Prepared-statement caches are per-handle, by design — sharing + /// a mutable LRU across threads would require an extra lock for + /// no real win (each worker prepares its own hot SQL). + #[test] + fn prep_cache_is_per_handle() { + let mut a = Connection::open_in_memory().unwrap(); + a.execute("CREATE TABLE t (a INTEGER);").unwrap(); + let mut b = a.connect(); + + let _ = a.prepare_cached("SELECT a FROM t").unwrap(); + let _ = a.prepare_cached("SELECT a FROM t").unwrap(); + assert_eq!(a.prepared_cache_len(), 1); + // The sibling's cache is untouched. + assert_eq!(b.prepared_cache_len(), 0); + let _ = b.prepare_cached("SELECT a FROM t").unwrap(); + assert_eq!(b.prepared_cache_len(), 1); + } + + /// Static check: `Connection` is `Send + Sync`. Required so it can + /// be moved across threads (or wrapped in `Arc`) without a typestate + /// adapter — the headline contract Phase 10.1 puts in place. + #[test] + fn connection_is_send_and_sync() { + fn assert_send() {} + fn assert_sync() {} + assert_send::(); + assert_sync::(); + } + #[test] fn prepare_cached_executes_the_same_as_prepare() { let mut conn = Connection::open_in_memory().unwrap();