Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,7 @@ sqlrite> DELETE FROM users WHERE age < 30;
| `UPDATE` | Multi-column `SET`; `WHERE`; UNIQUE + type enforcement; arithmetic in assignments (`SET age = age + 1`) |
| `DELETE` | `WHERE` predicate or full-table delete |
| `BEGIN` / `COMMIT` / `ROLLBACK` | Real transactions, snapshot-based; WAL-backed commit; single-level (no savepoints); auto-rollback if `COMMIT`'s disk write fails |
| `PRAGMA auto_vacuum` | Read (`PRAGMA auto_vacuum;`) returns the trigger threshold as a single-row result set; set (`PRAGMA auto_vacuum = 0.5;` / `= OFF;` / `= NONE;`) tunes or disables auto-VACUUM at the SQL layer for SDK / FFI / MCP consumers |

Expressions in `WHERE` and `UPDATE`'s `SET` RHS:

Expand Down
1 change: 1 addition & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ The engine never depends on the SDK crates; the SDK crates each depend on the en
| [`src/sql/hnsw.rs`](../src/sql/hnsw.rs) | Standalone HNSW algorithm — insert / search / layer assignment / beam search. Phase 7d.1. |
| [`src/sql/fts/`](../src/sql/fts/) | Full-text search — standalone tokenizer, BM25 scorer, and in-memory `PostingList` inverted index. Wired into the executor via the `fts_match` / `bm25_score` scalar functions and the `try_fts_probe` optimizer hook. Phase 8a-8b; persistence in 8c. See [`docs/fts.md`](fts.md). |
| [`src/sql/json.rs`](../src/sql/json.rs) | JSON column type + path-extraction functions (`json_extract`, `json_type`, `json_array_length`, `json_object_keys`). Phase 7e. |
| [`src/sql/pragma.rs`](../src/sql/pragma.rs) | `PRAGMA` dispatcher (SQLR-13). `try_parse_pragma` peeks at the SQL token stream before sqlparser sees it and routes any `PRAGMA …` shape to `execute_pragma`. First pragma wired up: `auto_vacuum` (read + set, with `OFF` / `NONE` to disable). Add new pragmas as a single arm in `execute_pragma`. |
| [`src/sql/pager/`](../src/sql/pager/) | On-disk file format and I/O — see [file-format.md](file-format.md) and [pager.md](pager.md) for details. WAL + checkpointer + shared/exclusive lock modes (Phase 4a-4e) live here. |

## Flow of a SQL statement
Expand Down
2 changes: 1 addition & 1 deletion docs/pager.md
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ Format-version side effect: a save that produces a non-empty freelist promotes t

After SQLR-6, the file still required a manual `VACUUM;` to actually shrink — the freelist absorbed orphan pages but the high-water mark stayed put. SQLR-10 adds a heuristic that fires `vacuum_database` automatically after a page-releasing DDL (`DROP TABLE`, `DROP INDEX`, `ALTER TABLE DROP COLUMN`) when the freelist exceeds a configurable fraction of `page_count`.

Configuration lives on `Database::auto_vacuum_threshold: Option<f32>` and is exposed at the connection level via `Connection::set_auto_vacuum_threshold` / `auto_vacuum_threshold`. Defaults: `Some(0.25)` (SQLite parity at 25%); pass `None` to opt out per connection. The threshold is per-`Connection` runtime state and is not persisted in the file header — every reopen starts at the default. A SQL-level `PRAGMA auto_vacuum` is tracked separately (out of scope for SQLR-10).
Configuration lives on `Database::auto_vacuum_threshold: Option<f32>` and is exposed at the connection level via `Connection::set_auto_vacuum_threshold` / `auto_vacuum_threshold`, and via SQL through `PRAGMA auto_vacuum` (SQLR-13 — see [`src/sql/pragma.rs`](../src/sql/pragma.rs)). Defaults: `Some(0.25)` (SQLite parity at 25%); pass `None` (or `PRAGMA auto_vacuum = OFF`) to opt out per connection. The threshold is per-`Connection` runtime state and is not persisted in the file header — every reopen starts at the default.

The trigger lives at the end of [`process_command_with_render`](../src/sql/mod.rs), immediately after the auto-save. Order matters: the freelist isn't accurate until the bottom-up rebuild runs during save, so we save first, then check the ratio. The check itself is `freelist::should_auto_vacuum(pager, threshold)`, which:

Expand Down
13 changes: 11 additions & 2 deletions docs/supported-sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -551,7 +551,16 @@ conn.set_auto_vacuum_threshold(Some(0.5))?; // fire only when freelist > 50%
conn.set_auto_vacuum_threshold(None)?; // disable entirely (manual VACUUM only)
```

The setting is per-`Connection` runtime state — it's not persisted in the file header, so every reopen starts at the default `Some(0.25)`. A SQL-level `PRAGMA auto_vacuum` knob is on the roadmap but not yet implemented (SDK consumers currently configure it via the per-binding glue or fall back to the default).
…or via SQL (SQLR-13), which is the path SDK / FFI / MCP consumers reach for since they can't call the Rust setter directly:

```sql
PRAGMA auto_vacuum; -- read; renders a single-row result set
PRAGMA auto_vacuum = 0.5; -- arm the trigger at 50%
PRAGMA auto_vacuum = 0; -- arm at 0% (compact on any released page)
PRAGMA auto_vacuum = OFF; -- disable; equivalent: NONE, 'OFF', 'NONE'
```

Out-of-range values (anything outside `0.0..=1.0`, `NaN`, `±∞`) and unknown identifiers like `WAL` / `FULL` are rejected with a typed error — the trigger never silently saturates or falls back to a default. The setting is per-`Connection` runtime state — it's not persisted in the file header, so every reopen starts at the default `Some(0.25)`.

---

Expand Down Expand Up @@ -621,7 +630,7 @@ For context when you hit `NotImplemented`. See [Roadmap](roadmap.md) for when th

### Session / schema
- Multiple attached databases (`ATTACH DATABASE`, `DETACH DATABASE`)
- `PRAGMA` statements beyond what the parser accepts (none currently executed)
- `PRAGMA` statements other than `auto_vacuum` (SQLR-13). The dispatcher is in place — adding a pragma is a single arm in `execute_pragma`. `journal_mode`, `synchronous`, `cache_size`, etc. are not yet wired up
- `REPLACE INTO`, `INSERT OR IGNORE`, `INSERT OR REPLACE` (conflict-resolution clauses)

---
Expand Down
9 changes: 9 additions & 0 deletions src/sql/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ pub mod hnsw;
pub mod pager;
pub mod params;
pub mod parser;
pub mod pragma;
// pub mod tokenizer;

use parser::create::CreateQuery;
Expand Down Expand Up @@ -89,6 +90,14 @@ pub fn process_command(query: &str, db: &mut Database) -> Result<String> {
/// to stdout.** The REPL is responsible for printing whatever it wants
/// from the returned struct.
pub fn process_command_with_render(query: &str, db: &mut Database) -> Result<CommandOutput> {
// SQLR-13 — intercept `PRAGMA` before sqlparser sees it. sqlparser's
// pragma-value parser rejects bare `OFF` / `NONE` (and other classic
// SQLite pragma idioms), so we tokenize and dispatch ourselves. Non-
// PRAGMA input falls through to the regular dispatcher unchanged.
if let Some(stmt) = pragma::try_parse_pragma(query)? {
return pragma::execute_pragma(stmt, db);
}

let dialect = SqlriteDialect::new();
let mut ast = Parser::parse_sql(&dialect, query).map_err(SQLRiteError::from)?;

Expand Down
163 changes: 163 additions & 0 deletions src/sql/pager/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3889,6 +3889,169 @@ mod tests {
assert_eq!(db.auto_vacuum_threshold(), None);
}

// ---------------------------------------------------------------
// SQLR-13 — `PRAGMA auto_vacuum` SQL-level coverage. Mirrors the
// SQLR-10 setter tests above, but routed through SQL so SDK / FFI
// / MCP consumers (which can't reach the Rust setter directly)
// get the same guarantees.
// ---------------------------------------------------------------

/// `PRAGMA auto_vacuum = N;` set + `PRAGMA auto_vacuum;` read
/// round-trip the threshold, observable via `auto_vacuum_threshold`.
#[test]
fn pragma_auto_vacuum_set_and_read_via_sql() {
let mut db = Database::new("t".to_string());

let resp = process_command("PRAGMA auto_vacuum = 0.5;", &mut db).expect("set");
assert!(
resp.contains("PRAGMA"),
"set form should produce a PRAGMA status, got: {resp}"
);
assert_eq!(db.auto_vacuum_threshold(), Some(0.5));

// Read form — status mentions a returned row.
let resp = process_command("PRAGMA auto_vacuum;", &mut db).expect("read");
assert!(resp.contains("1 row"), "expected a 1-row read, got: {resp}");
}

/// `PRAGMA auto_vacuum = OFF;` (bare identifier — sqlparser's own
/// pragma-value parser would reject this, the SQLR-13 dispatcher
/// must accept it) and `= NONE;` both disable the trigger. So does
/// the quoted form `'OFF'`.
#[test]
fn pragma_auto_vacuum_off_disables_trigger() {
for raw in ["OFF", "off", "NONE", "none", "'OFF'", "'NONE'"] {
let mut db = Database::new("t".to_string());
assert_eq!(db.auto_vacuum_threshold(), Some(0.25));

let stmt = format!("PRAGMA auto_vacuum = {raw};");
process_command(&stmt, &mut db)
.unwrap_or_else(|e| panic!("`{stmt}` should disable: {e}"));
assert_eq!(
db.auto_vacuum_threshold(),
None,
"`{stmt}` should clear the threshold"
);
}
}

/// Out-of-range numeric values surface as a typed error via the
/// shared `set_auto_vacuum_threshold` validator — no silent
/// saturation. Mirrors the SQLR-10 setter coverage.
#[test]
fn pragma_auto_vacuum_rejects_out_of_range_via_sql() {
let mut db = Database::new("t".to_string());
for bad in ["-0.01", "1.01", "1.5"] {
let stmt = format!("PRAGMA auto_vacuum = {bad};");
let err = process_command(&stmt, &mut db).unwrap_err();
assert!(
format!("{err}").contains("auto_vacuum_threshold"),
"expected range error for `{stmt}`, got: {err}"
);
}
// Default survives all the rejected sets.
assert_eq!(db.auto_vacuum_threshold(), Some(0.25));
}

/// Junk strings (anything that isn't a number or `OFF`/`NONE`) are
/// rejected at parse time with a typed error, not silently treated
/// as "disable".
#[test]
fn pragma_auto_vacuum_rejects_unknown_strings_via_sql() {
let mut db = Database::new("t".to_string());
let err = process_command("PRAGMA auto_vacuum = WAL;", &mut db).unwrap_err();
assert!(
format!("{err}").contains("OFF/NONE"),
"expected OFF/NONE-style error, got: {err}"
);
// Default unaffected.
assert_eq!(db.auto_vacuum_threshold(), Some(0.25));
}

/// Pragmas SQLRite doesn't know about return `NotImplemented` —
/// not a generic parser error. Future pragmas plug in here.
#[test]
fn pragma_unknown_returns_not_implemented() {
let mut db = Database::new("t".to_string());
let err = process_command("PRAGMA journal_mode = WAL;", &mut db).unwrap_err();
assert!(
matches!(err, SQLRiteError::NotImplemented(_)),
"unknown pragma must surface NotImplemented, got: {err:?}"
);
}

/// Setting the threshold via SQL must produce identical behavior to
/// the Rust setter on the actual auto-VACUUM trigger: `= 0.99`
/// suppresses, `= OFF` disables, default fires. Sanity-checks that
/// `process_command_with_render`'s pre-parse step doesn't desync
/// the in-memory state from the file.
#[test]
fn pragma_auto_vacuum_drives_real_trigger() {
// Sub-case A — `PRAGMA auto_vacuum = OFF;` keeps file at HWM.
{
let path = tmp_path("av_pragma_off");
let mut db = auto_vacuum_setup(&path);
process_command("PRAGMA auto_vacuum = OFF;", &mut db).expect("disable via PRAGMA");
assert_eq!(db.auto_vacuum_threshold(), None);

let pages_before = db.pager.as_ref().unwrap().header().page_count;
process_command("DROP TABLE bloat;", &mut db).expect("drop");
let pages_after = db.pager.as_ref().unwrap().header().page_count;
assert_eq!(
pages_after, pages_before,
"PRAGMA-driven OFF must keep page_count at the HWM"
);
cleanup(&path);
}

// Sub-case B — high threshold via PRAGMA suppresses the
// trigger on a single drop.
{
let path = tmp_path("av_pragma_high");
let mut db = auto_vacuum_setup(&path);
process_command("PRAGMA auto_vacuum = 0.99;", &mut db).expect("set high");
assert_eq!(db.auto_vacuum_threshold(), Some(0.99));

let pages_before = db.pager.as_ref().unwrap().header().page_count;
process_command("DROP TABLE bloat;", &mut db).expect("drop");
let pages_after = db.pager.as_ref().unwrap().header().page_count;
assert_eq!(
pages_after, pages_before,
"high PRAGMA threshold must suppress the trigger"
);
cleanup(&path);
}

// Sub-case C — re-arm via PRAGMA after disable: the trigger
// fires again on the next page-releasing DDL.
{
let path = tmp_path("av_pragma_rearm");
let mut db = auto_vacuum_setup(&path);
process_command("PRAGMA auto_vacuum = OFF;", &mut db).unwrap();
// Drop with the trigger off — pages land on the freelist
// but the file stays at HWM.
process_command("DROP TABLE bloat;", &mut db).unwrap();
let pages_after_off_drop = db.pager.as_ref().unwrap().header().page_count;
assert!(db.pager.as_ref().unwrap().header().freelist_head != 0);

// Re-arm via PRAGMA, then drop one more thing — the
// accumulated freelist still exceeds 25%, so auto-VACUUM
// fires.
process_command("PRAGMA auto_vacuum = 0.25;", &mut db).expect("re-arm");
process_command("CREATE INDEX idx_keep_n ON keep (n);", &mut db).unwrap();
process_command("DROP INDEX idx_keep_n;", &mut db).expect("drop index");

let pages_after_rearm = db.pager.as_ref().unwrap().header().page_count;
assert!(
pages_after_rearm < pages_after_off_drop,
"re-armed PRAGMA must let auto-VACUUM fire: was {pages_after_off_drop}, \
now {pages_after_rearm}"
);
assert_eq!(db.pager.as_ref().unwrap().header().freelist_head, 0);
cleanup(&path);
}
}

/// VACUUM modifiers (FULL, REINDEX, table targets, …) are rejected
/// with NotImplemented — only bare `VACUUM;` is supported.
#[test]
Expand Down
Loading
Loading