Commit 718ba31
Release engineering + full SDK surface (Phases 2.5 → 6d) (#15)
* [Cargo{.toml,.lock},rust-toolchain.toml] Bump to edition 2024, resolver 3; upgrade all deps to current majors (rustyline 18, clap 4, sqlparser 0.61, thiserror 2, env_logger 0.11, prettytable 0.10); [src/**.rs] port to new dependency APIs: sqlparser struct-variant Statements, ColumnOption::PrimaryKey split, ValueWithSpan, DataType::Integer variant; rustyline Editor<H,I>, CmdKind highlight_char, removed OutputStreamType; clap 4 Command; eliminate dead code warnings
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/*] Phase 1: implement SELECT, UPDATE, DELETE execution; eliminate insert_row panics
[src/sql/parser/select.rs] new — parse sqlparser Query AST into a minimal SelectQuery (projection, WHERE, single-column ORDER BY, integer LIMIT); explicit NotImplemented errors for joins/group-by/having/distinct/offset
[src/sql/executor.rs] new — expression evaluator over sqlparser::Expr (identifiers, literals, nested, unary +/-/NOT, binary =/<>/</<=/>/>=/AND/OR) with NULL-as-false WHERE semantics; execute_select renders results via prettytable; execute_delete + execute_update share the evaluator and row-matching logic
[src/sql/db/table.rs] add runtime Value enum (Integer/Text/Real/Bool/Null) + Row::rowids/get + Table::rowids/get_value/column_names/delete_row/set_value; set_value enforces declared datatype + UNIQUE at write time and refreshes the column index; replace every .unwrap() in insert_row and validate_unique_constraint with typed error returns so malformed INSERTs no longer crash the REPL
[src/sql/mod.rs] wire Statement::Query / Statement::Update / Statement::Delete into the executor; replace stubbed tests with real end-to-end coverage (34 tests total: SELECT *, projection, WHERE eq/string/numeric, LIMIT, unknown table/column errors, DELETE with/without WHERE, UPDATE cell mutation, UNIQUE violation on UPDATE, bad INSERT no longer panics)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/executor.rs] add arithmetic + string-concat to the expression evaluator
- Binary +, -, *, /, % on Integer/Real with Integer×Integer staying Integer and any Real promoting to f64
- || (StringConcat) with NULL propagation
- Divide/modulo by zero returns a clean error (no panic); wrapping arithmetic on integer overflow
- Enables UPDATE t SET col = col + 1 and WHERE age * 2 > 55 style expressions
[src/sql/mod.rs] tests for UPDATE SET age = age + 1, SELECT ... WHERE age * 2 > 55, and a division-by-zero guard (37 tests total)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [README.md] rewrite Usage and Roadmap to reflect current state
- Usage: replaces the TBD `--help` stub with a real REPL session (CREATE/INSERT/SELECT/UPDATE/DELETE), the full supported-SQL matrix (statements, clauses, expression operators), and the meta-command status table
- Roadmap: replaces the ad-hoc progress + future lists with the agreed 6-phase plan — Phase 0 (modernization, done), Phase 1 (SQL execution, done), Phase 2 (single-file persistence, 4 KiB pages, explicit .open/.save/.tables — in progress), Phase 2.5 (Tauri desktop app), Phase 3 (on-disk B-Tree + auto-save pager), Phase 4 (WAL + file locks + multi-reader/single-writer), Phase 5 (lib split + C FFI + WASM), Phase 6 (vector columns, NL→SQL, agent-era)
- Moves joins / GROUP BY / aggregates / alternate storage engines / benchmarks under a "possible extras" section that isn't pinned to a phase
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/*] Phase 2: single-file persistence via 4 KiB paged layout
[Cargo.{toml,lock}] add bincode 2.0 (+serde feature) for schema/table serialization
[src/error.rs] add SQLRiteError::Io(#[from] std::io::Error) for clean ? propagation from pager I/O; drop derived PartialEq (io::Error isn't PartialEq) and hand-roll an impl so existing tests that compare error variants keep working
[src/sql/pager/page.rs] 4 KiB page format with a 7-byte header (type tag, next-page u32, payload-len u16); PageType enum (SchemaRoot / TableData / Overflow); encode_page / decode_page primitives. Payload-per-page = 4089 bytes.
[src/sql/pager/header.rs] DbHeader on page 0 (magic "SQLRiteFormat\0\0\0", format version u16, page size u16, total page count u32, schema-root page u32). decode_header rejects bad magic, unsupported version, or page-size mismatch with typed errors.
[src/sql/pager/file.rs] FileStorage: thin wrapper around std::fs::File exposing read_page / write_page / read_header / write_header (page-indexed seeks) + flush (fsync).
[src/sql/pager/mod.rs] save_database(db, path) — bincode-encode each Table to a chain of pages, then encode the Vec<(name, start_page)> schema catalog to its own chain, then write page 0 last so a crash mid-save leaves the file recognizably unopenable. open_database(path) — verify header, follow schema catalog chain, then each per-table chain. Four unit tests cover: basic round-trip, mutate-after-load round-trip, reject garbage file, multi-page overflow chain (200-row table).
[src/meta_command/mod.rs] MetaCommand gains Save(PathBuf), Tables; Open changes from String to PathBuf. handle_meta_command now takes &mut Database and dispatches to pager::{open,save}_database. `.open` on a missing file creates an empty in-memory DB (matching SQLite behavior). `.tables` lists table names sorted deterministically. Added meta-command tests for argument parsing, the tables listing, and a save→open round-trip.
[src/main.rs] thread &mut db through to handle_meta_command
[src/sql/mod.rs] declare pub mod pager
43 tests total (up from 37). Two-process REPL smoke test confirmed: create→insert→save, restart, open→select→update→save survives across invocations. Resulting .sqlrite file is a clean 3 × 4096 = 12 KiB for a tiny schema.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [README.md] flip Phase 2 to done; promote Phase 3 ahead of 2.5
- Meta-command status table: .open/.save/.tables all marked working with short descriptions
- Roadmap: Phase 2 bullets flipped to [x] and expanded to record the actual page format shipped (4 KiB pages, 7-byte page header, typed page tags, bincode 2.0 serialization, header-written-last crash detection)
- Reorder: Phase 3 (on-disk B-Tree + auto-save pager) is now marked "in progress, next" and Phase 2.5 (Tauri desktop app) comes after it — the GUI will demo real-time saves once Phase 3's pager lands rather than explicit-save-only
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/{sql,meta_command}/**] Phase 3a: auto-save every committing statement
[src/sql/db/database.rs] add `source_path: Option<PathBuf>` to Database (skipped by serde so the on-disk format is unchanged). Hand-rolled PartialEq compares state only, ignoring the path — two DBs loaded from different files can still be equal.
[src/sql/mod.rs] process_command now classifies the statement as read-only or writing; after a writing statement (CreateTable/Insert/Update/Delete) succeeds, if db.source_path is Some, flush via pager::save_database. SELECTs skip the save to avoid pointless I/O. A save error propagates out — the in-memory state already mutated, so the caller needs to know disk is out of sync.
[src/meta_command/mod.rs] `.open FILE` now sets `db.source_path` and, when the file doesn't yet exist, eagerly materializes an empty DB on disk so the path is valid for subsequent auto-saves (also catches permission errors up front). `.save FILE` still works but prints a different message when the target matches the active source_path ("Flushed... auto-save is already on.") so the user isn't misled about what their explicit save actually did.
[README.md] meta-command table: `.open` now notes auto-save activation; `.save` described as "rarely needed once .open is in play". Phase 3 broken into sub-phases 3a–3e with 3a marked done.
Tests: 44 total (up from 43). New coverage — auto-save persists writes across process restarts with zero `.save` calls; `.open` of missing file materializes the file + enables auto-save; three-process REPL smoke test confirms data round-trips through INSERT/UPDATE with nothing but `.open` + `.exit`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/*] Phase 3b: long-lived Pager with diffing commits
[src/sql/pager/pager.rs] new — Pager owns the DB file, a HashMap<u32, Box<[u8; PAGE_SIZE]>> snapshot of every page currently on disk, and a staging map for the next commit. `commit` diffs staged vs snapshot and writes only pages whose bytes actually changed, then refreshes the snapshot. `open` pre-loads every page; `create` initializes an empty-schema-catalog file. Shrinking the page count truncates the file. Three unit tests: create-then-open round-trip; commit-writes-only-dirty-pages (3 writes then re-stage same bytes + 1 change → 1 write); file shrinks when page count drops from 5 to 3.
[src/sql/pager/file.rs] drop the per-page encode/decode helpers (moved to pager/mod.rs); add low-level seek_to / read_exact / write_all / truncate_to_pages that the Pager drives directly.
[src/sql/pager/page.rs] drop encode_page / decode_page — inlined into pager/mod.rs where raw buffers are managed.
[src/sql/pager/mod.rs] rewrite save_database / open_database around the Pager. save iterates tables in sorted order so page numbers are stable across saves (required for the diff to skip unchanged tables). When save_database's target matches db.source_path, it takes the long-lived Pager off the Database, stages all pages, commits (only diffs written), and re-attaches the Pager — so subsequent auto-saves benefit from the accumulated snapshot.
[src/sql/db/database.rs] Database gains pager: Option<Pager> (serde-skipped, like source_path). open_database attaches a Pager; save_database reuses it for diff-based commits.
[src/sql/mod.rs] auto-save path passes &mut db so the Pager can be borrowed mutably through save_database.
[src/meta_command/mod.rs] handle_open sets source_path on a fresh db before save, so the first .open on a missing file takes the same-path code path and attaches a Pager correctly. handle_save now takes &mut db.
[README.md] Phase 3 roadmap: mark 3b done with the pager description.
47 tests (up from 44). Pager unit tests verify diff-only writes and truncate-on-shrink. End-to-end REPL smoke test confirms two-table DB survives restarts with only the mutated table's pages rewritten on auto-save.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [docs/**,README.md] add a developer guide under docs/
Ten markdown files covering the project end-to-end:
- _index.md — navigation hub listing every doc and a short "project state" note
- getting-started.md — toolchain, build, first REPL session, first persistent session, repo layout
- usage.md — complete meta-command table, every supported SQL statement with semantics, full expression operator table, explicit "not yet supported" list
- architecture.md — layered ASCII diagram from REPL down to disk, module map, end-to-end flow of an UPDATE statement, concerns-vs-location table
- design-decisions.md — 11 "why" records: sqlparser choice, 4 KiB page size, single-file DB, header-written-last, bincode for now, long-lived Pager, deterministic page ordering, runtime Value vs storage Row, NULL-as-false, no transactions yet, Phase 3 sub-phase granularity
- file-format.md — byte-level layout: page 0 header fields, page header fields, page types, chaining mechanics, an example file layout, invariants
- pager.md — Pager struct, open/create lifecycle, staging + diffing commit algorithm, why the diff earns its keep, what it doesn't do yet, Database integration
- storage-model.md — in-memory layout of Database / Table / Column / Row / Index, column-oriented rationale, ROWID handling, index lifecycle, NULL handling, write paths (INSERT / UPDATE / DELETE / CREATE), read path (SELECT)
- sql-engine.md — process_command flow, sqlparser → internal struct trimming, executor dispatch, expression evaluator operators + type promotion + NULL semantics, two-pass UPDATE/DELETE pattern, auto-save hook, error types
- roadmap.md — every phase from 0 (done) to 6 (research), with commit hashes for the shipped ones and open design questions for 3c
README.md: new "Developer guide" section pointing at docs/_index.md and listing the major files by topic, dropped the stale "SQLite3 required" line from the Requirements list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/{varint,cell}.rs] Phase 3c.1: varint + cell codec
Data-layer building blocks for the cell-based page format. Nothing is wired
into save/open yet — that's Phase 3c.4. These modules are standalone and
tested in isolation.
[src/sql/pager/varint.rs] LEB128 + ZigZag helpers:
- write_u64 / read_u64 — unsigned LEB128 (1–10 bytes)
- write_i64 / read_i64 — ZigZag-encoded i64 on top of LEB128
- u64_len / i64_len — size predictors (no allocation)
- Bounds-checked reads: truncated buffers and >10-byte overlong encodings
error cleanly instead of overflowing
[src/sql/pager/cell.rs] Cell format — one row per cell:
- cell_length (varint, does not count itself)
- rowid (zigzag varint)
- col_count (varint)
- null bitmap (⌈col_count/8⌉ bytes, bit 0 of byte 0 = column 0)
- value blocks in declared column order, only for non-NULL columns:
tag=0 Integer (i64 zigzag varint)
tag=1 Real (f64 little-endian, 8 bytes)
tag=2 Text (varint length, UTF-8 bytes)
tag=3 Bool (u8)
- Cell::encode / Cell::decode round-trip, decode consumes exactly the bytes
encode emits (asserted in every test)
- Null values travel through the bitmap; passing Value::Null as a value is
rejected at encode time rather than silently double-representing
Design rationale for the redundancy between bitmap and value tags:
- Bitmap is faster to scan for column projection (skip absent columns without
decoding lengths)
- Bitmap is more compact when many columns are null (1 bit vs 1+ byte)
- Tags stay because they're needed for type dispatch during decode
21 new tests (68 total, was 47). Covers: varint boundaries at 1/2/3/10 bytes,
zigzag sign round-trip, overlong and truncated varint rejection; cell
round-trip for integer-only / text / real / bool / mixed / nulls-interspersed
/ all-null / large-text / UTF-8 / negative-rowid / i64 boundaries / real
edges (inf, neg-inf, min, max); sequential decoding of concatenated cells;
null-bitmap byte-boundary cases (8 cols = 1 byte, 9 cols = 2 bytes); decoder
rejection of truncated buffers and unknown tags.
[src/sql/pager/mod.rs] register both modules with a Phase-3c.4-scoped
#[allow(dead_code)] until the higher layers start calling them.
[.gitignore] add *.sqlrite so stray demo files from smoke tests don't get
committed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/table_page.rs] Phase 3c.2: TablePage with slot directory
Table-leaf page layout for cell-based storage. Still standalone — the
save/open path doesn't use these yet. That's Phase 3c.4.
Layout inside the 4089-byte payload area:
0..2 slot_count (u16 LE)
2..4 cells_top (u16 LE) — offset where cell content starts
4.. slot[0]..slot[n-1] — u16 LE each, rowid-ordered
[free space in the middle]
cells_top..4089 cell bodies (unordered in physical position)
Slots grow up from offset 4; cells grow down from offset 4089. Free space
is everything between slots_end and cells_top. Empty page has slot_count=0,
cells_top=4089, free=4085.
API on TablePage:
- empty() / from_bytes(&[u8; 4089]) / as_bytes() — round-trips through bytes
- slot_count(), cells_top(), free_space(), would_fit(size) — capacity queries
- rowid_at(slot) — fast rowid read without decoding the full cell body;
uses Cell::peek_rowid() (added to cell.rs) which skips past the cell
length prefix and reads just the rowid varint
- cell_at(slot) — full Cell::decode at the slot's offset
- find(rowid) — binary search returning Find::{Found(slot) | NotFound(insertion_slot)}
- lookup(rowid) — convenience wrapper over find + cell_at
- iter() — (slot, Cell) stream in rowid order
- insert(&cell) — rowid-ordered insert, rejects duplicate rowids, returns
a "page full" error when free_space < cell_size + 2
- delete(rowid) — drops the slot, leaves the cell body in place (O(slots)
vs O(page) compaction). free_space therefore underreports after deletes;
documented as "contiguous free bytes", which is what cell writes need.
Design choices worth capturing:
- Slot directory is at the start of the payload (right after the two
header u16s), cells grow down from the end. Matches SQLite's layout.
Original suggestion said "slot directory at page end" — that was wrong.
This is the correct SQLite-style layout; the outcome (slots + cells
growing toward each other, free space in the middle) is the same.
- Duplicate rowid on insert is rejected rather than silently replacing;
higher layers (executor UPDATE path) will explicitly delete-then-insert
when that's what they want.
- Deletion leaves holes. Compacting on every delete would be O(page);
the hole will be reclaimed by a future vacuum pass. For now,
free_space reports contiguous-free only.
[src/sql/pager/cell.rs] add two small helpers that TablePage needs to
avoid full decode on the hot path:
- Cell::peek_rowid(buf, pos) — skip the cell_length varint, read only
the rowid varint. Used by find() for binary search.
- Cell::encoded_size_at(buf, pos) — length of the cell starting at pos
without decoding its body. Not used yet but useful for a future vacuum.
11 new tests (79 total, was 68). Cover: empty-page invariants, insert in
reverse rowid order with lookups still working, iter yields rowid-
ascending, duplicate-rowid rejection, delete-then-reinsert, delete of
missing rowid, round-trip through as_bytes/from_bytes, find at
before/between/after/exact positions, would_fit gates insert and the
overflow cell returns "page full", free_space accounting across insert
and delete (holes are intentional), mixed-types + nulls per cell on one
page, peek_rowid agreement with cell_at.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/overflow.rs,cell.rs] Phase 3c.3: overflow chains + kind-tagged cells
Cells that don't fit on one leaf page now spill into a chain of
Overflow-typed pages. Slot directory still holds a fixed-size marker
(OverflowRef) on the main page so rowid ordering is preserved.
[src/sql/pager/cell.rs] prepend a kind_tag byte to every cell's body:
cell_length varint total bytes after this field, including kind_tag
kind_tag u8 0x01 = local cell (this type), 0x02 = overflow
rowid zigzag varint
col_count varint
null_bitmap ⌈col_count/8⌉ bytes
value_blocks ...
Cell::encode emits KIND_LOCAL; Cell::decode errors if it sees any other
kind_tag (callers that can't be sure should go through PagedEntry::decode).
Also added Cell::peek_kind for dispatch; updated Cell::peek_rowid to skip
the new tag byte.
[src/sql/pager/overflow.rs] new module:
- OverflowRef { rowid, total_body_len, first_overflow_page } — on-page
marker with its own encode/decode (emits the same cell_length + kind_tag
prefix, then rowid varint, total_body_len varint, first_overflow_page u32)
- PagedEntry enum { Local(Cell), Overflow(OverflowRef) } with a
kind-dispatching decode() that table_page will use in 3c.4 when iter()
and lookup() need to return either kind
- write_overflow_chain(pager, bytes, start_page) — packs bytes into a
chain of PageType::Overflow pages using consecutive page numbers,
reusing the existing 7-byte per-page header (type + next + payload_len)
- read_overflow_chain(pager, first_page, total_len) — walks the chain,
strictly validates that the chain delivers exactly total_len bytes
(length mismatch = corruption error)
- OVERFLOW_THRESHOLD constant = PAYLOAD_PER_PAGE / 4 ≈ 1022 bytes;
caller's choice when to overflow, this is a suggestion
Scope decision: **spill the entire body, inline only the rowid.** SQLite
inlines a prefix before spilling so small rowid lookups don't chase the
chain. For our rowid-keyed leaves, most reads need the body anyway (to
evaluate WHERE/project columns), so a prefix-inline optimization mostly
benefits "did this rowid exist?" queries. Deferring to a later pass; the
page format doesn't need to change to add it.
[src/sql/pager/mod.rs] register the overflow module with the same
dead-code allow as the other 3c.x modules.
8 new tests (87 total, was 79): OverflowRef round-trip, PagedEntry kind
dispatch for both variants, peek_rowid works for both kinds, chain write
+ read through a fresh Pager round-trips a 10 KB blob, read rejects a
length mismatch between what the chain carries and what the ref claims,
empty chain is rejected, OVERFLOW_THRESHOLD sanity bounds. Also updated
the cell decoder's kind-tag rejection test and added a "Cell::decode
refuses overflow kind" case.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/mod.rs + {table,table_page}] Phase 3c.4: cell-based table storage
Tables now persist as chains of TableLeaf pages full of cells rather than
as opaque bincode blobs. Large rows spill into the overflow chains added
in 3c.3. The schema catalog is still a bincode blob — that's Phase 3c.5.
[src/sql/pager/page.rs] rename PageType::TableData → TableLeaf. On-disk
tag value is unchanged (2); this is a source rename reflecting the new
payload semantics (slot directory + cells, not a bincode blob).
[src/sql/pager/mod.rs] full rewrite of save_database / open_database
around the cell layer:
- New internal types TableSchema (tb_name, columns, primary_key,
last_rowid) and CatalogEntry (name, schema, root_page)
- save_database: per-table flow stages cells across one or more
TableLeaf pages. For each rowid: extract the ordered values, build a
Cell. If its encoded length > OVERFLOW_THRESHOLD, spill the full cell
bytes to a write_overflow_chain and emit an OverflowRef on the leaf
instead. When a leaf fills, commit it with next_page pointing at the
following leaf and start fresh. Empty tables still get one empty
leaf so the load loop doesn't have to special-case zero.
- open_database: walk the catalog's bincode chain, then for each entry
build an empty Table from the schema and call load_table_rows which
walks the leaf chain, decodes each PagedEntry (Local or Overflow),
and calls Table::restore_row for each cell. Overflow entries first
reassemble the body via read_overflow_chain.
- strip_runtime_index rebuilds each Column with an empty Index before
serializing — indexes are runtime structures rebuilt by restore_row.
[src/sql/db/table.rs] two new methods on Table:
- restore_row(rowid, values) — replay one row during load. Takes one
typed Value per column (in declaration order), writes into the right
per-column Row BTreeMap, and populates the column's Index if unique.
Skips UNIQUE re-validation (on-disk state is trusted to be consistent).
- extract_row(rowid) — dump a row as Vec<Option<Value>> for Cell::new.
NULL columns (Value::Null) map to None so the null bitmap captures
them.
[src/sql/pager/table_page.rs] two new insert paths:
- insert_entry(rowid, encoded_bytes) — the primitive. Binary-searches
the slot directory, verifies fit, writes the bytes at cells_top-len,
maintains the slot array in rowid order.
- insert_paged_entry(&PagedEntry) — wrapper for either Cell or
OverflowRef. The existing insert(&cell) now delegates to insert_entry.
- entry_at(slot) — decode the paged entry at a slot, dispatching on
kind tag (via PagedEntry::decode).
88 tests pass. Covers the old round-trip / mutate-after-load / garbage-
file / 200-row-multi-leaf scenarios, plus a new huge_row_goes_through_
overflow test that round-trips a 10 KB text through write_overflow_chain
and back. End-to-end 3-process REPL smoke test confirmed: process 1
creates schema + inserts, process 2 reopens and updates one row, process
3 reopens and sees the update — all through the new cell layer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/mod.rs,header.rs,pager.rs] Phase 3c.5: sqlrite_master catalog
Promote the schema catalog from a bincode blob to a real cell-stored
internal table named `sqlrite_master`, following SQLite's approach.
File format version bumps from 1 to 2 — v1 files are rejected on open.
[src/sql/pager/header.rs] FORMAT_VERSION 1 → 2, with a short history
comment documenting the two layouts.
[src/sql/pager/pager.rs] Pager::create writes an empty TableLeaf at
page 1 (the initial sqlrite_master root) instead of an empty bincode
catalog. Fresh DBs now carry a usable sqlrite_master from the first
fsync.
[src/sql/pager/mod.rs] full rewrite of the save/open pair:
- New MASTER_TABLE_NAME = "sqlrite_master"
- build_empty_master_table() creates the hardcoded schema:
(name TEXT PRIMARY KEY, sql TEXT NOT NULL,
rootpage INTEGER NOT NULL, last_rowid INTEGER NOT NULL)
- table_to_create_sql(table) synthesizes a CREATE TABLE string from
the in-memory Column metadata. Deterministic — same schema, same
output — so the bytes on disk stay stable across saves when nothing
changes (required by the Pager's diffing commit to skip writes).
- parse_create_sql(sql) re-parses via sqlparser + CreateQuery::new to
recover the column list on open.
- save_database: stages each user table's leaf chain (sorted by name),
collects (name, sql, rootpage, last_rowid) tuples, populates an
in-memory sqlrite_master via restore_row, then stages the catalog
table via the same stage_table_rows flow. header.schema_root_page
points at the first leaf of sqlrite_master.
- open_database: loads sqlrite_master from the leaf chain at the
header's schema_root_page, then for each master row parses the SQL
to rebuild the Column list and walks the user table's rootpage.
- The old Catalog/TableSchema bincode types are gone.
[src/sql/pager/page.rs] retire PageType::SchemaRoot (tag value 1 stays
reserved, not re-used). PageType::from_u8 kept with #[allow(dead_code)]
— it'll return to active duty for Phase 3d's B-Tree interior pages.
[src/sql/mod.rs] reject CREATE TABLE sqlrite_master with a clean
"reserved name" error rather than letting a collision propagate into
save_database.
90 tests pass (+2: create_sql_synthesis_round_trips verifies the
synthesize-then-parse round-trip reproduces the column metadata;
sqlrite_master_is_not_exposed_as_a_user_table confirms .tables and
db.tables don't include the internal catalog).
End-to-end REPL smoke test: CREATE two tables + INSERT, restart,
.tables shows just the user tables, SELECT works, attempting to
CREATE TABLE sqlrite_master fails cleanly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [README.md,docs/{file-format,storage-model,roadmap}.md] Phase 3c docs update
- README roadmap: mark 3c done with a one-line summary of the shipped layout
- docs/file-format.md: full rewrite for format v2. New: TableLeaf payload
layout (slot directory + cells), cell format (kind_tag + length prefix
+ null bitmap + typed value blocks), overflow-cell on-page marker,
overflow page payload layout, sqlrite_master schema + bootstrap rules,
updated layout example with two user tables + a catalog leaf
- docs/storage-model.md: replace the speculative "what Phase 3c will
change" section with factual "what Phase 3c changed on disk (cells,
overflow, sqlrite_master) but kept in memory (still the
column-oriented HashMap<column, BTreeMap<rowid, value>>)" note, and
point the forward-looking section at 3d's B-Tree + lazy loading
- docs/roadmap.md: expand the 3c entry with the five sub-phase commit
hashes and a one-line summary of each; rewrite 3d / 3e to reflect
that 3c's data layer is in place and 3d just adds interior routing
Code state unchanged — docs only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/interior_page.rs,mod.rs,page.rs,cell.rs] Phase 3d: on-disk B-Tree
Tables are now stored as B-Trees with interior routing pages above the
existing TableLeaf chain from Phase 3c. Save builds the tree bottom-up
from the in-memory sorted rows; open descends to the leftmost leaf and
scans forward via sibling next_page pointers (the existing 3c chain,
now serving as the B+Tree sibling link).
Design decisions agreed on before implementation:
- Overfill-then-split semantics via bulk rebuild: writes stay
"save → rebuild the full tree", not in-place incremental splits.
Simpler, fully deterministic, works for any table size.
- No merge pass; vacuum is later work.
- Interior cells share the `cell_length | kind_tag | body` prefix with
local and overflow cells so Cell::peek_rowid works uniformly across
all three kinds.
- Eager-load on open (materialize the full Table from the tree). A
Cursor abstraction for streaming reads is deferred to Phase 5 — see
the note in load_table_rows.
[src/sql/pager/page.rs] new PageType::InteriorNode = 4.
[src/sql/pager/cell.rs] new KIND_INTERIOR = 0x03; documentation
updated to describe all three cell kinds.
[src/sql/pager/interior_page.rs] new module:
- InteriorCell { divider_rowid, child_page }: "rowids up to
divider_rowid live under child_page". Encodes with the shared
[cell_length varint | kind_tag | rowid zigzag varint | child u32 LE]
format; decode rejects non-interior kind tags.
- InteriorPage: 4089-byte payload with layout
[slot_count u16 | cells_top u16 | rightmost_child u32 | slots | cells]
Parallel to TablePage except for the extra 4-byte rightmost-child
pointer (catch-all for rowids larger than any divider). Supports
insert_divider (binary-search rowid ordering, duplicate rejection),
child_for (rowid → child page routing), leftmost_child (for tree
descent during open).
- 10 standalone tests: cell round-trip, peek_rowid works on interior
format, wrong-kind-tag rejection, empty page routes everything to
rightmost, dividers route correctly, out-of-order inserts still land
in the right slots, duplicate-divider rejection, bytes round-trip,
leftmost-child picks slot 0 or rightmost appropriately, many
dividers fit on one page (confirms fanout > 100).
[src/sql/pager/mod.rs] rewrite save/open around tree build + descent:
- stage_table_btree replaces stage_table_rows. Returns (root_page,
next_free_page). Step 1 packs rows into TableLeaf pages with
sibling next_page chaining (stage_leaves, tracking each leaf's max
rowid). Step 2: if just one leaf, root = leaf. Otherwise
stage_interior_level packs leaves into interiors; recurse up the
tree until one root remains.
- stage_interior_level: greedy pack, last child assigned becomes
rightmost. Each previous rightmost "graduates" to a divider when
the next child arrives.
- load_table_rows now calls find_leftmost_leaf first, which descends
via InteriorPage::leftmost_child until it hits a TableLeaf. Then
the existing sibling-chain iteration runs.
- build_row_entry extracted — encapsulates the local-or-overflow
decision for a single row.
- Both user-table save and sqlrite_master save call stage_table_btree;
catalog tables get the same B-Tree treatment.
No file format bump: v2 files with single-leaf tables are still valid
(their roots are leaves, which find_leftmost_leaf returns unchanged).
Writing on top of a v2 file now emits interior pages for any multi-leaf
table.
Tests: 102 total (was 90). +10 in interior_page module, +2 integration
(multi_leaf_table_produces_an_interior_root confirms the root is an
InteriorNode after 200 fat rows; deep_tree_round_trips forces a
3-level tree at 6000 rows using restore_row to bypass the REPL's
O(N) print-on-INSERT). REPL smoke test: 50 INSERTs survive a process
restart through the new tree code.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [README.md,docs/{file-format,storage-model,roadmap}.md] Phase 3d docs update
- README roadmap: mark 3d done with a summary of what shipped (bottom-up
tree build on save, leftmost-descent + sibling scan on open, shared
cell prefix across all three kinds, cursor work deferred)
- docs/file-format.md: add InteriorNode to the page type table; split
the chaining section into leaves-sibling-chain vs interior-routing;
new InteriorNode payload layout section (rightmost-child pointer in
the payload header); new Interior cell body format; update the
layout-example section to show a multi-leaf table with its interior
root; update format-evolution footer to note 3d added interior pages
without bumping the v2 version
- docs/storage-model.md: replace the speculative 3d section with a
factual description of what Phase 3d actually did (bottom-up bulk
rebuild on save, eager-load on open, cursor refactor deferred to
Phase 5); new "what 3e will change" pointer
- docs/roadmap.md: mark 3d done with the commit hash; Phase 5 entry
now explicitly includes "Cursor abstraction (deferred from Phase 3d)"
so the hand-off is tracked
Code state unchanged — docs only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/db/*] Phase 3e.1: replace Column.index with Table.secondary_indexes
Extract the per-column BTreeMap index into a dedicated module so Phase
3e.2 can add user-declared CREATE INDEX alongside the auto-created UNIQUE
ones. No user-visible behavior change — this is pure refactor.
[src/sql/db/secondary_index.rs] new module:
- SecondaryIndex { name, table_name, column_name, is_unique, origin, entries }
- IndexEntries: BTreeMap<i64, Vec<rowid>> (Integer) or BTreeMap<String, Vec<rowid>> (Text).
The Vec<rowid> shape lets a single type cover both unique (always len 1)
and future non-unique indexes (len > 1). NULL values are never inserted
and would_violate_unique always reports false for them.
- IndexOrigin enum — Auto (for UNIQUE/PK columns) vs Explicit (CREATE INDEX,
lands in 3e.2). Persisted so .sql in sqlrite_master can carry auto vs
user-supplied text (also 3e.4).
- auto_name(table, col) = "sqlrite_autoindex_<table>_<col>" — matches
SQLite's convention and stays stable across save/open.
- Helper methods: insert / remove / lookup / iter_entries / would_violate_unique.
- 7 standalone unit tests covering auto_name, real-column rejection,
unique-violation detection, NULL-is-not-indexed, list semantics for
non-unique entries, iteration order, and synthesized_sql.
[src/sql/db/table.rs] Table gains `secondary_indexes: Vec<SecondaryIndex>`
and drops `indexes: HashMap<String, String>` (never used) plus Column.index
and Column.is_indexed. Column shrinks to its actual content: column_name,
datatype, is_pk, not_null, is_unique.
Rewrote every write path around the new type:
- Table::new auto-creates a SecondaryIndex for every UNIQUE / PK column
of an Integer or Text type. Real / Bool UNIQUE columns still fall back
to a linear scan inside validate_unique_constraint (no auto-index for
floating-point / binary).
- validate_unique_constraint now uses the index's would_violate_unique
when present; keeps the linear-scan path for the Real/Bool case.
- insert_row, set_value, delete_row, restore_row all maintain
secondary_indexes in lockstep. Scoped borrow blocks let row storage
and index updates share the same `self` without conflict.
- Dropped Column::get_mut_index and Table::get_column_mut (both
callers gone).
[src/sql/pager/mod.rs] build_empty_table (the open-path constructor)
mirrors Table::new and seeds secondary_indexes for UNIQUE/PK columns.
load_table_rows keeps the same signature — restore_row populates the
indexes transparently.
109 tests (was 102). +7 SecondaryIndex tests; everything else kept
passing with no change in observable behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/{executor,mod}.rs] Phase 3e.2/3e.3: CREATE INDEX + executor optimizer
[src/sql/mod.rs] recognize Statement::CreateIndex. Treat it as a write
for auto-save purposes. Reject `CREATE TABLE sqlrite_master` is already
in place from 3c.5.
[src/sql/executor.rs] new execute_create_index:
- Single-column only for MVP (multi-column rejected with NotImplemented)
- Requires an explicit index name (anonymous CREATE INDEX rejected)
- Validates the table exists, the column exists, the name is unused
(IF NOT EXISTS turns a duplicate into a silent success)
- Builds a SecondaryIndex with IndexOrigin::Explicit
- Populates it by walking table.rowids(); if UNIQUE is requested and
existing data already has duplicates, surfaces the conflict with a
precise "column already contains the duplicate value X" message
- Attaches the new index to table.secondary_indexes so subsequent
writes keep it maintained via the same paths 3e.1 wired up
- clone_datatype helper: DataType intentionally doesn't derive Clone,
so this is the explicit 6-arm clone used at the one caller site
Optimizer path (3e.3):
- RowidSource enum { IndexProbe(Vec<i64>) | FullScan } abstracts the
seed-rowids step. select_rowids(table, selection) chooses between
them.
- try_extract_equality recognizes `col = literal` and `literal = col`,
peeling one layer of Expr::Nested. AND / OR / range predicates fall
through to FullScan — those can be layered on later without touching
callers.
- execute_select, execute_delete, and execute_update all call
select_rowids first; IndexProbe returns precise rowids already
satisfying the equality, so no per-row predicate evaluation is
needed on that path. FullScan retains the existing eval-per-row loop.
- Integer-promotion note: when a literal comes through as Value::Integer
via convert_literal and the indexed column is Integer, the index's
internal i64 key matches and the probe succeeds.
115 tests (was 109). New coverage:
- create_index_adds_explicit_index (happy path + attaches to table)
- create_unique_index_rejects_duplicate_existing_values
- where_eq_on_indexed_column_uses_index_probe (100-row table, 33 matches)
- where_eq_on_indexed_column_inside_parens_uses_index_probe
- where_eq_literal_first_side_uses_index_probe
- non_equality_where_still_falls_back_to_full_scan
End-to-end REPL smoke test confirmed: CREATE INDEX, duplicate CREATE
INDEX rejection, UNIQUE index over existing duplicate data rejection,
and SELECT WHERE col = literal returns the right rows through the
probe path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/pager/*] Phase 3e.4: persist secondary indexes (format v3)
File format version bumps to 3. Two changes on disk:
- sqlrite_master gains a `type` column (first position) that distinguishes
'table' rows from 'index' rows. v2 files with four-column master tables
are rejected on open.
- Each SecondaryIndex persists as its own cell-based B-Tree using the same
leaf / interior / sibling-chain shape as user tables. Leaf cells use the
new KIND_INDEX format.
[src/sql/pager/header.rs] FORMAT_VERSION 2 → 3 with a history entry.
[src/sql/pager/cell.rs] add KIND_INDEX = 0x04. Value-block encode/decode
helpers (encode_value / decode_value) are now `pub(super)` so
index_cell can reuse them.
[src/sql/pager/index_cell.rs] new module:
- IndexCell { rowid, value } — one (indexed value, original rowid) pair.
- encode() / decode() use the shared [cell_length | kind_tag | body]
prefix; body is [rowid zigzag varint | value tag + body], matching a
one-column local cell minus the null bitmap (NULLs are never indexed).
- encode refuses Value::Null up front; decode rejects wrong kind tags.
- Cell::peek_rowid works uniformly because the body layout keeps rowid
at the same position after the kind tag.
- 5 round-trip tests.
[src/sql/pager/mod.rs]
- build_empty_master_table now has 5 columns, `type` first.
- save_database iterates tables first, then each table's secondary
indexes (sorted by (table, index name) for deterministic page
numbers). Tables stage via stage_table_btree as before; indexes
stage via the new stage_index_btree / stage_index_leaves, which mirror
the table tree build but emit IndexCells keyed on their original
rowid (ensuring the slot directory's binary search stays valid).
- The catalog now writes one row per table (kind='table', last_rowid
populated) and one row per index (kind='index', last_rowid=0,
sql = SecondaryIndex::synthesized_sql).
- open_database walks the master in two passes: load tables first, then
attach indexes via attach_index(). attach_index re-parses the
`CREATE INDEX` SQL, finds the base table, discards any auto-built
stub index of the same name, and repopulates from the persisted tree
by walking the leaves and decoding IndexCells one by one
(load_index_rows). This keeps auto- and explicit-index handling
uniform on disk even though auto indexes are rebuilt at Table::new
time.
- parse_create_index_sql / clone_datatype helpers.
- IndexCatalogRow / CatalogEntry tiny internal structs for pass-1 /
pass-2 bookkeeping.
[src/sql/pager/table_page.rs] expose slot_offset_raw for decoders that
parse slot bytes with a different cell type (specifically IndexCell on
index leaves).
122 tests (was 115). +5 IndexCell round-trip; +2 end-to-end
(explicit_index_persists_across_save_and_open,
auto_indexes_for_unique_columns_survive_save_open). REPL smoke test:
CREATE INDEX + UNIQUE auto-index both survive a save/reopen and are
usable via the WHERE-col-equals-literal fast path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [README.md,docs/{file-format,storage-model,roadmap,usage}.md] Phase 3e docs update
Reflects the secondary-index work that shipped in 3e.1–3e.4.
- README roadmap: mark 3e done; usage matrix gains a CREATE INDEX row,
documents auto-indexes for UNIQUE/PK, notes the WHERE-col-equals-
literal optimizer probe.
- docs/usage.md: new CREATE INDEX section (syntax, scope, rejection of
multi-column and anonymous indexes); SELECT gains an "Optimizer"
bullet.
- docs/file-format.md: format version now 3. Page 0 header diagram
shows version=3. KIND_INDEX added to the cell-kind table. New Index
cell body section documents the rowid-then-value layout and the
"NULLs are never indexed" invariant. sqlrite_master schema updated
to 5 columns with `type` first; save order documented (tables then
indexes, both alphabetical). Format-evolution section gets a v3
entry.
- docs/storage-model.md: replaces the "what 3e will change" bullet
with factual "what Phase 3e changed" section; points at roadmap for
Phase 3f+ and parks the cursor/lazy-load work in Phase 5.
- docs/roadmap.md: Phase 3e expanded with sub-phase commit hashes
and a one-line summary of each slice.
Code state unchanged — docs only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [Cargo.toml,src/lib.rs,src/{main,meta_command,repl}.rs] Phase 2.5.1: split engine into lib + REPL bin
Pulling forward the library split from Phase 5 — needed to let the Tauri
app (Phase 2.5) import the engine without shelling out, and will be
reused by the C FFI / WASM targets in Phase 5.
[Cargo.toml] rename package `SQLRite` → `sqlrite` (lowercase, conventional
Rust; README branding stays). Declare explicit [lib] and [[bin]] targets
both named `sqlrite`, sharing the same `src/` tree. Dependencies stay
in a single block — feature-gating the CLI-only ones (rustyline, clap,
env_logger) is a later polish.
[src/lib.rs] new library root. Declares `pub mod error` and `pub mod
sql`, pulls in prettytable as a crate-level #[macro_use] (moved from
main.rs — it's needed inside sql::db::table::print_table_schema),
and re-exports the public surface:
- Database
- process_command
- open_database / save_database
- Result / SQLRiteError
- MASTER_TABLE_NAME constant
Intentionally small. The `sql::pager`, `sql::executor`, `sql::parser`
modules stay accessible via `sqlrite::sql::…` for tests and tooling,
but aren't part of the public surface — their shapes will churn through
Phase 4 (WAL) and Phase 5 (cursor / lazy-load).
[src/main.rs] drops the `mod error` and `mod sql` declarations (now in
lib.rs); keeps `mod meta_command` and `mod repl` (CLI-specific). All
engine references go through the library crate: `use sqlrite::{Database,
process_command};`. Removes the `#[macro_use] extern crate prettytable`
since the macros are now pulled into the lib crate.
[src/meta_command/mod.rs] `use crate::error::…` → `use sqlrite::error::…`,
same for the other sql paths. `use crate::repl::REPLHelper` stays — repl
is a sibling bin-crate module. Unit tests updated to match.
[src/repl/mod.rs] `use crate::sql::*` narrowed to `use sqlrite::sql::SQLCommand`
(the only thing actually used from the engine's SQL module).
[src/sql/db/database.rs, src/sql/db/table.rs] doctest cleanup: the
`/// ``` ... ```` code blocks in print_table_schema/print_table_data
are informative prose, not runnable Rust — switched to ```text to stop
rustdoc from trying to compile them. Database::new's doctest was
rewritten to actually compile (`use sqlrite::Database;`).
123 tests pass (113 lib + 9 bin + 1 doctest). REPL behavior unchanged —
`cargo run` still drops into the prompt, CREATE / INSERT / SELECT all
work.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [desktop/*,Cargo.toml,src/*] Phase 2.5.2/2.5.3: Tauri 2.0 desktop app + thread-safe engine
Workspace + Tauri 2.0 + Svelte 5 UI on top of the engine library from
2.5.1. The Rust side compiles clean; the frontend needs `npm install`
in `desktop/` and then `npm run tauri dev` to launch (documented in the
upcoming docs update).
## Thread-safe engine (the one non-obvious refactor)
Tauri's State<T> requires `T: Send + Sync`, so a `Mutex<Database>` in
app state means Database has to be `Send + Sync` too. The engine had
`Rc<RefCell<HashMap<String, Row>>>` inside Table since 2020 — `Rc` isn't
`Send`.
Swapped to `Arc<Mutex<HashMap<String, Row>>>`. Every `.borrow()` /
`.borrow_mut()` becomes `.lock().expect("rows mutex poisoned")`. The
code paths already scope each borrow tightly so single-threaded
correctness is identical; the Mutex just adds cross-thread safety.
Also dropped every `#[derive(Serialize, Deserialize)]` on engine types
(DataType / Row / Column / Table / Database / SecondaryIndex /
IndexEntries / IndexOrigin). Those derives have been dead since 3c.5 —
all persistence goes through the hand-rolled cell codec. Dropping them
let us remove `serde` + `bincode` from the engine's direct deps.
## Workspace shape
[Cargo.toml] root becomes a workspace with `.` (the engine) and
`desktop/src-tauri/` as members. `resolver = "3"` at workspace level.
Engine dep list trimmed (serde, bincode gone; everything else stays).
[desktop/src-tauri/Cargo.toml] new crate `sqlrite-desktop`. Depends on
the engine by path. `tauri = "2"` + `tauri-plugin-dialog = "2"` for
the file picker.
[src/lib.rs] `pub use ::sqlparser;` re-exports the AST so the Tauri
crate can reach into the parser without adding a second direct dep.
## Tauri backend — desktop/src-tauri/src/main.rs
AppState holds `Mutex<Database>`. Four commands:
- open_database(path) — opens an existing .sqlrite or creates one
- list_tables() — sidebar data
- table_rows(name, limit) — seeds the result grid when a table is clicked
- execute_sql(sql) — runs a user query through process_command, and for
SELECT statements re-inspects the AST to ship structured rows back to
the UI (the executor itself still print!s to stdout; this hack will
go away in Phase 5's Cursor refactor).
Everything returns serde-serializable structs; the frontend gets
`{ kind: 'rows' | 'status', ... }` tagged enums.
## Frontend — desktop/src/*
Vite + Svelte 5 (runes). Single `App.svelte` holds the three-pane layout:
- Header with "Open…" button (file picker via tauri-plugin-dialog)
- Sidebar: table list + per-table schema view
- Main: textarea query editor + result grid (with Cmd/Ctrl+Enter to run)
[desktop/src/app.css] dark theme, monospace grid cells, sticky header.
## Scaffolding
[desktop/src-tauri/tauri.conf.json] v2 config; bundling disabled for
dev; 1200x800 default window.
[desktop/src-tauri/capabilities/default.json] core + dialog permissions.
[desktop/package.json] Svelte 5, Vite 5, Tauri CLI 2, plugin-dialog 2.
[desktop/vite.config.ts] Port 1420 matching tauri.conf.json.
[desktop/src-tauri/icons/icon.png] placeholder 32×32 PNG generated by
hand (Tauri's generate_context! requires at least one icon even in
non-bundled dev builds).
[.gitignore] adds desktop/node_modules, desktop/dist,
desktop/src-tauri/{target,gen}.
## Stats
123 tests pass (same as before the Arc/Mutex migration). `cargo build
--workspace` compiles the REPL and the Tauri app. `cargo run` still
drops into the REPL. The Tauri app runs with `npm install && npm run
tauri dev` from the `desktop/` directory (needs a system webview; on
macOS that's WKWebView, comes with the OS).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [README.md,docs/{_index,desktop,getting-started,roadmap}.md] Phase 2.5 docs
- README roadmap: mark Phase 2.5 done with the three-commit summary
(lib/bin split, Tauri scaffold + Svelte UI, thread-safe engine)
- docs/_index.md: add docs/desktop.md to the Internals list
- docs/desktop.md (new): architecture diagram, running instructions,
command surface, the SELECT re-run hack explanation, what's not in
the MVP, troubleshooting
- docs/getting-started.md: repo layout updated for workspace +
desktop/; new "Running the desktop app" section pointing at the
docs/desktop.md page
- docs/roadmap.md: Phase 2.5 entry expanded with the two sub-phase
commit hashes and a one-line summary of each slice; notes the
`npm install && npm run tauri dev` bootstrap
Code state unchanged — docs only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [docs/smoke-test.md,_index.md] add a step-by-step smoke-test walkthrough
Captures a repeatable sanity-check covering the REPL (in-memory + a
three-session persistence round-trip), the index-probe optimizer path,
error-case handling that must stay non-fatal, and the Tauri desktop
app end-to-end.
Structured so future feature work can drop regression steps into the
right part without reshuffling the whole doc:
- Prerequisites (cargo + npm + platform webview)
- Part 1 — REPL (in-memory): .help, CREATE / INSERT / SELECT projections
+ WHERE + ORDER BY, UPDATE with arithmetic, DELETE range, UNIQUE
auto-index rejection, CREATE INDEX + WHERE-equality probe, five
error cases that must not crash the REPL, clean .exit
- Part 2 — REPL (persistent, multi-session): three separate REPL
launches against the same .sqlrite file verifying auto-save survives
restart; bonus "bad-magic file is rejected cleanly" case
- Part 3 — Desktop app: npm install, npm run tauri dev, Open... dialog,
CREATE TABLE via editor, click-through table browsing, SELECT,
CREATE INDEX, UNIQUE-violation error surfacing, relaunch + verify
persistence
- Regression checklist — condensed 12-item bulleted list for fast
before/after checks
- When something fails — quick diagnostic guide for the most likely
stumbling blocks (REPL exit, bad magic, version mismatch, binary
rename, blank Tauri window, missing icon, npm install hangs)
Each step shows the exact command to type and the expected key strings
in the output — usable as both a human smoke test and a reference for
scripting a headless regression later.
[docs/_index.md] link the new doc under "Start here" with a short
description.
Code state unchanged — docs only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/main.rs,desktop/src/*,docs/*] CLI file argument + desktop "New…" button + help rewrite
Addresses feedback from the first hands-on pass:
- REPL's --help was terse (just "Print help"); no way to open a file
from the shell prompt
- Desktop app had an "Open…" button but no way to create a new .sqlrite;
system open dialogs either refuse nonexistent paths or silently
create garbage the engine then rejects
- Desktop app's query textarea pre-filled with `SELECT * FROM
sqlrite_master;`, which errors because the engine hides the catalog
from db.tables
[src/main.rs]
- Positional [FILE] argument via clap's .arg(). When present, opens the
file if it exists or creates it fresh; either way the long-lived
pager is attached so subsequent writes auto-save. Without the arg,
falls back to the transient in-memory default.
- Rewrote about / long_about with the full meta-command table and a
one-line summary of supported SQL + operators + optimizer hint.
Points readers at docs/usage.md and docs/smoke-test.md.
- New open_or_create helper — the same logic handle_open in
meta_command/ uses, hoisted so the CLI startup path can share it.
[desktop/src/App.svelte]
- Import `save as saveFileDialog` alongside `open as openFileDialog`
from tauri-plugin-dialog.
- New `onNewClick` handler: uses the save-dialog to let the user type
a fresh filename, then hands it to the existing `open_database`
Tauri command (which is already create-if-missing).
- Extract shared success path into `loadDatabase(path)` — both New
and Open go through it. If tables are present after load, the first
is auto-selected and its rows populate the grid.
- Default textarea content swapped from `SELECT * FROM sqlrite_master;`
(errored — the catalog isn't exposed to user queries) to a comment-
only placeholder that survives being Run without any SQL.
[desktop/src/app.css] small `.actions { gap: 8px }` so New… and Open…
don't butt up against each other.
[docs/usage.md] "Launching" section updated to mention the positional
FILE argument and --help's inline content.
[docs/desktop.md] Header section describes both New… and Open… buttons
and notes the open-dialog-refuses-missing-paths distinction. New entry
in the "what's not here yet" list explains the sqlrite_master-from-SQL
quirk so future users don't hit the same thing.
[docs/smoke-test.md]
- Part 1.1 extended with a --help verification step
- New Part 2.5 "CLI file argument (shortcut)" round-trip
- Part 3 renumbered: 3.4a covers New…, 3.4b covers Open…
- Regression checklist grows from 12 to 18 items, covering the new
affordances and the .tables meta-command (which was always there but
not called out explicitly)
Code paths touched: 123 tests still pass; `cargo run -- mydb.sqlrite`
creates / opens; `--help` prints the long description; no desktop-side
Rust changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [src/sql/mod.rs,desktop/src/App.svelte] fix: panic on empty-AST input; Svelte a11y warning
## The crash
Desktop app pre-fills the query textarea with a comment-only placeholder
so Run-without-typing doesn't error. That placeholder parses through
sqlparser into a zero-statement AST. `process_command` checked
`ast.len() > 1` but blindly `pop().unwrap()`'d — which panics on empty.
Tauri's main thread panicking is fatal: the window dies.
Reported terminal output:
```
thread 'main' panicked at src/sql/mod.rs:56:27:
called `Option::unwrap()` on a `None` value
fatal runtime error: failed to initiate panic, error 5, aborting
```
[src/sql/mod.rs] replace the `unwrap()` with a let-else that returns a
clean "No statement to execute." status. No write flag is set, so
auto-save doesn't run on a no-op. The REPL's input validator already
rejects statements without a `;`, so it never triggered this path —
but the Tauri app sends raw textarea contents, which does.
New regression test covers four inputs: empty string, whitespace-only,
single comment, multi-line comments. 114 lib tests (was 113) pass.
## The a11y warning
```
[vite-plugin-svelte] src/App.svelte:173:12
Non-interactive element `<li>` cannot have interactive role 'button'
```
[desktop/src/App.svelte] the sidebar's clickable `<li>` items were
tagged `role="button"`, which Svelte correctly flagged — `<li>` isn't
in the set of elements ARIA allows to carry an interactive role.
Switched to the proper list-selection pattern:
- Parent `<ul>` gets `role="listbox"` + `aria-label="Tables"`
- Each `<li>` gets `role="option"` + `aria-selected={selected ? ... }`
Same visual behavior; screen readers now describe the sidebar as "a
listbox of tables, users selected" instead of "a button, a button…".
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [desktop/src/{App.svelte,app.css},docs/*] editor: line-number gutter + ⌘/ comment toggle
Two code-editor staples on the desktop query surface. No Rust changes —
everything is in the Svelte component, its CSS, and the docs.
## Line-number gutter
[desktop/src/App.svelte]
- `textareaRef` / `gutterRef` bindings via `bind:this` so the scroll
handler can cross-reference them.
- `lineNumbers` is `$derived` from `sql.split("\n").length` and rendered
as a dense `[1, 2, …, n]` array so Svelte's {#each} iterates every
slot (a sparse `Array(n)` would skip indices on some runtimes).
- `onEditorScroll` (attached to the textarea's `onscroll`) copies
`textareaRef.scrollTop` onto `gutterRef.scrollTop` so numbers stay
aligned with text rows even after the user scrolls a long query.
[desktop/src/app.css]
- New `.editor-surface` flex wrapper hosts the gutter + textarea
side-by-side.
- `.gutter` has `overflow: hidden` (so the sync scroll doesn't expose
a second scrollbar) plus identical `font-family` / `font-size` /
`line-height` to the textarea. That symmetry is required — any drift
in line-height breaks the row alignment; the CSS has a comment
pinning the invariant.
- `.line-num { height: calc(1em * 1.5) }` matches the textarea's
`line-height: 1.5` so each gutter row sits exactly one text row tall.
- Textarea picks up `tab-size: 4` while we're in there.
## Comment toggle (Cmd/Ctrl + /)
[desktop/src/App.svelte]
- New `toggleComment()` async function. Expands the selection outward
to whole-line boundaries, splits into lines, decides the toggle
direction by checking whether every *non-blank* line in the range
already starts with `--`, then either strips the leading `-- ` (or
bare `--`) with a regex or prepends `-- ` to each non-blank line.
Blank lines stay blank either way. Matches VS Code's behavior for a
mixed-state selection (adds prefix uniformly).
- Cursor restoration via `await tick()` — re-selects the edited block
so a second ⌘/ toggles it back. Keeps the textarea focused.
- `onKey` dispatcher adds a branch for `(ctrl|meta)+/` alongside the
existing `(ctrl|meta)+Enter` run binding.
## Editor toolbar
- New `.shortcut-hint` on the toolbar row: "Run: ⌘↵ · Comment: ⌘/"
replaces the hint that used to live inside the Run button label.
- Toolbar now `justify-content: space-between` so hints are on the
left, Run button on the right.
## Docs
[docs/desktop.md] Main-area bullet lists both shortcuts and notes the
font-size/line-height lockstep requirement.
[docs/smoke-test.md]
- New Part 3.4c section exercises the gutter (count + scroll sync) and
the comment toggle (single-line toggle, round-trip via a second ⌘/,
multi-line selection, mixed-state selection).
- Regression checklist grows by two items.
Workspace still compiles (`cargo check --workspace` clean). Ran
through the comment-toggle scenarios on paper: single line on/off,
multi-line all-commented → uncomment, multi-line mixed → comment,
blank-line cursor = no-op, selection ending at a newline = expected
whole-line behavior.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [desktop/*,docs/*] add Save As… command + button to persist in-memory databases
Plugs a real gap: the desktop app let users start with no file and
build up schema in memory, but there was no button to persist that
work. The REPL has `.save FILE` for this; the desktop had nothing.
## Backend
[desktop/src-tauri/src/main.rs] new Tauri command
`save_database_as(path)`:
1. Calls `sqlrite::save_database(db, path)` to flush the current
in-memory state to the new path.
2. Reopens the just-saved file via `sqlrite::open_database` to
reattach the Pager bound to that path and set `source_path` —
every subsequent committing statement now auto-saves there.
Subtle semantic difference from the REPL's `.save FILE`: the REPL
writes without adopting the new path (current session stays pointed
at whatever `.open`'d earlier, if anything). In a GUI context users
expect "Save As…" to switch the active file, so the Tauri command
adopts. Documented in docs/desktop.md under Commands.
Registered in the invoke_handler! macro alongside the existing four
commands.
## Frontend
[desktop/src/App.svelte]
- `onSaveAsClick` handler — uses the save dialog (default path =
current db path if any, else `untitled.sqlrite`), invokes
`save_database_as`, refreshes the sidebar, preserves the selected
table if it survived the save, updates the status line with
"Saved as …. N tables. Auto-save enabled."
- Third header button **Save As…** next to New… / Open….
## Docs
[docs/desktop.md]
- Header bullet describes all three buttons and their interaction
with the pager lifecycle
- New "Commands" sub-table under "State" listing every Tauri command
and what it does
[docs/smoke-test.md]
- New Part 3.4c "Save As… (save an in-memory DB to a file)" — starts
fresh session, creates schema in memory, Save As…, relaunch, open
the saved file, verify rows persist
- Old 3.4c (gutter + comment toggle) renumbered to 3.4d
- Regression checklist grows one item
Workspace builds clean (`cargo check --workspace`). No engine changes
— this is pure wiring on top of `save_database` + `open_database`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [desktop/src/App.svelte,docs/*] editor: Run button executes only the selected SQL when a selection is active
DataGrip / DBeaver / pgAdmin convention — if you have text selected in
the editor, Run executes exactly that substring; otherwise it runs the
full editor contents. Unblocks the "many statements in one textarea,
run them one by one" workflow that was previously blocked by the
engine's single-statement-per-call rule.
[desktop/src/App.svelte]
- `onRunSql` consults `textareaRef.selectionStart/selectionEnd`. If
the two differ, it slices `ta.value` for that range and sends only
those bytes to the backend; otherwise it sends the full `sql` state.
sqlparser handles leading/trailing whitespace, so no trimming
needed.
- `hasSelection` `$state` flag backed by a document-level
`selectionchange` listener in a `$effect` (covers mouse drag,
keyboard shift+arrow, programmatic selection — all the ways a
user can change selection state). Clears when the textarea loses
focus.
- Run button label becomes **Run selection** when `hasSelection` is
true. Shortcut hint appends "· selection only" so the state is
legible without hovering the button.
[docs/desktop.md] Main-area bullet updated to describe the
selection-aware Run + the dynamic button label.
[docs/smoke-test.md]
- New Part 3.4e "Run-selected-only": put several statements in the
editor, observe that no-selection Run errors with
"Expected a single query statement", then select one and verify
it runs in isolation + the button label flips.
- 3.4d (gutter + comment) reordered so the numbering goes
a/b/c/d/e (it had been a/b/c/e/d after the earlier Save As
insertion).
- Regression checklist grows one more item.
`cargo check --workspace` clean. No Rust changes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [README.md,docs/desktop.md,images] embed desktop screenshot
[images/SQLRite - Desktop.png] new screenshot showing the app with a
users table open: header with New… / Open… / Save As… buttons, sidebar
listing the table + its schema (id INTEGER PK, name TEXT), query
editor with the live line-number gutter, result grid with one row
(Josh).
[README.md] new "Desktop app" section slotted between the existing
REPL asciicast and the Developer guide block. Embeds the screenshot,
points at desktop/ and docs/desktop.md, and summarizes the key
affordances (file-lifecycle buttons, gutter, ⌘/ comment toggle,
selection-aware Run).
[docs/desktop.md] screenshot at the top right after the intro, with a
short caption describing the layout so readers know what to look at
first.
Both references use the `<path with spaces>` markdown syntax so the
filename's space doesn't break the link (GitHub + most renderers
respect the angle-bracket form).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* [Cargo.toml,src/sql/pager/pager.rs,src/main.rs,docs/*,README.md] Phase 4a: exclusive file lock on the Pager
First slice of Phase 4. Every Pager::open / Pager::create now takes a
non-blocking OS advisory exclusive lock on the `.sqlrite` file via
`fs2::FileExt::try_lock_exclusive` — `flock(LOCK_EX | LOCK_NB)` on
Unix, `LockFileEx(LOCKFILE_EXCLUSIVE_LOCK | LOCKFILE_FAIL_IMMEDIATELY)`
on Windows. A second SQLRite process that tries to open the same file
gets a clean error:
database '/…1 parent 688bee9 commit 718ba31
108 files changed
Lines changed: 28739 additions & 979 deletions
File tree
- .github/workflows
- desktop
- src-tauri
- capabilities
- icons
- src
- src
- docs
- examples
- c
- go
- nodejs
- python
- rust
- wasm
- images
- scripts
- sdk
- go
- nodejs
- src
- test
- python
- src
- tests
- wasm
- src
- sqlrite-ffi
- include
- src
- src
- meta_command
- repl
- sql
- db
- pager
- parser
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
0 commit comments