|
| 1 | +# AGENTS.md |
| 2 | + |
| 3 | +This file provides guidance to Codex (Codex.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Project |
| 6 | + |
| 7 | +SQLRite is a from-scratch SQLite-style embedded database written in Rust. It's published on crates.io as `sqlrite-engine` (imported as `use sqlrite::…` — the lib target keeps the short name) and ships as: a REPL binary (`sqlrite`), a Tauri 2 + Svelte 5 desktop app, a Model Context Protocol stdio server (`sqlrite-mcp`), a C FFI shim (`sqlrite-ffi`), and language SDKs (Python via PyO3, Node via napi-rs, Go via cgo, WASM via wasm-bindgen). Phases 1–7 are shipped; the current branch `phase-8-plan` drafts inverted-index + BM25 full-text search and hybrid retrieval. |
| 8 | + |
| 9 | +## Workspace layout |
| 10 | + |
| 11 | +`Cargo.toml` is a workspace whose members are: `.` (the engine, package `sqlrite-engine`, lib `sqlrite`), `desktop/src-tauri`, `sqlrite-ffi`, `sqlrite-ask`, `sqlrite-mcp`, `sdk/python`, `sdk/nodejs`, `benchmarks`. `sdk/wasm` and `sdk/go` are deliberately **not** workspace members (wasm32 target / cgo separation). |
| 12 | + |
| 13 | +- `src/` — engine. Public API is `Connection`/`Statement`/`Rows`/`Row`/`Value` from [src/connection.rs](src/connection.rs), re-exported via [src/lib.rs](src/lib.rs). Any new SDK should bind only to this surface. |
| 14 | +- `sqlrite-ask/` — pure-Rust LLM adapter (Anthropic/OpenAI/Ollama) for natural-language → SQL. The engine's `ask` feature provides the thin `ConnectionAskExt::ask` glue. |
| 15 | +- `sqlrite-mcp/` — MCP stdio server. Seven tools: `list_tables`, `describe_table`, `query`, `execute`, `schema_dump`, `vector_search`, `ask`. `--read-only` opens with a shared lock and hides `execute`. |
| 16 | +- `sqlrite-ffi/` — C ABI cdylib + generated `sqlrite.h` header. Backs the Go SDK and any C consumer. |
| 17 | +- `desktop/` — Tauri 2 + Svelte 5 GUI. Embeds the engine directly (no FFI hop). |
| 18 | +- `benchmarks/` — SQLR-4 / SQLR-16 bench harness. `Driver` trait + SQLRite + SQLite (rusqlite-bundled) drivers + criterion-driven workloads. Excluded from the default CI build/test/clippy/doc commands; run locally with `make bench` (or `make bench-duckdb`). See [docs/benchmarks-plan.md](docs/benchmarks-plan.md). |
| 19 | +- `web/` — marketing + docs site (Next.js 15 + Tailwind v4). Independent of the Cargo workspace; lives in-repo for now but is structured to lift into its own repository later. See [web/README.md](web/README.md). |
| 20 | + |
| 21 | +Architecture deep-dive: [docs/architecture.md](docs/architecture.md). The full doc index is [docs/_index.md](docs/_index.md). |
| 22 | + |
| 23 | +## Engine data flow |
| 24 | + |
| 25 | +SQL string → [src/sql/mod.rs](src/sql/mod.rs) `process_command` parses with the external `sqlparser` crate (SQLite dialect) → [src/sql/parser/](src/sql/parser/) trims the AST into internal structs (`CreateQuery`, `InsertQuery`, `SelectQuery`) → [src/sql/executor.rs](src/sql/executor.rs) runs the statement against the in-memory `Database` ([src/sql/db/database.rs](src/sql/db/database.rs)). On any write, auto-save serializes changed pages through [src/sql/pager/](src/sql/pager/) — 4 KiB pages, cell-encoded B-trees per table and index, WAL + crash-safe checkpoint, fs2 advisory locks. Vector search (Phase 7d) goes through [src/sql/hnsw.rs](src/sql/hnsw.rs); KNN uses a bounded-heap top-k in the executor. Transactions snapshot the in-memory state; ROLLBACK restores it. There is no query optimizer beyond the KNN/HNSW shortcut, no joins, no aggregates yet. |
| 26 | + |
| 27 | +## Commands |
| 28 | + |
| 29 | +CI is the source of truth — the workspace excludes that follow are required because the desktop crate needs a Svelte build first, the PyO3/napi-rs cdylibs can't link standalone test binaries, and the `benchmarks/` harness deliberately stays out of CI (criterion is noisy on shared runners; the rusqlite-bundled build is heavy). |
| 30 | + |
| 31 | +```sh |
| 32 | +# Build / test the Rust workspace (matches CI) |
| 33 | +cargo build --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets |
| 34 | +cargo test --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks |
| 35 | + |
| 36 | +# Single test (exact name; --nocapture to see println!) |
| 37 | +cargo test <test_name> -- --nocapture |
| 38 | + |
| 39 | +# Lint (CI runs all three) |
| 40 | +cargo fmt --all -- --check |
| 41 | +cargo clippy --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --all-targets |
| 42 | +cargo doc --workspace --exclude sqlrite-desktop --exclude sqlrite-python --exclude sqlrite-nodejs --exclude sqlrite-benchmarks --no-deps |
| 43 | + |
| 44 | +# Run the REPL (default features include cli + ask + file-locks) |
| 45 | +cargo run # in-memory |
| 46 | +cargo run -- path/to/db.sqlrite # open/create file |
| 47 | +cargo run -- --readonly path/to/db.sqlrite # shared-lock open |
| 48 | + |
| 49 | +# Crate-specific |
| 50 | +cargo build --release -p sqlrite-ffi # C cdylib + sqlrite.h |
| 51 | +cd desktop && npm install && npm run tauri dev # desktop app dev mode |
| 52 | +cargo run -p sqlrite-mcp -- /path/to.sqlrite # MCP server (stdio) |
| 53 | + |
| 54 | +# Benchmarks (SQLR-4 / SQLR-16) — local-only, never CI |
| 55 | +make bench # SQLRite + SQLite (lean) |
| 56 | +make bench-duckdb # adds DuckDB driver (Group B only) |
| 57 | + |
| 58 | +# Release plumbing |
| 59 | +scripts/bump-version.sh 0.2.0 # bumps version across 11 manifests |
| 60 | +``` |
| 61 | + |
| 62 | +`SQLRITE_LLM_API_KEY` is required for the `.ask` REPL command, the engine's `ask` feature, and the MCP `ask` tool. Clippy is **not** `-D warnings` yet (intentional — see top of [.github/workflows/ci.yml](.github/workflows/ci.yml)); deny-by-default lints still fail CI. |
| 63 | + |
| 64 | +## Project-specific conventions |
| 65 | + |
| 66 | +- **Errors.** Single `SQLRiteError` enum (thiserror, six variants) with a project-wide `Result<T>` alias. All public APIs return typed errors; no panics. The enum hand-rolls `PartialEq` because `std::io::Error` doesn't derive it. |
| 67 | +- **Storage isn't bincode.** Tables and indexes share a cell-encoded B-tree format with a 4 KiB page size; the file header carries a format version (currently v4 after the Phase 7a vector column work). The diff-based pager only writes changed pages. See [docs/file-format.md](docs/file-format.md) and [docs/pager.md](docs/pager.md). |
| 68 | +- **B-tree commit strategy.** Bottom-up rebuild on every commit (O(N), correct-by-construction). No in-place splits — deferred design decision. |
| 69 | +- **Feature gates matter.** `default = ["cli", "ask", "file-locks"]`. The REPL `[[bin]]` `required-features = ["cli", "ask"]`. WASM and lean library embeddings build with `default-features = false` to avoid rustyline / clap / fs2 / sqlrite-ask. Don't pull these into the always-on dependency set. |
| 70 | +- **Don't reinvent the SQL parser.** `sqlparser` is the tokenizer and AST source; project code only narrows that AST. New SQL features start by mapping the existing `sqlparser` AST node, not by extending a custom grammar. |
| 71 | +- **Phase numbering is real.** The roadmap is sequenced in [docs/roadmap.md](docs/roadmap.md); design discussions live at github.com/sqlrite/design and feature work generally tracks an open phase plan in `docs/phase-*-plan.md`. Treat in-flight phase plans as load-bearing context. |
| 72 | +- **Concurrency.** Engine mutates state through `Arc<Mutex<_>>` (Tauri-friendly). On-disk concurrency uses fs2 advisory locks: shared for readers, exclusive for the single writer. |
| 73 | + |
| 74 | +## Knowledge Base |
| 75 | + |
| 76 | +### Project-specific — `~/Documents/josh-obsidian-synced/Projects/rust_sqlite/` |
| 77 | + |
| 78 | +- **Code:** `/Users/joaoh82/projects/rust_sqlite` |
| 79 | +- **Context (read first):** `~/Documents/josh-obsidian-synced/Projects/rust_sqlite/context.md` |
| 80 | +- **Notes (running journal):** `~/Documents/josh-obsidian-synced/Projects/rust_sqlite/notes.md` |
| 81 | +- **Project wiki:** `~/Documents/josh-obsidian-synced/Projects/rust_sqlite/wiki/` |
| 82 | + |
| 83 | +**How to use each:** |
| 84 | + |
| 85 | +- `context.md` — stable background (product goals, stakeholders, domain). Read before starting non-trivial work. Update only when underlying facts change. |
| 86 | +- `notes.md` — append-only dated journal. Add entries under `## YYYY-MM-DD` headings for decisions, blockers, TODOs, and incidents — anything worth preserving but not stable enough for `context.md`. |
| 87 | +- `wiki/` — reference sub-docs (e.g. `Architecture.md`, `Local Dev Setup.md`, `Tech Services.md`). Create new files as topics emerge. |
| 88 | + |
| 89 | +**When to save:** |
| 90 | + |
| 91 | +- New stable fact about the product/domain → update `context.md`. |
| 92 | +- A decision, incident, or working note → append a dated entry to `notes.md`. |
| 93 | +- Reusable reference material (setup steps, credential locations, architecture) → new/updated file in `wiki/`. |
| 94 | + |
| 95 | +### Cross-project knowledge — `~/Documents/josh-obsidian-synced/vault/` |
| 96 | + |
| 97 | +- **General wiki:** `~/Documents/josh-obsidian-synced/vault/wiki/` — start at `_master-index.md`, then drill into the relevant topic's `_index.md`. |
| 98 | +- **Raw dumps:** `~/Documents/josh-obsidian-synced/vault/raw/` — drop unprocessed research here as `YYYY-MM-DD-{slug}.md`. |
| 99 | + |
| 100 | +Read the general wiki when the question isn't specific to this project. Drop raw research or imported notes into `vault/raw/` so it's captured even before it's distilled. |
0 commit comments