Skip to content

Commit 27c8029

Browse files
joaoh82claude
andauthored
docs: pre-release sweep — surface Phase 7 completion + MCP discoverability (#74)
Audit pass over docs/ + README + examples + SDK READMEs ahead of the v0.1.25 release that ships sqlrite-mcp (Phase 7h) for the first time. Phase 7 is now feature-complete except for the deferred FTS scope, so a lot of "in progress" / "pending" / "shipping in 7g.X-7g.Y" wording turned stale and needed flipping. The biggest single gap was MCP discoverability — `sqlrite-mcp` was only mentioned in its own README + a few cross-refs in the docs we just shipped. Anyone landing on the root README, an SDK README, or the Embedding doc would have no idea it existed. Added a one-liner "sibling product" cross-ref everywhere. ## Changes - **README.md** — flipped the Phase 7 "(in progress)" marker; added a full "MCP server" section near the desktop install with `cargo install sqlrite-mcp` + a Claude Code config snippet; rewrote the "Per-product wrappers ship in 7g.3-7g.8" claim into past tense with the MCP tool included; flipped the stale Phase 4 marker; flipped "examples for every language" checkbox to done; dropped the stale `0.1.0` REPL banner literal; rewrote the Phase 2 onboarding hint about persistence ("`.open` coming in Phase 2" → present-day "use `.open <path>`"). - **docs/_index.md** — refreshed the "Project state" with Phase 7 achievements (vector + HNSW, JSON, ask family, MCP); added MCP to the published-artifacts list; flipped active frontier from "Phase 7" to "Phase 8 (FTS)"; bumped May 2026. - **docs/roadmap.md** — added an "Active frontier" callout at the top; dropped "(in progress)" from Phase 3 and Phase 4 headings. - **docs/phase-7-plan.md** — flipped status from "approved 2026-04-26 — implementation pending" to "complete except 7f, deferred to Phase 8"; rewrote "Next steps" as a retrospective with pointers to the two scope-correction retros (7d's HNSW split into three sub- phases, 7g.2's dep-direction flip). - **docs/embedding.md** — rewrote the Phase-5 status block (was future tense across 5b-5g) into a retrospective; added 7g + 7h bullets with the framing "use SDKs when YOUR code drives the DB, use MCP when an LLM AGENT drives it"; rewrote the "Per-product wrappers ship in 7g.3-7g.8" line. - **docs/architecture.md** — added a Workspace layout table covering every workspace member (sqlrite-engine root, sqlrite-ffi, sqlrite- ask, sqlrite-mcp, the four SDKs, desktop) with what each does; refreshed the engine module map to include connection.rs, ask/, hnsw.rs, json.rs (all Phase 7 additions); rewrote the stale "What's deliberately missing" section (transactions/locks/concurrency now shipped — flipped to the actually-still-missing list: optimizer beyond eq-probe, joins, aggregates, network layer, streaming cursor). - **docs/supported-sql.md** — added an HNSW indexes subsection under CREATE INDEX with the `USING hnsw` syntax + the recall@10 ≥ 0.95 default + a note that brute-force still works without the index. - **docs/smoke-test.md** — dropped the stale "123 tests pass" pin; added a one-line smoke step for `cargo run --bin sqlrite-mcp -- --help` to catch regressions in the stdio_redirect dance; updated the "unsupported format version" note (3 → 4, since 7a bumped it); dropped the `0.1.0` REPL banner literal. - **docs/getting-started.md** — refreshed the repo-tree map to include sqlrite-ffi/, sqlrite-ask/, sqlrite-mcp/, sdk/{python, nodejs,go,wasm}/, examples/, scripts/, plus the new engine modules (connection.rs, ask/, hnsw.rs, json.rs); dropped the `0.1.0` REPL banner literal. - **docs/usage.md** — flipped the future-tense "SDKs land in Phases 5b – 5g" to past tense; added MCP cross-ref. - **docs/desktop.md** — rewrote the "see Roadmap Phase 5" cursor- API line to point specifically at the deferred 5a.2. - **sdk/{python,nodejs,go,wasm}/README.md** — added a "Sibling products" subsection to each pointing at sqlrite-mcp with the "your code vs LLM agent" framing. - **sqlrite-ask/README.md** — NEW. The crate had no README, which means the crates.io page would render only the Cargo.toml description. Now: install snippet, "most users want the engine's ask feature instead", configuration env-var table, features table, architecture notes, sibling-products list. Cargo.toml gets the matching `readme = "README.md"` field plus keywords + categories for crates.io discoverability. - **examples/README.md** — added an MCP row to the "Languages" table pointing at docs/mcp.md (no examples/mcp/ subdir — the doc covers the wiring directly, lower friction than maintaining parallel quickstart files). No code changes; no new tests. Existing test suite still passes (workspace builds clean, `cargo test --workspace` minus the SDKs that need their own runners shows ~340 tests passing across the engine + sqlrite-mcp + sqlrite-ask + sqlrite-ffi). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 860f843 commit 27c8029

18 files changed

Lines changed: 248 additions & 50 deletions

README.md

Lines changed: 32 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,32 @@ A cross-platform Tauri 2.0 + Svelte 5 desktop GUI ships alongside the REPL (see
8080
8181
**From source**`cd desktop && npm install && npm run tauri dev`. The header's New… / Open… / Save As… buttons cover the file lifecycle; the query editor has a live line-number gutter, `⌘/` (Ctrl+/) SQL comment toggle, and selection-aware Run (highlight a statement to run just that one).
8282

83+
### MCP server (drive SQLRite from an LLM agent)
84+
85+
`sqlrite-mcp` exposes a SQLRite database as a [Model Context Protocol](https://modelcontextprotocol.io/) stdio server. Spawn it from any MCP client (Claude Code, Cursor, `mcp-inspector`, …) and the agent gets seven tools — `list_tables`, `describe_table`, `query`, `execute`, `schema_dump`, `vector_search`, plus `ask` for natural-language → SQL — without any custom integration code.
86+
87+
```sh
88+
cargo install sqlrite-mcp
89+
```
90+
91+
Or grab a per-platform pre-built binary from the [latest release](https://github.com/joaoh82/rust_sqlite/releases/latest) (Linux x86_64/aarch64, macOS aarch64, Windows x86_64).
92+
93+
Wire it into Claude Code (`~/.claude.json`):
94+
95+
```json
96+
{
97+
"mcpServers": {
98+
"sqlrite": {
99+
"command": "sqlrite-mcp",
100+
"args": ["/absolute/path/to/your.sqlrite"],
101+
"env": { "SQLRITE_LLM_API_KEY": "sk-ant-…" }
102+
}
103+
}
104+
}
105+
```
106+
107+
`--read-only` opens the DB with a shared lock and hides the `execute` tool. Full docs + the other six tools' references in [`docs/mcp.md`](docs/mcp.md).
108+
83109
### Developer guide
84110

85111
In-depth documentation lives under [`docs/`](docs/). Start at [`docs/_index.md`](docs/_index.md) — it navigates to:
@@ -100,10 +126,10 @@ Build and launch the REPL:
100126
cargo run
101127
```
102128

103-
You'll drop into a REPL connected to a transient in-memory database. On-disk persistence (`.open`, `.save`) is coming in Phase 2.
129+
You'll drop into a REPL connected to a transient in-memory database. Use `.open <path>` to switch to a file-backed database (auto-saves on every statement; see Meta commands).
104130

105131
```
106-
SQLRite - 0.1.0
132+
SQLRite
107133
Enter .exit to quit.
108134
Enter .help for usage hints.
109135
Connected to a transient in-memory database.
@@ -183,7 +209,7 @@ println!("Why: {}", resp.explanation);
183209

184210
**Defaults:** `claude-sonnet-4-6`, `max_tokens: 1024`, schema dump cached for 5 minutes via Anthropic prompt caching (configurable to 1h or off via `AskConfig::cache_ttl`). Bring your own API key — set `SQLRITE_LLM_API_KEY` or pass it on `AskConfig`.
185211

186-
In the REPL: `.ask <question>`. From an open `Connection` (this section). Per-product wrappers desktop "Ask" button, `conn.ask()` in the Python / Node / Go SDKs, MCP `ask` tool, WASM with a JS-callback shape so the API key never enters the browser — ship in **7g.3-7g.8** as follow-up sub-phases. See [`docs/phase-7-plan.md`](docs/phase-7-plan.md) §7g for the full surface plan.
212+
In the REPL: `.ask <question>`. From an open `Connection` (this section). Per-product wrappers shipped across **7g.3 – 7g.8**: the desktop "Ask…" composer, `conn.ask()` / `db.ask()` in the Python / Node / Go SDKs, the WASM SDK's split `db.askPrompt()` / `db.askParse()` shape (so the API key never enters the browser), and the MCP `ask` tool exposed by [`sqlrite-mcp`](docs/mcp.md) (so any LLM agent over MCP can call `ask` against your database directly). See [`docs/phase-7-plan.md`](docs/phase-7-plan.md) §7g for the full surface plan.
187213

188214
For the canonical Ask reference covering every surface (REPL, desktop, Rust library, Python / Node / Go / WASM), env vars, defaults, prompt caching, and the security model — read [`docs/ask.md`](docs/ask.md). For copy-paste backend proxy templates the WASM SDK needs (Cloudflare Workers, Vercel Edge, Deno Deploy, Firebase, AWS Lambda, Express), see [`docs/ask-backend-examples.md`](docs/ask-backend-examples.md).
189215

@@ -230,7 +256,7 @@ The project is staged in phases, each independently shippable. A finished phase
230256
- [x] **Tauri 2.0 backend**: four commands (`open_database`, `list_tables`, `table_rows`, `execute_sql`) wrap the engine; results are tagged enums shipped to the UI via the JSON IPC bridge.
231257
- [x] **Svelte 5 frontend**: dark-themed three-pane layout — header with "Open…" file picker, sidebar with table list + schema, query editor with Cmd/Ctrl+Enter to run, result grid with sticky header.
232258

233-
**Phase 4 — Durability and concurrency** *(in progress)*
259+
**Phase 4 — Durability and concurrency** *(done)*
234260
- [x] **4a — Exclusive file lock**: `Pager::open` / `::create` takes an OS advisory lock (`fs2::try_lock_exclusive`); a second process on the same file gets a clean "already in use" error. Lock releases automatically when the Pager drops.
235261
- [x] **4b — Write-Ahead Log (`<db>.sqlrite-wal`) file format + frame codec**: 32-byte WAL header (magic / version / page size / salt / checkpoint seq), 4112-byte frames carrying `(page_num, commit_page_count, salt, checksum, body)`. Rolling-sum checksum. Torn-write recovery: corrupt or partial trailing frames are silently truncated at the boundary. Standalone module; not wired yet.
236262
- [x] **4c — WAL-aware Pager**: `Pager::open` / `::create` now own both the main file and its `-wal` sidecar. Reads resolve `staged → wal_cache → on_disk` with a page-count bounds check; commits append a WAL frame per dirty page plus a final commit frame carrying the new page 0 (encoded header). The main file stays frozen between checkpoints — reopening replays the WAL and the decoded page-0 frame overrides the (stale) main-file header.
@@ -246,7 +272,7 @@ The project is staged in phases, each independently shippable. A finished phase
246272
- [x] **5e — Go SDK**: new `sdk/go/` module at `github.com/joaoh82/rust_sqlite/sdk/go`; cgo-wired against `libsqlrite_c` from Phase 5b. Implements the full `database/sql/driver` surface so users get the standard-library experience (`sql.Open("sqlrite", path)`, `db.Query/Exec/Begin`, `rows.Scan(&id, &name)`). 9-test `go test` integration suite. `examples/go/hello.go` runs after `cargo build --release -p sqlrite-ffi`. Module publish landed in Phase 6i — `go get github.com/joaoh82/rust_sqlite/sdk/go@vX.Y.Z` resolves directly via VCS tag.
247273
- [ ] **5f — Rust crate polish** *(deferred — Phase 6c companion)*: crate metadata, docs.rs config, prep for `cargo publish`. Deferred to land alongside the actual publish workflow.
248274
- [x] **5g — WASM** build: new `sdk/wasm/` crate via `wasm-bindgen`; engine runs entirely in a browser tab. Feature-gated root crate (`cli` + `file-locks` optional, both default-on) so WASM disables fs2 / rustyline / clap / env_logger cleanly. `Database` class with `exec/query/columns/inTransaction`; rows as plain JS objects in projection order. ~1.8 MB wasm / ~500 KB gzipped. Three `wasm-pack` targets (web/bundler/nodejs). `examples/wasm/` ships a self-contained HTML SQL console.
249-
- [ ] Code examples for every language under `examples/{rust,python,nodejs,go,wasm}/`
275+
- [x] Code examples for every language under `examples/{rust,python,nodejs,go,wasm,c}/`
250276

251277
**Phase 6 — Release engineering + CI/CD**
252278
Lockstep versioning — one dispatch bumps every product to the same `vX.Y.Z`. Two-workflow design: `release-pr.yml` opens a Release PR with the version bumps (human reviews + merges), then `release.yml` fires on merge to tag + publish everything. Trusted-publishing via OIDC for PyPI + npm (no long-lived tokens). Full plan: [`docs/release-plan.md`](docs/release-plan.md).
@@ -265,7 +291,7 @@ Lockstep versioning — one dispatch bumps every product to the same `vX.Y.Z`. T
265291
- [ ] macOS Apple Developer ID cert → `codesign` + `notarytool` in `tauri-action`
266292
- [ ] Windows code-signing cert → `signtool` in `tauri-action`
267293

268-
**Phase 7 — AI-era extensions** *(in progress — full plan in [`docs/phase-7-plan.md`](docs/phase-7-plan.md))*
294+
**Phase 7 — AI-era extensions** *(7a–7h shipped; FTS deferred to Phase 8 — full plan in [`docs/phase-7-plan.md`](docs/phase-7-plan.md))*
269295
- [x] **7a — `VECTOR(N)` column type** *(v0.1.10)*: dense f32 vectors with bracket-array literal syntax (`[0.1, 0.2, ...]`); file format bumped to v4
270296
- [x] **7b — Distance functions** *(v0.1.11)*: `vec_distance_l2/cosine/dot` + `ORDER BY <expr> LIMIT k` so KNN queries work end-to-end
271297
- [x] **7c — Bounded-heap top-k optimization** *(v0.1.12)*

docs/_index.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -36,28 +36,29 @@ These documents go into the implementation of each subsystem.
3636

3737
## Project state
3838

39-
As of April 2026, SQLRite has:
39+
As of May 2026, SQLRite has:
4040

4141
- A working SQL engine (in-memory + on-disk with a real B-Tree per table + secondary indexes, Phases 0 – 3 complete)
4242
- WAL-backed persistence with crash-safe checkpointing, shared/exclusive lock modes, and real `BEGIN` / `COMMIT` / `ROLLBACK` transactions (Phase 4 complete)
4343
- A stable public Rust embedding API plus C FFI shim and SDKs for Python, Node.js, Go, and WASM (Phase 5 complete except the optional 5f crate-polish task)
4444
- A Tauri 2.0 + Svelte desktop app (Phase 2.5 complete)
45-
- A fully-automated release pipeline that ships every product to its registry on every release with one human action — Rust crate to crates.io (`sqlrite-engine`), Python wheels to PyPI (`sqlrite`), Node.js + WASM to npm (`@joaoh82/sqlrite` + `@joaoh82/sqlrite-wasm`), Go module via `sdk/go/v*` git tag, plus C FFI tarballs and unsigned desktop installers as GitHub Release assets (Phase 6 complete)
45+
- AI-era extensions across the product surface (Phase 7 complete except FTS): VECTOR columns + HNSW indexes (7a-7d), JSON columns (7e), the `ask()` natural-language → SQL family across the REPL / desktop / Rust / Python / Node / Go / WASM (7g.1-7g.7), and the [`sqlrite-mcp`](mcp.md) Model Context Protocol server with seven tools including `ask` (7h + 7g.8)
46+
- A fully-automated release pipeline that ships every product to its registry on every release with one human action — Rust engine + `sqlrite-ask` + `sqlrite-mcp` to crates.io, Python wheels to PyPI (`sqlrite`), Node.js + WASM to npm (`@joaoh82/sqlrite` + `@joaoh82/sqlrite-wasm`), Go module via `sdk/go/v*` git tag, plus C FFI tarballs, MCP binary tarballs, and unsigned desktop installers as GitHub Release assets (Phase 6 complete)
4647

47-
The active frontier is **Phase 7AI-era extensions**.
48+
The active frontier is **Phase 8Full-text search + hybrid retrieval** (the deferred 7f scope).
4849

4950
See the [Roadmap](roadmap.md) for the full phase plan.
5051

5152
## Release engineering
5253

53-
- [Release plan](release-plan.md) — Phase 6 design doc: lockstep versioning, PR-based release flow, OIDC trusted publishing, the ten-file version-bump surface
54+
- [Release plan](release-plan.md) — Phase 6 design doc: lockstep versioning, PR-based release flow, OIDC trusted publishing, the version-bump surface
5455
- [Release secrets runbook](release-secrets.md) — one-time web-UI setup for crates.io, PyPI, npm, GitHub `release` environment, and `main` branch protection
5556
- [`scripts/`](../scripts/) — runnable tooling used by release workflows + reproducible locally (start with `scripts/bump-version.sh`)
5657

5758
## Future work
5859

59-
- [Phase 7 plan](phase-7-plan.md) — AI-era extensions (vector column type + HNSW, JSON, NL→SQL `ask()` API across REPL/library/SDKs/desktop/MCP, MCP server). **Approved 2026-04-26**, implementation starts at sub-phase 7a.
60-
- Phase 8 — Full-text search (FTS5-style BM25) + hybrid retrieval, deferred from Phase 7 per the plan-doc's Q1.
60+
- [Phase 7 plan](phase-7-plan.md) — AI-era extensions (vector column type + HNSW, JSON, NL→SQL `ask()` API across REPL/library/SDKs/desktop/MCP, MCP server). **Implementation complete except 7f, which deferred to Phase 8.**
61+
- Phase 8 — Full-text search (FTS5-style BM25) + hybrid retrieval, deferred from Phase 7 per the plan-doc's Q1. **Active frontier as of May 2026.**
6162

6263
## Conventions
6364

0 commit comments

Comments
 (0)