docs(bench): design plan for SQLRite vs SQLite benchmark suite (SQLR-4)

[joaoh82]

Summary

• Drafts docs/benchmarks-plan.md — design proposal for the long-standing "Benchmarks against SQLite" item from the Possible extras list. Mirrors the shape of phase-7-plan.md / phase-8-plan.md: goals, scope philosophy, viability call per comparator, workloads table, methodology, repo layout, sub-phases, open questions, risks.
• Wires it into docs/_index.md "Future work" and replaces the bare line in roadmap.md "Possible extras" with a richer entry that points at the plan + records the viability outcomes.
• Docs-only — no engine code.

Comparison-target viability (the call)

• ✅ SQLite — mandatory baseline via rusqlite; tuned to journal_mode=WAL + synchronous=NORMAL for apples-to-apples vs SQLRite's mandatory WAL.
• ✅ DuckDB — analytical-slice only (Group B workloads), opt-in via --features duckdb. Different storage model (columnar OLAP); included to ground the "we're not DuckDB" positioning with measurement.
• 🟡 libSQL (Turso embedded) — defer. Embedded SQL path tracks SQLite within a few percent; revisit when we want a non-extension competitor for the vector workload (W10).
• ❌ Cloudflare D1 — out-of-scope. Network-resident; benchmarks would measure HTTP, not the engine.
• ❌ rqlite — out-of-scope. Distributed (Raft over HTTP); same network-dominates-everything problem.

Workload shape

10 curated workloads in three groups:

• Group A — OLTP baseline (W1 read-by-PK, W2 range scan, W3 bulk insert, W4 single-row insert, W5 mixed OLTP, W6 secondary-index lookup)
• Group B — SQL-feature scaling (W7 aggregate, W8 GROUP BY, W9 INNER JOIN — exercises SQLR-3 + SQLR-5 surface)
• Group C — Differentiators (W10 vector top-10, W11 BM25 top-10, W12 hybrid retrieval — proves the Phase 7d / 8 / 8d work has measured value)

Sub-phases

Six independently-shippable PRs, ~1.5 kLOC total:

• 9.1 Harness scaffolding + Driver trait + W1 end-to-end
• 9.2 Group A workloads (W2–W6)
• 9.3 Group B workloads (W7–W9)
• 9.4 Group C differentiators (W10–W12)
• 9.5 DuckDB driver (optional, feature-gated)
• 9.6 Reporting (compare.py, first official run, docs/benchmarks.md canonical reference)

Open questions

8 design questions in the doc (Q1 host pinning · Q2 SLO gates · Q3 SQLite tuning profile · Q4 DuckDB gating · Q5 libSQL timing · Q6 D1/rqlite confirmation · Q7 publishing surface · Q8 workload versioning). Leans included for each. Implementation kicks off after Q-resolution.

Test plan

• All referenced relative-link targets in the new doc exist on disk
• No engine code touched → existing cargo build / cargo test impact is nil
• Plan reviewed; Q1–Q8 answered before sub-phase 9.1 starts (separate Marvin task)

🤖 Generated with Claude Code