feat(bench): harness scaffolding + W1 read-by-PK (sub-phase 9.1)

[joaoh82]

Summary

First executable code for the SQLRite-vs-SQLite benchmark suite designed in #101 (SQLR-4). Sub-phase 9.1 of the plan.

• Q1–Q8 resolved. Plan-doc updated to mirror Phase 7's "Decisions (was: open questions)" shape — each question keeps its original options + a > **Decided: …** blockquote at the top. See docs/benchmarks-plan.md.
• benchmarks/ workspace member with a small Driver trait, two engine impls (SQLRite via Connection, SQLite via rusqlite-bundled), one workload (W1 read-by-PK), a criterion entry point, and an aggregator that walks target/criterion/**/estimates.json into a single results envelope under benchmarks/results/.
• CI excludes sqlrite-benchmarks alongside sqlrite-desktop + the SDK cdylibs; runs locally via make bench.

Plan-doc decisions (Q1–Q8)

#	Decision
Q1	Pinned local M-series MBP for v1; bare-metal post-9.6
Q2	No PR-gating; needs ~3 baselines first
Q3	Headline = SQLite WAL+synchronous=NORMAL+64 MB cache; default-mode is opt-in secondary
Q4	DuckDB opt-in (--features duckdb); Group B only
Q5	libSQL deferred to post-9.6
Q6	D1 / rqlite out of scope
Q7	In-repo: docs/benchmarks.md + raw JSON in benchmarks/results/
Q8	Workloads carry an explicit version (W1.v1, W1.v2, …)

Harness shape

• Driver trait is small enough to express every planned workload (open / execute / execute_with_params / query_one / query_all). &self receiver so drivers are cheap to clone into criterion closures.
• SQLRite driver inlines ? params via SQL formatting — the engine has no parameter binding yet; the inliner is unit-tested for arity, escaping, and ?-in-strings. Once the engine grows real binds, this driver flips to the bound path and a workload v bump captures the methodology change.
• SQLite driver runs the Q3-tuned profile at open time (WAL + synchronous=NORMAL + 64 MB cache + temp_store=MEMORY) and prepare_cached on every hot-path SELECT — so we measure execution cost, not per-iter parse cost.
• Per-run TempDir — no page-cache / WAL bleed across iterations.
• Correctness gate — every workload runs correctness_check against sample keys before timing, so a divergent-result "speedup" can't silently land.
• Q8 versioning baked in — workloads carry WorkloadId { id, name, version } and the JSON envelope tags every sample with W{n}.v{v}.

JSON output (locked in this PR)

BenchSample per (workload, driver) pair: median + 95% CI + mean + std dev + sample count + ops/s.

ResultsEnvelope carries schema_version, run_started_at (RFC 3339), run_duration_secs, host fingerprint (CPU brand, RAM, OS kind + release, arch), and commit info (sha, branch, dirty bit). aggregate.rs reads sysctl / /proc / git directly — no chrono / sysinfo deps.

benchmarks/results/.gitignore keeps casual local runs out of git until 9.6 commits the first official pinned-host run.

End-to-end smoke run (illustrative — not the official 9.6 publication)

W1.v1/sqlrite/default   ~12 µs/op   ~84k ops/s
W1.v1/sqlite/default    ~2.5 µs/op  ~395k ops/s

About a 5× gap on the read-by-PK hot path, driven by SQLRite re-parsing the SQL on every iteration. That's the baseline a future "prepared-statement support" follow-up needs to move.

Test plan

• cargo fmt --all -- --check
• cargo clippy -p sqlrite-benchmarks --all-targets clean (existing engine warnings unchanged from baseline)
• Full CI-equivalent: cargo build / test / doc --workspace --exclude {desktop,python,nodejs,benchmarks} green
• cargo test -p sqlrite-benchmarks — 5 tests pass (4 inliner + 1 ymd-from-days round-trip)
• cargo bench -p sqlrite-benchmarks --bench suite runs end-to-end; aggregator wrote a 2-sample JSON envelope
• make bench target wired through scripts/run.sh
• CI excludes verified — --exclude sqlrite-benchmarks added to all four cargo commands in .github/workflows/ci.yml

Follow-ups

• 9.2 (Group A) — W2–W6. W4 (single-row INSERT) is the one most likely to expose the bottom-up-rebuild commit cost.
• Prepared-statement support in SQLRite — closing the per-iter parse gap is post-9.6, behind a W1.v2 bump.
• 9.5 — DuckDB driver — Cargo.toml feature + commented module placeholder ready to land.
• 9.6 — docs/benchmarks.md + first official pinned-host JSON committed + top-level README "Benchmarks" section.

References

• Plan: docs/benchmarks-plan.md
• Plan PR: docs(bench): design plan for SQLRite vs SQLite benchmark suite (SQLR-4) #101
• Roadmap: docs/roadmap.md "Possible extras"

🤖 Generated with Claude Code