feat(bench): publication — first pinned-host run + canonical docs (sub-phase 9.6)

[joaoh82]

Summary

Sub-phase 9.6 of docs/benchmarks-plan.md — closes out the benchmark suite (SQLR-4 / SQLR-16). Builds on 9.1–9.5 (#102 / #103 / #104 / #105 / #106).

Three deliverables:

1. benchmarks/scripts/compare.py — pure-stdlib Python diff renderer. Reads two JSON envelopes, matches samples by (workload, driver), prints a Markdown table with deltas + ratios. Same-version-only per Q8; cross-host pairs flagged.
2. First "official" pinned-host run committed at benchmarks/results/2026-05-07-apple-9ffd55a5.json — 35 samples across all 12 workloads × {SQLRite, SQLite-WAL+NORMAL, DuckDB on Group B}. M1 Pro, macOS 23.5.0, criterion defaults.
3. docs/benchmarks.md — canonical user-facing reference. Mirrors how docs/fts.md is the canonical FTS reference for Phase 8.

Headline numbers (from the official run)

Workload	SQLRite	SQLite (WAL+NORMAL)	DuckDB
W1 read-by-PK	9.87 µs	2.05 µs	—
W2 range-100	23.99 ms	60.50 µs	—
W2 range-1k	24.92 ms	585.21 µs	—
W2 range-10k	30.15 ms	6.24 ms	—
W3 bulk insert	1.029 s	166.43 ms	—
W4 single-row insert	6.76 ms	9.78 µs	—
W5 mixed OLTP	55.63 ms	9.96 µs	—
W6 index lookup	10.45 µs	2.50 µs	—
W7 SUM (1M rows)	109.47 ms	31.14 ms	468.74 µs
W8 GROUP BY card-10	201.80 ms	438.09 ms	761.40 µs
W8 GROUP BY card-1k	1.372 s	251.13 ms	871.80 µs
W8 GROUP BY card-100k	skipped	238.96 ms	19.58 ms
W9 INNER JOIN (10k×10k)	34.25 s	2.23 µs	699.23 µs
W10 vector top-10 (brute-force)	138.66 ms	—	—
W10 vector top-10 (HNSW)	126.81 ms	—	—
W11 BM25 top-10 (1k docs)	1.079 ms	25.03 µs	—
W12 hybrid (1k docs)	713.53 µs	—	—

Note on dirty=true in the JSON: working-tree state at run time was dirty (this PR's docs were uncommitted). The measurements themselves only depend on the bench binary built from the committed bench-9.5-duckdb tip; future official runs will land with dirty=false.

Top-level README + docs/_index.md

• README.md gains a "Benchmarks" section pointing at docs/benchmarks.md.
• docs/_index.md promotes benchmarks.md to a top-level reference (alongside fts.md), and demotes benchmarks-plan.md to a "design rationale, historical" pointer.
• Plan-doc status banner updated to "all six sub-phases shipped."

benchmarks/results/.gitignore rewrite

The old rule was strict "ignore everything until 9.6." Now it documents the convention ("only pinned-host runs get committed; PR review enforces it") and allows *.json to be added explicitly. Local runs still show up as untracked in git status, which is the prompt to either commit them as official or leave them out.

Test plan

• cargo fmt --all -- --check clean
• cargo clippy -p sqlrite-benchmarks --all-targets clean (default features)
• Full CI-equivalent: cargo build / test / doc --workspace --exclude {desktop,python,nodejs,benchmarks} green
• cargo test -p sqlrite-benchmarks — 5 tests pass
• compare.py round-trips a synthetic envelope pair into a Markdown diff
• make bench-duckdb ran end-to-end: 35 samples, all 12 workloads × all eligible drivers

What's left

The plan's exit criterion is met: docs/benchmarks.md exists, the README has a "Benchmarks" section pointing at it, the first dated results JSON is committed.

Follow-ups still open (separately tracked, all "informational, not a release gate" per the plan):

• SQLR-21 — Phase 8.1 FTS overflow chaining (W11/W12 corpus cap)
• SQLR-20 — W9 JOIN inner-side index pushdown
• SQLR-19 — W8 GROUP BY high-cardinality blowup
• SQLR-18 — W4 single-row INSERT gap
• SQLR-17 — desktop-build CI hang (only act if recurs)

Plan post-9.6 ideas still parked:

• libSQL driver (when a vector-only benchmark page wants a non-extension competitor)
• Per-PR regression detector (needs ~3 baseline runs first)
• Concurrency workloads (gated on multi-writer support)
• Larger datasets (10M / 100M scale)

References

• Parent: SQLR-4 (design), SQLR-16 (execution)
• Plan: docs/benchmarks-plan.md
• Canonical reference: docs/benchmarks.md
• Compare script: benchmarks/scripts/compare.py
• First run: benchmarks/results/2026-05-07-apple-9ffd55a5.json

🤖 Generated with Claude Code