feat(bench): Group C differentiators — W10..W12 (sub-phase 9.4)

[joaoh82]

Summary

Sub-phase 9.4 of docs/benchmarks-plan.md. Builds on #102 / #103 / #104. Lands the three Phase 7 / Phase 8 differentiator workloads:

ID	Shape	Status
W10	Vector top-10 (cosine), brute-force + HNSW variants	✅ SQLRite-only
W11	BM25 top-10 — SQLRite USING fts vs SQLite FTS5 virtual table	✅
W12	Hybrid retrieval (BM25 + cosine fusion, mirrors examples/hybrid-retrieval/)	✅ SQLRite-only

Smoke-run numbers (M1 Pro, criterion --warm-up-time 1 --measurement-time 1 --sample-size 10)

Workload	SQLRite	SQLite	Ratio
W10 brute-force (10k×384-d)	~122 ms	—	parser cost dominates
W10 hnsw (10k×384-d)	~132 ms	—	same band as brute-force
W11 BM25 top-10 (1k docs)	~533 µs	~24 µs	~22× (healthy band)
W12 hybrid (1k docs)	~654 µs	—	RAG headline number

W11's ~22× is one of the smaller gaps in the suite — SQLRite's BM25 path sits in the same per-iter-parse-dominated band as W1 / W6, not the per-row-commit-dominated W4 / W5.

W10's HNSW vs brute-force is essentially flat at 10k vectors — both paths are dominated by the per-iter ~4 KB SQL parse cost (the 384-dim bracket-array literal in ORDER BY). HNSW would shine at much larger corpus sizes; at 10k the parse cost masks the algorithmic win. A future "vector-bind" or "prepared-vector-query" path would surface the real gap.

Two plan deviations, documented + tracked

W11 / W12: 10k docs → 1000 docs

Engine constraint — SQLR-21. SQLRite's FTS index serializes a per-doc-length sidecar as a single cell. That cell must fit inside one 4 KiB page (overflow chaining is Phase 8.1, not yet shipped). At ~3 bytes per doc, the practical cap is ~1,360 docs. 10k docs crashed setup with:

FTS posting cell 1 of 31754 bytes exceeds empty-page capacity 4085
(term too long or too many postings; overflow chaining is Phase 8.1)

v1 ships at 1000 docs with margin. Bumping back to 10k follows SQLR-21 + a W11.v2 / W12.v2 tag.

W10 sqlite-vec comparator deferred

Plan said "sqlite-vec if installable, else SQLRite-only baseline." rusqlite[bundled] doesn't ship the extension; loading a pre-compiled .dylib at runtime would need extra plumbing. Out of scope for v1 — both W10 variants ship as SQLRite-only. Adding sqlite-vec is a future follow-up alongside libSQL (post-9.6 vector benchmark page).

Notable engineering finds

1. W11 SQLite FTS5 default is AND, not OR — needed an explicit OR-join in fts.rs::bench_iter to match SQLRite's fts_match any-of semantics. Without it, multi-term queries against the synthetic dictionary returned 0 hits.
2. FTS dictionary engineering — used a 10k synthetic word dictionary (down from a naive 32-word version that produced 30 KB posting lists per term). Posting-list size now well under the page cap.
3. W10 HNSW barely beats brute-force at 10k scale — informational; the parser cost per-iter dominates the algorithmic difference. Documented as the methodology note in the README.

Test plan

• cargo fmt --all -- --check clean
• cargo clippy -p sqlrite-benchmarks --all-targets clean
• Full CI-equivalent: cargo build / test / doc --workspace --exclude {desktop,python,nodejs,benchmarks} green
• cargo test -p sqlrite-benchmarks — 5 tests pass
• Smoke cargo bench -p sqlrite-benchmarks --bench suite -- '^W1[012]' runs all three workloads end-to-end
• W10 / W11 / W12 correctness gates pass on both drivers (engine-asymmetric W11 setup + the OR-join fix on the SQLite path)

Follow-ups

• SQLR-21 — Phase 8.1 FTS overflow chaining for postings + doc_lengths sidecar. Unblocks bumping W11 / W12 corpus back to 10k.
• sqlite-vec driver wiring — adding the extension load path on the rusqlite driver. Lets W10 grow a SQLite comparator column.
• Prepared-vector-query / VECTOR bind support — would close the W10 brute-force gap by removing the per-iter 4 KB SQL parse. Same root cause as the W1 prepared-statement follow-up (post-9.6).
• Carried open: SQLR-18 (W4), SQLR-19 (W8 high-cardinality), SQLR-20 (W9 join planner).

What's left

9.5 — DuckDB driver, optional, Group B only. Cargo feature wiring + the make bench-duckdb target.
9.6 — scripts/compare.py + first official pinned-host JSON committed + docs/benchmarks.md (canonical user-facing reference) + top-level README.md "Benchmarks" section.

References

• Parent task: SQLR-16
• 9.1: #102, 9.2: #103, 9.3: #104
• Plan: docs/benchmarks-plan.md
• Workloads: benchmarks/src/workloads/{vector,fts,hybrid}.rs

🤖 Generated with Claude Code