feat(library-selection): #215 add resolve_warm criterion bench (P-01 mini)

[zackees]

Summary

• Adds crates/fbuild-library-select/benches/resolve_warm.rs, a Criterion bench mirroring resolve_cold.rs that builds the same ~30-library MiniFramework fixture, primes a KvStore with one untimed resolve_cached call, and times only the cache-hit path on the second invocation. Each iteration asserts from_cache == true so a silent regression to miss work can never be reported as warm.
• Wires the bench into the .github/workflows/bench-205.yml matrix as a third lane (target P-01-mini (warm cache against MiniFramework, baseline TBD)). No CI failure gate yet — the threshold gate lands in a follow-up once we have a stable baseline number.
• Updates crates/fbuild-library-select/benches/README.md, bench/README.md, and bench/fastled-examples/README.md to reflect that Phase 4 K/V memoization (PR feat(library-selection): #205 Phase 4 zccache K/V memoization #212) shipped and the synthetic warm baseline now exists. The real per-board, per-example FastLED matrix remains future work tracked under bench/fastled-examples/.

Local baseline numbers

Windows host, --release Criterion, same MiniFramework (30 libs, chain depth 5):

bench	time
resolve_cold	~5.0 ms
resolve_warm	~1.7 ms

Warm is ~3x faster than cold; the gap is bounded below by the cache-key computation itself, which still hashes seed bodies and canonical library headers (per the cache_key design in cache.rs). That is by design — the cache key has to reflect on-disk content to be sound. Real warm-path wins on a Teensyduino-class tree should be larger because the cold side scales with the LDF reconciliation cost, while the warm side stays near constant.

Test plan

• uv run soldr cargo build --benches -p fbuild-library-select (clean)
• uv run soldr cargo bench -p fbuild-library-select --bench resolve_warm (completed; baseline above)
• uv run soldr cargo bench -p fbuild-library-select --bench resolve_cold (re-run for comparison)
• uv run soldr cargo clippy --workspace --all-targets -- -D warnings (clean)
• uv run soldr cargo fmt --all -- --check (clean)

Out of scope

• Wiring orchestrators to call resolve_cached (tracked under #205 follow-up: wire library-selection cache into orchestrators (teensy, stm32) #214, separate PR by another agent).
• Real FastLED warm matrix in bench/fastled-examples/ (still gated on a checked-out FastLED tree + orchestrator wiring).
• A CI failure gate on the warm number — intentionally deferred until we have a stable baseline.

Closes #215
Refs #205

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

Summary by CodeRabbit

• Tests

  • Introduced a new warm-cache performance benchmark to measure library resolution speed when cache hits occur. The benchmark validates cache behavior to ensure accurate warm-path performance measurements throughout iterations.

• Documentation

  • Updated benchmark documentation and workflow configuration with comprehensive instructions for running the new warm-cache benchmarks alongside existing cold-path tests. Clarified behavior and measurement approaches for both benchmark types.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 653d7c9d-9da2-4ada-89a5-d4dc96763a8a

📥 Commits

Reviewing files that changed from the base of the PR and between eb4e6de and 82348bf.

📒 Files selected for processing (6)

• .github/workflows/bench-205.yml
• bench/README.md
• bench/fastled-examples/README.md
• crates/fbuild-library-select/Cargo.toml
• crates/fbuild-library-select/benches/README.md
• crates/fbuild-library-select/benches/resolve_warm.rs

---

📝 Walkthrough

Walkthrough

This PR adds a Criterion benchmark that measures the warm cache-hit path for the library resolution algorithm. It builds on the resolve_cached() API introduced in PR #212, creating a companion benchmark to the existing resolve_cold bench using the same MiniFramework fixture but with a pre-populated KvStore. The benchmark is registered in Cargo.toml, documented extensively across multiple README files, and integrated into the CI workflow.

Changes

Warm-cache resolve benchmark

Layer / File(s)	Summary
Benchmark Implementation crates/fbuild-library-select/benches/resolve_warm.rs	New Criterion benchmark with fixture construction (MiniFramework with 30-library chain), cache priming via untimed resolve_cached() miss, validation of cache hit, and a timed loop that asserts every iteration returns a cache hit.
Benchmark Registration crates/fbuild-library-select/Cargo.toml	New [[bench]] target resolve_warm with harness = false added to the package configuration.
Per-Crate Documentation crates/fbuild-library-select/benches/README.md	Clarifies resolve_cold invokes resolve() directly without cache; adds new resolve_warm section describing the warm-cache-hit flow with fixture building, KvStore setup, cache priming, and per-iteration assertions.
Top-Level Documentation bench/README.md, bench/fastled-examples/README.md	Root README adds --bench resolve_warm command; FastLED examples README rewords placeholder to explain synthetic warm baseline exists while the real multi-board harness is reserved; "running today" section lists both cold and warm synthetic benches against MiniFramework.
CI Workflow Integration .github/workflows/bench-205.yml	Adds fbuild-library-select / resolve_warm entry to the job matrix with target threshold P-01-mini (warm cache against MiniFramework, baseline TBD).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related issues

• #205 follow-up: Phase 7 warm-cache resolve bench (resolve_warm) #215: This PR directly implements the Phase 7 warm-cache resolve benchmark scope—adding resolve_warm.rs, Cargo.toml registration, documentation, and CI workflow integration as described in issue #215.

Possibly related PRs

• FastLED/fbuild#212: The new resolve_warm benchmark directly exercises the resolve_cached()/KvStore API introduced in PR #212, providing measurement infrastructure for the warm cache-hit path.

Poem

🐰 A warm cache bench hops into view,
Measuring hits when resolve_cached's through—
No cold-path tread, just cache-kissed speed,
P-01 baseline, a metrics seed!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: adding a resolve_warm Criterion benchmark for the fbuild-library-select crate as specified in issue #215.
Linked Issues check	✅ Passed	All primary objectives from #215 are met: resolve_warm.rs benchmark added with MiniFramework fixture, KvStore cache-hit path tested, Cargo.toml entry created, workflow integration added, and documentation updated.
Out of Scope Changes check	✅ Passed	All changes directly support the stated objectives; no out-of-scope modifications detected. Documentation updates appropriately clarify scope boundaries (bench/fastled-examples remains future work).
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/215-resolve-warm-bench

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}