Skip to content

feat(bb.js): paired A/B Chonk bench mode in browser-test-app#23472

Draft
AztecBot wants to merge 2 commits into
merge-train/barretenbergfrom
cb/122192aeee90
Draft

feat(bb.js): paired A/B Chonk bench mode in browser-test-app#23472
AztecBot wants to merge 2 commits into
merge-train/barretenbergfrom
cb/122192aeee90

Conversation

@AztecBot
Copy link
Copy Markdown
Collaborator

Adds a paired A/B benchmarking mode to the existing bb.js browser-test-app so PR-vs-base Chonk prove comparisons on real BrowserStack hardware produce statistically defensible numbers, reusing the harness we already trust instead of a separate minimal rig.

What it does

window.runChonkAb(options) (auto-runs on a base64url ?bench= param) runs a Chonk prove repeatedly, interleaving two bb.wasm variants on the same device in one session (alternating order per pair to remove first-mover bias), then reports the paired median Δ with a bootstrap 95% CI. Variant selection is at runtime via bb.js's existing wasmPath option — both binaries are hosted side by side, no rebuild and no second bundle:

bench-wasm/{base,pr}/barretenberg{,-threads}.wasm.gz

memMaxPages maps to Barretenberg.new({ memory: { maximum } }) (set 16384 = 1 GiB on iOS Safari, which rejects the default 4 GiB SAB).

Why

Between-session variance on real devices (different physical unit, thermal state, co-tenant) dwarfs single-digit prover deltas. Pairing both variants back-to-back on one device, N≥10 times, and reporting a CI is what distinguishes a real 3–6% delta from noise — or honestly reports that it cannot be distinguished. See BENCH.md.

Files

  • src/index.tswasmPath/memory passthrough on proveChonk; new installChonkAbGlobal() (interleave loop, bootstrap-CI stats, /progress + /result POST, window.__benchResult).
  • scripts/serve-bench.mjs — node server hosting dest/, /wasm/<variant>/, /inputs/<flow>/ivc-inputs.msgpack, /progress + /result sinks, with COOP/COEP for SAB. bb.js fetches CRS from the public CDN, so no CRS proxy.
  • package.jsonserve-bench script.
  • BENCH.md — usage, the reporting rule (no signed delta when the CI crosses zero), and the iOS two-session workaround.

Verification status — DRAFT, not yet validated

  • TypeScript transpiles cleanly (esbuild). Not yet full-built/type-checked against the bb.js portal types, and not yet BrowserStack smoke-run on this branch.
  • Before this is marked ready: a full yarn build of the app + a paired-A/B smoke run on at least one desktop and one mobile BS target, confirming verified proofs and a populated paired block. I can run that as the next step.

Methodology background: https://gist.github.com/AztecBot/0483ec5b0293ff9e31621ad035854eb0 · protocol: https://gist.github.com/AztecBot/a8d3e5c14475c8fb5af520706e408222

This consolidates browser benching onto bb.js and supersedes the unmerged codex/browserstack-wasm-bench minimal harness (and barretenberg-claude#3799).


Created by claudebox · group: slackbot

@AztecBot AztecBot added ci-barretenberg Run all barretenberg/cpp checks. claudebox Owned by claudebox. it can push to this PR. labels May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-barretenberg Run all barretenberg/cpp checks. claudebox Owned by claudebox. it can push to this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant