|
| 1 | +# `mithril-stm` benchmarks |
| 2 | + |
| 3 | +This folder holds the benchmark harnesses for `mithril-stm`. The two Halo2 **circuit** benchmark suites are |
| 4 | +the focus of this document: |
| 5 | + |
| 6 | +- the **recursive IVC circuit** — [`ivc_halo2_snark`](ivc_halo2_snark.rs), which ships a small **CLI** to |
| 7 | + list and select what runs; |
| 8 | +- the **non-recursive certificate circuit** — [`halo2_snark`](halo2_snark.rs) and |
| 9 | + [`halo2_prover_modes`](halo2_prover_modes.rs). |
| 10 | + |
| 11 | +Every harness is declared `harness = false` in `Cargo.toml` (a custom `main`, or a Criterion-generated |
| 12 | +`main`), and each delegates to a **benchmark-only façade** in the library — |
| 13 | +`circuits::halo2_ivc::bench::helpers` and `circuits::halo2::bench::helpers` — so the measured operations run |
| 14 | +the same production code paths. (The one exception is the IVC `verify/kzg_opening` diagnostic, which |
| 15 | +reproduces just the KZG-opening sub-step of `IvcProof::verify`.) The façades live in the library (not here) |
| 16 | +because they call `pub(crate)`/private production code; these harness files are the thin public-API |
| 17 | +front-ends. |
| 18 | + |
| 19 | +## Prerequisites |
| 20 | + |
| 21 | +- **Toolchain:** the crate's dev-dependencies require `rustc ≥ 1.88`. The commands below pin `+1.88.0`; use |
| 22 | + any installed toolchain `≥ 1.88`, or set it as the default and drop the `+1.88.0`. |
| 23 | +- **Features:** all circuit benches require `--features future_snark,benchmark-internals`. |
| 24 | +- **Resources:** the recursive circuit runs at degree 19 (GB-scale RAM, minutes per proof); the non-recursive |
| 25 | + `production` tier needs ≥ 70 GB RAM (server-class). Scope your run accordingly. |
| 26 | + |
| 27 | +General invocation: |
| 28 | + |
| 29 | +```bash |
| 30 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench <name> -- <args> |
| 31 | +``` |
| 32 | + |
| 33 | +Everything after `--` is passed to the benchmark binary. |
| 34 | + |
| 35 | +## Benchmark index |
| 36 | + |
| 37 | +| Bench | Circuit / area | What it measures | Selection | |
| 38 | +| ------------------------------------------------- | -------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------- | |
| 39 | +| `ivc_halo2_snark` | recursive IVC circuit | prove / verify / fold per transition path + setup (cold/warm), single observation | **custom CLI** (`--list`, literal id/prefix filter) | |
| 40 | +| `halo2_snark` | non-recursive certificate circuit | constraints, VK size, proof size, prove & verify time across parameter tiers | Criterion (filter `certificate/<tier>`) | |
| 41 | +| `halo2_prover_modes` | non-recursive certificate circuit | mock-prover vs real-prover cost projected to an e2e run, across `k` tiers | no arguments | |
| 42 | +| `multi_sig`, `schnorr_sig`, `stm`, `size_benches` | other crate areas (not Halo2 circuits) | see each file | — | |
| 43 | + |
| 44 | +--- |
| 45 | + |
| 46 | +## Recursive IVC circuit — `ivc_halo2_snark` |
| 47 | + |
| 48 | +Exercises the recursive IVC prover/verifier at its production degree (19), using the small committed |
| 49 | +certificate as the inner proof. It measures, for each of the three transition **paths** — `genesis`, |
| 50 | +`same_epoch`, `next_epoch`: |
| 51 | + |
| 52 | +- **prove** with each transcript: `prove/poseidon`, `prove/blake2b`; |
| 53 | +- **verify**: `verify/full` (message binding + KZG opening + folded-accumulator pairing) and |
| 54 | + `verify/kzg_opening` (the isolated opening); |
| 55 | +- **fold**: the off-circuit accumulator fold (`genesis` has none — it is a passthrough); |
| 56 | + |
| 57 | +plus **setup** measured cold vs warm: `setup/srs` and `setup/keys`. Each measurement is a single |
| 58 | +observation, printed as a small table when the run finishes. |
| 59 | + |
| 60 | +### The CLI |
| 61 | + |
| 62 | +Because a full run is expensive, this harness parses its own arguments (fail-closed, so a stray option can |
| 63 | +never silently trigger a multi-minute key generation). Arguments go after `--`: |
| 64 | + |
| 65 | +```bash |
| 66 | +# List every benchmark id — no benchmark setup or key generation (Cargo may still compile the target): |
| 67 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench ivc_halo2_snark -- --list |
| 68 | + |
| 69 | +# Show usage: |
| 70 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench ivc_halo2_snark -- --help |
| 71 | + |
| 72 | +# Run everything (tens of minutes; performs TWO recursive key generations — the shared per-path |
| 73 | +# environment and the cold setup/keys measurement — then all proofs): |
| 74 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench ivc_halo2_snark |
| 75 | + |
| 76 | +# Run a subset: pass ONE literal id or prefix (substring match against the ids from --list). |
| 77 | +# One transition path (prove + verify + fold): |
| 78 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench ivc_halo2_snark -- ivc/same_epoch |
| 79 | +# One path's verification only: |
| 80 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench ivc_halo2_snark -- ivc/genesis/verify |
| 81 | +# SRS cold vs warm (no key generation): |
| 82 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench ivc_halo2_snark -- ivc/setup/srs |
| 83 | +# Keys cold vs warm (cold performs a full recursive key generation): |
| 84 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench ivc_halo2_snark -- ivc/setup/keys |
| 85 | +``` |
| 86 | + |
| 87 | +The filter is a **literal** substring, not a regex. The parser rejects (rather than silently ignores): |
| 88 | +regex metacharacters in the filter, any value-taking option (`--sample-size`, `--color`, `--save-baseline`, |
| 89 | +…), and `--exact` / `--test` / `--profile-time` (the last two would force a recursive keygen). Valueless |
| 90 | +flags that `cargo bench` injects (`--bench`, `--nocapture`, `--quiet`, `--verbose`) are tolerated. |
| 91 | + |
| 92 | +Benchmark ids: |
| 93 | + |
| 94 | +``` |
| 95 | +ivc/{genesis,same_epoch,next_epoch}/prove/{poseidon,blake2b} |
| 96 | +ivc/{genesis,same_epoch,next_epoch}/verify/{full,kzg_opening} |
| 97 | +ivc/{same_epoch,next_epoch}/fold # genesis has no fold |
| 98 | +ivc/setup/{srs,keys} |
| 99 | +``` |
| 100 | + |
| 101 | +Filtering to a subset still builds the shared environment it needs (one recursive keygen), so isolated |
| 102 | +verify/fold runs pay that cost up front. |
| 103 | + |
| 104 | +--- |
| 105 | + |
| 106 | +## Non-recursive certificate circuit — `halo2_snark`, `halo2_prover_modes` |
| 107 | + |
| 108 | +### `halo2_snark` |
| 109 | + |
| 110 | +For each parameter **tier** it reports constraint rows vs `2^k`, advice columns, VK size, proof size, and |
| 111 | +proving/verification time. The two cheap tiers (`small`, `medium`) are **Criterion-sampled** (10 samples); |
| 112 | +the two expensive tiers (`large`, `production`) print a **single manually-timed observation** (10 Criterion |
| 113 | +samples would be prohibitive). |
| 114 | + |
| 115 | +**Always pass a `certificate/<tier>` filter.** The harness runs every tier that a positional filter does not |
| 116 | +exclude, so a bare invocation — or a run whose only arguments are Criterion control flags such as `--list`, |
| 117 | +which are not positional filters — executes **all four tiers, including `production` (≥ 70 GB RAM)**. |
| 118 | +Criterion's list/filter modes do **not** gate the manually-timed `large`/`production` tiers; only a |
| 119 | +positional `certificate/<tier>` filter does. |
| 120 | + |
| 121 | +```bash |
| 122 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench halo2_snark -- certificate/small |
| 123 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench halo2_snark -- certificate/medium |
| 124 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench halo2_snark -- certificate/large |
| 125 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench halo2_snark -- certificate/production |
| 126 | +``` |
| 127 | + |
| 128 | +| Tier | Quorum | `k` | Measurement | |
| 129 | +| ------------ | ------ | --- | --------------------- | |
| 130 | +| `small` | 3 | 13 | Criterion, 10 samples | |
| 131 | +| `medium` | 32 | 16 | Criterion, 10 samples | |
| 132 | +| `large` | 1024 | 21 | single observation | |
| 133 | +| `production` | 1944 | 22 | single observation | |
| 134 | + |
| 135 | +`small` is the lightest and a good smoke test after touching the circuit or the façade. `production` requires |
| 136 | +≥ 70 GB RAM (server only). |
| 137 | + |
| 138 | +### `halo2_prover_modes` |
| 139 | + |
| 140 | +Takes no arguments — it sweeps a range of `k` tiers and prints, for each, the mock-prover vs real-prover |
| 141 | +timings projected onto a standard e2e run (~80 certificates): |
| 142 | + |
| 143 | +```bash |
| 144 | +cargo +1.88.0 bench -p mithril-stm --features future_snark,benchmark-internals --bench halo2_prover_modes |
| 145 | +``` |
0 commit comments