@@ -32,34 +32,67 @@ Compares proving time for an identical u64 wrapping Fibonacci computation.
3232# Approximate workload steps (converted with 5 steps/iteration)
3333./bench_vs/run.sh --steps 1000000 2000000 4000000 8000000
3434
35+ # Project to a target cycle count
36+ ./bench_vs/run.sh --target-cycles 500000000
37+
3538# Run only one prover
3639./bench_vs/run.sh --lambda-only
3740./bench_vs/run.sh --sp1-only
3841```
3942
40- ## What it measures
43+ ## What is measured
4144
4245Both provers execute the same program: iterative Fibonacci with ` u64::wrapping_add ` .
43- Only ** proving time** is compared (wall-clock, no recursion/compression on either side).
44-
45- - ** Lambda VM** : Generates RISC-V assembly at runtime, assembles to ELF, proves via the CLI.
46- - ** SP1 v6** : Compiles a Rust guest program to RISC-V, proves via ` sp1-sdk ` core mode.
4746
48- The linear projection uses a common axis for both provers: target workload steps.
49- When you pass ` --steps ` , that target is explicit. When you pass ` -n ` , the script
50- approximates workload as ` steps ~= 5 * n ` . ` SP1 cycles ` are still reported, but
51- only as telemetry and not as the regression axis.
47+ The timing window on both sides is ** end-to-end single-shot proving, with no
48+ verification and no recursion/compression** . Concretely:
49+
50+ | Phase | Lambda VM timer | SP1 v6 timer |
51+ | --------------------------------------------| :---------------:| :------------:|
52+ | Read ELF + input from disk | ❌ | ❌ |
53+ | Pre-pass execution to count cycles | ❌ | ❌ |
54+ | ` setup ` / verifying-key derivation | N/A (none) | ✅ |
55+ | ELF parse + guest execution (inside prove) | ✅ | ✅ |
56+ | Trace build | ✅ | ✅ |
57+ | AIR construction | ✅ | ✅ |
58+ | STARK prove (` core ` mode) | ✅ | ✅ |
59+ | Proof serialization / write | ❌ | ❌ |
60+ | Verify | ❌ | ❌ |
61+
62+ Both sides run one extra execution pass ** outside** the timer to report dynamic
63+ instruction counts (SP1's ` execute(...) ` / Lambda's executor pre-pass). This
64+ costs wall-clock time in the CI job but does not inflate the measured proving
65+ time, and the cost is symmetric between the two provers.
66+
67+ Lambda VM uses the default proof options from ` prover::prove_with_inputs `
68+ (` GoldilocksCubicProofOptions::with_blowup(2) ` , 50 FRI queries). SP1 v6 uses
69+ the ` core ` proof mode exposed by ` sp1-sdk::ProverClient::from_env() ` .
70+
71+ ## Projection axis
72+
73+ The linear projection uses ** measured cycles** per prover — Lambda's executor
74+ log count and SP1's ` report.total_instruction_count() ` . For Fibonacci the two
75+ values agree to within ~ 1% (both compile to the same inner loop shape on
76+ RISC-V). When cycle data is missing, the script falls back to the approximate
77+ ` target_workload_steps ~= 5 * n ` label that was passed on the command line.
5278
5379## Output
5480
5581```
5682=== Summary ===
5783Program: Fibonacci (u64 wrapping)
5884
59- Target steps Iterations Lambda VM SP1 v6 SP1 cycles Ratio
60- ------------ ---------- --------- ------ ---------- -----
61- 1000000 200000 ...s ...s 1004794 ...
62- 2000000 400000 ...s ...s 2004794 ...
85+ Target steps Iterations Lambda (s) Lambda cycles SP1 (s) SP1 cycles Ratio
86+ ------------ ---------- ---------- ------------- ------- ---------- -----
87+ 1000000 200000 ...s 1004794 ...s 1004794 ...
88+ 2000000 400000 ...s 2004794 ...s 2004794 ...
6389
90+ Timing window covers single-shot end-to-end proving; SP1 includes setup; both exclude verification.
6491Green ratio = Lambda VM faster, Red = SP1 faster
6592```
93+
94+ With ` --report-dir DIR ` the script writes:
95+ - ` results.tsv ` — raw per-run data (` target_steps ` , ` iterations ` , ` lambda_time_s ` , ` lambda_axis_value ` , ` lambda_cycles ` , ` sp1_time_s ` , ` sp1_axis_value ` , ` sp1_cycles ` , ` ratio ` ).
96+ - ` metrics.txt ` — key=value pairs including ` timing_window=setup_plus_end_to_end_prove_no_verify ` .
97+ - ` summary.md ` — the same table plus linear projection to ` TARGET_CYCLES ` cycles.
98+ - ` raw/ ` — stdout/stderr of every individual run.
0 commit comments