Skip to content

benchmark-execution.md

Latest commit

 

History

History
160 lines (113 loc) · 5.12 KB

benchmark-execution.md

File metadata and controls

160 lines (113 loc) · 5.12 KB
Raw

Benchmark Execution

This is the source-of-truth workflow guide for ere-hosts.

Overview

ere-hosts consumes fixtures, resolves guest binaries, runs benchmarks across selected zkVMs, and optionally persists or verifies proofs.

Inspect the current CLI surface from the repo root:

cargo run -p ere-hosts -- --help

Prerequisites:

  • Docker is required because zkVM hosts are managed through ere-dockerized.
  • Fixture JSON files are expected in zkevm-fixtures-input/ unless --input-folder is provided.
  • stateless-validator accepts both generated repo fixtures and EEST blockchain_tests fixtures that contain statelessInputBytes and statelessOutputBytes.

Common Benchmark Commands

Run the stateless validator with Reth:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    stateless-validator --execution-client reth

Run the stateless validator with Ethrex:

cargo run -p ere-hosts --release -- --zkvms risc0 \
    stateless-validator --execution-client ethrex

Run the empty program:

cargo run -p ere-hosts --release -- --zkvms sp1 empty-program

Use a custom fixture folder:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    stateless-validator --execution-client reth \
    --input-folder my-fixtures

Run directly from an EEST fixture checkout:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    stateless-validator --execution-client reth \
    --input-folder /data/code-data/execution-specs/fixtures

When an input folder contains an EEST blockchain_tests/ subdirectory, only that subtree is used for stateless-validator inputs.

Filter the selected fixtures by prefix:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    stateless-validator --execution-client reth \
    --fixture test_sha256.py::test_sha256 \
    --fixture test_memory.py::test_mcopy

Action Model

ere-hosts supports three actions:

  • --action execute: execute the guest only. This is the default.
  • --action prove: execute and generate a proof.
  • --action verify: verify proofs loaded from disk or downloaded from a .tar.gz archive.

Timeouts are action-scoped:

  • Default execute timeout: 5m
  • Default prove timeout: 15m
  • Default verify timeout: 2s

Override the timeout for the selected action only:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    --timeout 90s \
    empty-program

Inputs And Outputs

  • Fixture input folder default: zkevm-fixtures-input/
  • Metrics output folder default: zkevm-metrics/
  • Verification proof folder default: zkevm-fixtures-proofs/
  • Zisk profile output folder default: zisk-profiles/

Use the focused references when you need exact schemas or file layouts:

Dump the raw serialized guest inputs used for a run:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    --dump-inputs debug-inputs \
    stateless-validator --execution-client reth

These input dumps are zkVM-independent, so each fixture input is written once even if multiple zkVMs are selected.

Proof Persistence And Verification

Generate and save proofs:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    --action prove \
    --save-proofs my-proofs \
    stateless-validator --execution-client reth

Verify proofs from a local folder:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    --action verify \
    --proofs-folder my-proofs \
    stateless-validator --execution-client reth

Verify proofs from a remote archive:

cargo run -p ere-hosts --release -- --zkvms sp1 \
    --action verify \
    --proofs-url https://example.com/proofs.tar.gz \
    stateless-validator --execution-client reth

When --proofs-url is used, the archive is downloaded, extracted to a temporary directory, and cleaned up after verification.

Operational Notes

  • Guest binaries are downloaded automatically unless --bin-path is set. Tagged ere-guests dependencies use release assets for that tag; commit or branch dependencies use GitHub Actions artifacts for the resolved commit and require GITHUB_TOKEN or GH_TOKEN.
  • Use --force-rerun to ignore existing output and rerun a workload.
  • --resource gpu selects GPU proving resources where supported.
  • --zisk-profile only works with --zkvms zisk and --action execute.

Common Failure Modes

  • cargo run from the repo root must include -p ere-hosts; the workspace has multiple binaries.
  • Docker availability issues will block benchmark execution.
  • --save-proofs is only valid with --action prove.
  • --proofs-url is only valid with --action verify.
  • Direct EEST fixture runs may produce normal benchmark crash reports until the selected EL guest binary can decode canonical EEST stateless input bytes.
  • Local offline failures may come from guest downloads, RPC access, or proof archive downloads rather than from a compile regression.