Skip to content

fixture-generation.md

Latest commit

 

History

History
148 lines (102 loc) · 5.16 KB

fixture-generation.md

File metadata and controls

148 lines (102 loc) · 5.16 KB
Raw

Fixture Generation

This guide explains when fixture generation is needed and how to produce fixtures for ere-hosts. For Docker usage and binary-local behavior, see Witness Generator CLI notes.

Overview

ere-hosts stateless-validator accepts two fixture formats:

  • Direct EEST blockchain_tests fixtures: EEST JSON files whose executable blocks include statelessInputBytes and statelessOutputBytes.
  • Legacy generated fixtures: repo-native JSON files with a top-level stateless_input field.

If you already have EEST blockchain_tests fixtures with stateless bytes, you usually do not need to run witness-generator-cli. Point ere-hosts at the EEST fixture checkout or archive root with --input-folder.

Use witness-generator-cli when you need legacy generated fixtures:

  • You want fixtures from live RPC blocks.
  • You have pre-collected raw JSON-RPC inputs.
  • You want to convert EEST cases into the legacy generated fixture shape.
  • You need ere-hosts to run the selected execution client's legacy host conversion path instead of passing EEST stateless bytes directly to the guest.

For the exact schemas accepted by ere-hosts, see Benchmark Execution Inputs.

Direct EEST Fixtures

Direct EEST fixtures are consumed as input, not generated by this repository. Prefer this path when the EEST blockchain_tests files already include canonical stateless input and output bytes.

cargo run -p ere-hosts --release -- --zkvms sp1 \
    stateless-validator --execution-client reth \
    --input-folder /path/to/execution-specs/fixtures

When the input folder contains a blockchain_tests/ subdirectory, ere-hosts uses only that subtree for stateless-validator inputs.

Legacy Generated Fixtures

witness-generator-cli writes legacy generated fixtures as JSON files. By default it writes them to zkevm-fixtures-input/; override that with -o, --output-folder <PATH>.

Inspect the current CLI surface from the repo root:

cargo run -p witness-generator-cli -- --help

From EEST

Use witness-generator-cli tests only when you need to convert EEST cases into the legacy generated fixture shape.

cargo run -p witness-generator-cli --release -- tests --include 10M --include Prague

Use a specific benchmark release tag:

cargo run -p witness-generator-cli --release -- tests --tag tests-benchmark@v0.0.9

Use a local EEST fixture directory instead of downloading a release:

cargo run -p witness-generator-cli --release -- tests \
    --eest-fixtures-path /path/to/local/eest/fixtures \
    --include Prague

Notes:

  • --tag and --eest-fixtures-path are mutually exclusive.
  • --include and --exclude may be repeated to narrow the generated set.
  • EEST generation runs matching tests in parallel. Lower RAYON_NUM_THREADS if fixture generation is memory constrained.
  • EEST cases that fail before producing an executable block witness are logged and skipped. The generated count reflects fixtures written to disk.

From RPC

Generate fixtures from the latest N blocks:

cargo run -p witness-generator-cli --release -- rpc \
    --last-n-blocks 2 \
    --rpc-url <RPC_URL>

Generate a specific block:

cargo run -p witness-generator-cli --release -- rpc \
    --block 20000000 \
    --rpc-url <RPC_URL>

Follow finalized blocks continuously:

cargo run -p witness-generator-cli --release -- rpc \
    --follow \
    --rpc-url <RPC_URL>

Use custom headers:

cargo run -p witness-generator-cli --release -- rpc \
    --last-n-blocks 2 \
    --rpc-url <RPC_URL> \
    --rpc-header "Authorization:Bearer <TOKEN>" \
    --rpc-header "X-Trace-Id:bench-run-01"

Use a custom or devnet genesis file when the RPC chain ID is not baked into the Reth chain config set:

cargo run -p witness-generator-cli --release -- rpc \
    --last-n-blocks 2 \
    --rpc-url <RPC_URL> \
    --genesis /path/to/genesis.json

Notes:

  • --last-n-blocks, --block, and --follow are mutually exclusive.
  • --rpc-header values must use key:value or key: value.
  • --genesis expects a geth-style genesis.json and the CLI will fail fast if eth_chainId does not match genesis.config.chainId.

From Raw Inputs

Generate fixtures from a folder containing chain_config.json and raw_input_parts.txt:

cargo run -p witness-generator-cli --release -- raw-input \
    --input-folder /path/to/raw-inputs

Each raw-input pair should resolve to eth_block.json and debug_executionWitness.json.

Output Behavior

  • Legacy generated fixtures are JSON files written to zkevm-fixtures-input/ unless --output-folder is set.
  • The CLI creates the output directory if it does not already exist.
  • The generated fixture set can be reused across multiple benchmark runs.

Common Failure Modes

  • cargo run from the repo root must include -p witness-generator-cli; the workspace has multiple binaries.
  • --last-n-blocks 0 is rejected.
  • Unsupported RPC chain IDs require --genesis.
  • Header formatting errors must be fixed to key:value.
  • RPC, GitHub, or EEST download failures may be environmental rather than code regressions.