Releases: ethereum/execution-spec-tests
v5.4.0
🇯🇵 Summary
EEST v5.4.0 is our final Osaka release following the success of the Fusaka upgrade!
This release adds additional tests for Osaka but predominantly includes tests for pre-Osaka forks, catching critical coverage edge cases (added by @chfast and @pdobacz). These edge cases are within both the develop and stable releases. Benchmark tests are skipped in this release.
🧪 Test Cases
- ✨ Expand EIP-6110 modified contract tests, where the extra event log has no topics at all (#1693).
- ✨ Add a CREATE/2 test cases for when it runs OOG on code deposit (#1705).
- ✨ Expand cases to test *CALL opcodes causing OOG (#1703).
- ✨ Add a test case for base fee in block check after London (#1702).
- ✨ Add tests for
modexpandripemdprecompiled contracts (#1691, #1781). - ✨ Add
ecrecoverprecompile tests originating formevmoneunittests (#1685). - ✨ Add test to validate withdarawls root (#1746).
- ✨ Add test for old behavior of zero gasprice txs (#1736).
- ✨ Add stack overflow tests and expand
BLOCKHASHtests (#1728). - ✨ Ported tests for
ripemdprecompile going OOG for all forks (#1732). - ✨ Add tests for ecadd/ecmul/ecpairing constant gas repricing (#1738).
- ✨ Add tests that EIP-1559 and EIP-2930 typed txs are invalid and void before their fork (#1754).
- ✨ Add tests for an old validation rule for gas limit above 5000 (#1731).
- ✨ Add tests for OOG in EXP, LOG and others (#1686).
zkEVM@v0.4.0
This is a new release of projects/zkevm which:
- Based on
bal-devnet-7(bal@v7.1.0) - Testing framework:
- Support a new test marker
@pytest.mark.skip_stateless_validationthat allows test authors to skip stateless execution & filling. This is currently used only in very limited cases where tests violate the limits of CL-types (e.g., the number of withdrawals). These blocks do not represent a valid payload for CLs, thus indirectly can’t be properly SSZ-encoded followingconsensus-specsdefinitions of max sizes. (More info in PR) - Automatically skip stateless runs in fixtures that utilise the raw-RLP-modifier. This is done because RLP modifications can’t have a direct semantic mapping to meaningful ones for the guest program, given that we use SSZ. In the near future, more testing focused on raw-bytes guest input modifications (i.e., raw SSZ modifiers/mutations) is planned to ensure this testing angle is covered. (More info in PR — thx @Gabriel-Trintinalia for reporting)
- Support a new test marker
- Specs:
- Add a
schema-idtwo-byte prefix to the stateless guest program input to indicate how the following bytes should be interpreted for deserialization. This is required since SSZ is not a self-describing encoding format, so the guest program might not know which fork the provided block corresponds to (e.g.ExecutionPayloadis different between forks, thusNewPayloadRequesttoo). This prefix also allows the use of other encoding formats (this is not planned today), thus should not be strictly understood as fork-related encodings, but more generically. (More info in PR). - Change
ChainConfigtype present inStatelessInputto contain a clearer description of the chain configuration to be used. (More info in the PR. Also, the EthR&D #l1-zkevm-protocol thread.) - Fill the current
public_keysfield inStatelessInput. This is a field we had declared in the specs for a long time, but it wasn’t properly filled in the fixtures. The goal is to provide the guest program with the public keys corresponding to the payload transactions, since verifying signatures against public keys is significantly faster than performing a full ecrecover. Note that, temporarily, guest programs could still do an ecrecover, but it is highly recommended that they leverage this optimisation opportunity, since last year's benchmarks show a ~20% performance improvement in proving times. (More info in PR)
- Add a
zkevm@v0.3.4
This is a new release of projects/zkevm which:
- Based on bal@v5.6.1.
- Testing framework:
- Add support for
block_tests_enginefixture format (PR).
- Add support for
- Specs:
- Fix
NewPayloadRequest.execution_requeststypes to comply to consensus-specs (more info). - Fix
Withdrawal.amounttype to comply with consensus-specs (more info). - Fix
statelessInputBytesin fixtures to properly include invalid transactions (more info) (thx @Gabriel-Trintinalia for reporting).
- Fix
Note: fixtures that use RLP-modifiers should not be run. These will properly avoid generating stateless fields in fixtures in the next release!
zkevm@v0.3.3
This is a new release of projects/zkevm which:
- Is based on a newly created bal@v5.6.1.
- Specs:
- The sparse MPT implementation of the guest program now fails if it walks a
HashedNode. The guest program must not conclude anything about the state if not enough MPT nodes were provided in the witness, i.e. this means an invalid execution witness. - Added tighter checks (1, 2, 3) and overall simplifications in the guest program code.
- The sparse MPT implementation of the guest program now fails if it walks a
- Execution witness:
- Correct construction:
- Invalid execution witness for stateless execution:
- Nine new tests covering invalid
executionWitness.statescenarios. In these cases, the stateless execution must fail because information is missing. The following are examples of these scenarios which were implemented:- Missing nodes for the required proof of presence
- Missing nodes for the required proof of absence
- Missing sibilings required for post-state root calculation due to branch compressions.
- Nine new tests covering invalid
zkevm@v0.3.2
This release contains no changes compared to zkevm@v0.3.2, but a rebase on top of the latest bal-devnet-3 release bal@v5.5.1.
zkevm@v0.3.1
This is a new release of projects/zkevm based on bal@v5.4.0 (i.e. the latest tag of bal-devnet-3).
In this release, the following are the notable changes:
- Invalid block tests are now supported — they contain all the new fixtures fields for zkEVM testing.
- A new test for 7702 was added to cover an OOG border case in CALL/CALLCODE containing a balance transfer without enough funds.
- 15 new tests available to cover
executionWitness.statefield border cases. - The above new
executionWitness.statetests uncovered a bug in the guest program specs regarding the correct post-state root calculation ordering to minimise branch compressions. Please see here for more details, since it is a subtle spec detail. The execution witness generation was correct on this front.
zkevm@v0.3.0
This is a new release of projects/zkevm based on bal@v5.4.0 (i.e. the latest tag of bal-devnet-3).
On the specs side, changes:
- Any bytecode successfully created during block execution is considered available for further transactions requiring the same
code_hash. For more details, see the corresponding PR.
On the test front, 117 new tests were created to provide much better coverage for bytecodes and headers fields border cases in the execution witness:
- Bytecodes: 104 new tests in files
test_witness_bytecodes_*.pyandtest_witness_7702.py - Headers: 13 new tests in file
test_witness_headers.py.
To ease testing for EL clients, future releases will always be based on the latest release of the bal-devnet-x active devnet. Minor versions will be bumped whenever a new devnet is released.
zkevm@v0.2.0
This is the first release of the project/zkevm feature branch. It contains the first version of execution witness specification and all tests filled targeting fork --fork=Amsterdam (~12k fixtures). Additionally, it contains a first draft of the guest program specs -- while this can't be considered fully internally tested, we encourage interested EL clients to take a look since it contains already most of the required logic.
zkEVM Execution witness
Each filled fixture contains, at the block level, a new executionWitness field containing:
state: Merkle Patricia Trie (MPT) nodes corresponding to MPT proofs for accessed pre-state during block execution, plus required extra nodes for post-state root calculation. This field is ordered lexicographically.codes: required contract bytecodes accessed during block execution. Empty contracts are not included since they can be identified by the empty code hash. Note that created contracts aren’t included either since their value is calculated at runtime. This field is ordered lexicographically.headers: contains the RLP-encoded block headers of ancestors. This is required to resolve EIP-2935 and executed BLOCKHASH. This field is ordered by block number ascending.
You can find the implementation of the witness generation in the amsterdam fork spec in the projects/zkevm branch in execution-specs . The filled fixtures comply with the latest change in the fixture output directory structure.
This is the first release of the execution witness specs and filled tests, so if you think there is any bug please create an issue in execution-specs or add a message in the created thread in #l1-zkevm-protocol channel in EthR&D discord server.
Guest program
In this release, the forks/amsterdam folder contains an experimental/draft guest program spec. The entry point of the guest program can be found in the run_stateless_guest(...) function in stateless_guest.py file. Over the coming days/weeks, we'll continue iterating and polishing the implementation to target a fully tested version in the next release. We encourage ELs to provide feedback through the same channels as mentioned in the previous section.
static@v0.0.1
Historical Static Tests Fixture Snapshot v0.0.1
This release contains a snapshot of the static tests (ethereum/tests within EELS) filled using the framework --fill-static-tests flag. The --post-verifications flag was added so fixtures contain all the post state verification checks added in: ethereum/execution-specs#2552
These fixtures will be used as a snapshot for comparison as part of the static test port to python via following issue/PRs:
Fast Download
Run the following command to download to your EELS repo:
uv run consume cache --input static@v0.0.1 --extract-to ./fixtures_staticsnobal-devnet-6@v1.1.0
This is a patch release that fixes some issues with the fixture generation for some tests relating to triggerable withdrawals.
Fixes
Removed a tx_gas_limit mutation in the EIP-7002 withdrawal-request conftest that persisted across fixture-format runs (blockchain_test, blockchain_test_engine, etc.), causing the same parametrize cell to fill with different block hashes; the per-request state-gas bump now lives only in WithdrawalRequestContract.transactions() and is gated by an explicit fund_state_reservoir flag, opted into only by the one call_depth_high test that pins tx_gas_limit at the EIP-7825 cap.