Skip to content

feat(tests): update python tests after running the script#4

Open
spencer-tb wants to merge 30 commits intoleolara:leolara/port-static-tests-with-postfrom
spencer-tb:spencer-tb/fix-fixture-generator
Open

feat(tests): update python tests after running the script#4
spencer-tb wants to merge 30 commits intoleolara:leolara/port-static-tests-with-postfrom
spencer-tb:spencer-tb/fix-fixture-generator

Conversation

@spencer-tb
Copy link
Copy Markdown

Summary

Regenerates all ported static tests with fork-divergent post-state support and lint fixes.

Compiled static tests ran with:

uv run fill tests/static/ --fill-static-tests --output compiled_static/ -n 10 --until Amsterdam

These can alternatively be found here: u

Porting the to python tests:

uv run python scripts/fixture_to_python.py --fixtures compiled_static --fillers tests/static/state_tests --output tests/ported_static

Verification with hasher:

uv run hasher compare compiled_static/state_tests /tmp/ported_fixtures/state_tests -d 0

Note: only compare state_tests used blockchain_tests embed source paths in their directory layout causing structural hash differences, not content differences.

Verification with compare_fixtures.py:

uv run python scripts/compare_fixtures.py compiled_static /tmp/ported_fixtures

Gives us:

Paired: 2180
Matched: 2180/2180
Right only: 91
The 91 "right only" are the _from_<fork> functions (fork-range tests) which don't exist in compiled_static.

For example, precompsEIP2929Cancun in compiled_static is one fixture that exists in for_cancun/, for_prague/, and for_osaka/ directories. In the ported fixtures, that becomes three separate test functions:

  • test_precomps_eip2929_cancun (matches the compiled_static name)
  • test_precomps_eip2929_cancun_from_prague (extra. "right only")
  • test_precomps_eip2929_cancun_from_osaka (extra, right only")

leolara and others added 26 commits March 9, 2026 20:38
- Use EOA(key=...) for sender instead of raw Address + secret_key
- Use pre.deploy_contract() for contract accounts (fallback to
  Account() for oversized code > 24576 bytes)
- Omit default transaction values (value=0, data=b"", nonce=0,
  access_list=[])
- Fix name collisions: + maps to _plus_, - maps to _minus_
- Conditionally import Hash only when needed
- Add @pytest.mark.slow for slow categories (stQuadraticComplexityTest,
  stStaticCall, stTimeConsuming)
- Remove code from post assertions — code doesn't change and just
  bloats the generated tests
- Fix func_doc IndexError when filler comment is "This " with no body
- Fix addr_vars lookup using padded vs unpadded address keys
- Merge pre-state across all fixture cases (fixes MissingAccountError)
- Derive valid_from from filler network field instead of fixture dir
- Deduplicate fixtures across fork directories (keep earliest fork)
- Parametrize `to` field for contract creation (to=None) support
- Escape docstrings containing backslashes or triple quotes
- Prevent ruff stack overflow with RUST_MIN_STACK=16777216
- Pad access list addresses to 40 hex chars
- Support both static/ and non-static/ fixture output layouts
- Add Amsterdam to FORK_ORDER
- Fix compare_fixtures.py: strip test_ prefix, skip ported_static/
find_fixture_files() was picking up blockchain_tests fixtures which
share the same test names as state_tests, causing identity collisions
during dedup that resulted in 0 state_tests being generated.
When a test has different expected post-states across forks (e.g.
precompile gas changes in Prague), generate separate test functions
per fork range with correct valid_from/valid_until markers.

- Refactor find_fixture_files into grouped variant that returns all
  fork paths per identity
- Add detect_fork_ranges() to compare post-state hashes across forks
- Generate _from_<fork> suffixed functions for divergent ranges
- Skip ranges beyond filler's valid_until to avoid empty fork ranges

Verified: 2180/2180 fixtures match in roundtrip comparison.
@marioevz marioevz force-pushed the spencer-tb/fix-fixture-generator branch from 1452e08 to 4de5c0d Compare March 13, 2026 17:15
Regenerated all 2,180 ported test files using the updated
fixture_to_python.py script. Key improvements:

- Fork-divergent post-states: 88 tests with separate _from_<fork>
  functions when expected results differ across forks
- All 59 categories including slow tests (stQuadraticComplexityTest,
  stStaticCall, stTimeConsuming)
- Verified: 149,490 test cases passed, 0 failed
- Hasher compare: state_tests hashes match compiled_static exactly
- Add RUST_MIN_STACK to tox static env for ruff stack overflow
- Fix ruff lint errors in fixture_to_python.py (docstrings, unused
  vars, dict comprehension)
- Add mypy: ignore-errors to 4 files with deeply nested Op chains
- Remove leftover debug script
@marioevz marioevz force-pushed the spencer-tb/fix-fixture-generator branch from 4de5c0d to a65b9c5 Compare March 13, 2026 17:16
@marioevz
Copy link
Copy Markdown

I'll go ahead and merge these commits on top of Leo's branch, and then will try to merge the parent PR to ethereum/execution-specs.

@marioevz marioevz force-pushed the leolara/port-static-tests-with-post branch 4 times, most recently from 8be9328 to dd2632c Compare March 13, 2026 23:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants