Skip to content

feat(test-specs): benchmark verification module#2640

Open
LouisTsai-Csie wants to merge 1 commit intoethereum:forks/amsterdamfrom
LouisTsai-Csie:trace-verification
Open

feat(test-specs): benchmark verification module#2640
LouisTsai-Csie wants to merge 1 commit intoethereum:forks/amsterdamfrom
LouisTsai-Csie:trace-verification

Conversation

@LouisTsai-Csie
Copy link
Copy Markdown
Collaborator

@LouisTsai-Csie LouisTsai-Csie commented Apr 9, 2026

🗒️ Description

Add a pluggable block-level verification system for benchmark tests. Instead of adding ad-hoc boolean flags for each new check, tests can now specify a list of BlockVerification rules that are dispatched against the transition tool result for each block.

🔗 Related Issues or PRs

N/A.

✅ Checklist

  • All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    just static
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
  • Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

Cute Animal Picture

Put a link to a cute animal picture inside the parenthesis-->

@LouisTsai-Csie LouisTsai-Csie self-assigned this Apr 9, 2026
@LouisTsai-Csie LouisTsai-Csie added C-feat Category: an improvement or new feature A-test-specs Area: execution_testing.specs labels Apr 9, 2026
@LouisTsai-Csie LouisTsai-Csie changed the title feat(test-benchmark): benchmark verification module feat(test-specs): benchmark verification module Apr 9, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.24%. Comparing base (a48e0b3) to head (3cf3a13).

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2640   +/-   ##
================================================
  Coverage            86.24%   86.24%           
================================================
  Files                  599      599           
  Lines                36984    36984           
  Branches              3795     3795           
================================================
  Hits                 31895    31895           
  Misses                4525     4525           
  Partials               564      564           
Flag Coverage Δ
unittests 86.24% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@LouisTsai-Csie LouisTsai-Csie marked this pull request as ready for review April 13, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-test-specs Area: execution_testing.specs C-feat Category: an improvement or new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant