Skip to content

feat(test-benchmark): accept float values for --gas-benchmark-values flag#2693

Open
mandeepmourya007 wants to merge 2 commits intoethereum:forks/amsterdamfrom
mandeepmourya007:fix/bench-gas-float-values
Open

feat(test-benchmark): accept float values for --gas-benchmark-values flag#2693
mandeepmourya007 wants to merge 2 commits intoethereum:forks/amsterdamfrom
mandeepmourya007:fix/bench-gas-float-values

Conversation

@mandeepmourya007
Copy link
Copy Markdown
Contributor

Accept fractional Mgas values in --gas-benchmark-values to speed up the bench-gas CI check. Changes GasBenchmarkValues.root from List[int] to
List[Annotated[float, Gt(0)]], adds int() cast at the multiplication boundary so all downstream code still receives int, updates
format_gas_limit_prefix to handle float formatting, and reduces the Justfile default from 1 to 0.5.

🔗 Related Issues or PRs

Fixes #2673

✅ Checklist

  • All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    just static
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
  • Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

The `just bench-gas` CI check is the bottleneck at ~18 minutes. To allow
reducing the gas benchmark value below 1 Mgas (e.g., 0.5), change the
`GasBenchmarkValues.root` type from `List[int]` to
`List[Annotated[float, Gt(0)]]` and update `format_gas_limit_prefix` to
handle fractional values.

Closes ethereum#2673
@LouisTsai-Csie LouisTsai-Csie added A-test-benchmark Area: execution_testing.benchmark and tests/benchmark C-feat Category: an improvement or new feature labels Apr 16, 2026
@LouisTsai-Csie LouisTsai-Csie changed the title fix(benchmark): accept float values for --gas-benchmark-values flag feat(test-benchmark): accept float values for --gas-benchmark-values flag Apr 16, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 16, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.26%. Comparing base (64dc4ac) to head (faea639).
⚠️ Report is 5 commits behind head on forks/amsterdam.

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #2693   +/-   ##
================================================
  Coverage            86.25%   86.26%           
================================================
  Files                  599      599           
  Lines                37032    37038    +6     
  Branches              3795     3795           
================================================
+ Hits                 31943    31949    +6     
  Misses                4525     4525           
  Partials               564      564           
Flag Coverage Δ
unittests 86.26% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@LouisTsai-Csie
Copy link
Copy Markdown
Collaborator

Thank you for this PR. I have some feedback:

  • Unit tests: Please add unit tests to test_benchmark.py, you could check test_benchmarking_mode_configured_with_option as an example.
  • CI gas limit: The CI fails because 0.5M is too low for some benchmarks. We need to decide whether to (1) skip these benchmarks in CI, or (2) raise the gas limit to 0.7M or higher.
  • Precision loss: The current format_gas_limit_prefix design inconsistent for 0.25M (it becomes for_osaka_at_000_0.2M). We should discuss the desired granularity here.

cc @danceratopz

- Revert Justfile gas value to 1 (0.5M too low for BLS12 precompile tests)
- Fix precision loss in format_gas_limit_prefix: use consistent decimal
  padding across all values for lexicographic sortability (0.25 no longer
  truncated to 0.2)
- Add unit tests for float gas benchmark values (pytester CLI test and
  parametrized format_gas_limit_prefix test)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-test-benchmark Area: execution_testing.benchmark and tests/benchmark C-feat Category: an improvement or new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Speed-up bench-gas check

2 participants