feat(benchmarks): fix benchmark for warm SSTORE/SLOAD for Osaka#1915
Conversation
|
cc @jochem-brouwer if you want to take a look |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## forks/amsterdam #1915 +/- ##
================================================
Coverage 82.61% 82.61%
================================================
Files 417 417
Lines 26862 26862
Branches 2492 2492
================================================
Hits 22193 22193
Misses 4230 4230
Partials 439 439
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
LouisTsai-Csie
left a comment
There was a problem hiding this comment.
Thanks for this PR! Leave some comments about the usage of tx_gas_limit.
execution-specs/tests/benchmark/conftest.py
Lines 90 to 93 in 2a2d0d5
One question for this test (test_storage_access_warm), I wonder if we should take care of the OOG issue for this test? It seems these benchmark test transaction would OOG, and my understanding is that the storage would revert due to OOG, will this affect the intention of this test?
Since this test (test_storage_access_cold) do consider this scenario.
Fixed! I'll also do this change in other open PRs.
I think the non-OOG case is of main relevance for this warm flavor.. In this test we exhaust the gas limit in a single storage slot on warm accesses, so the potential post-state-root calculation would only imply a single storage slot. For the cold case, I added OOG and non-OOG since at a minimum non-OOG is quite relevant, since we target as many storage slots as possible, so those are many writes that will triger post-state root calculation. |
LouisTsai-Csie
left a comment
There was a problem hiding this comment.
We have recently bumped our default branch from forks/osaka to forks/amsterdam, could you please help update your base branch please. Otherwise, i have no other requested changes! Really appreciate the help, thanks a lot
27ee240 to
7f7358d
Compare
|
@LouisTsai-Csie, all done. |
|
Thanks a lot!! i will merge after |
3d2eede to
9e161b6
Compare
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
7f7358d to
383427c
Compare
|
LGTM |
🗒️ Description
This PR fixes the
test_storage_access_warmbenchmark to work in Osaka.For 20M gas benchmark value:
test_storage.py::test_storage_access_warm[fork_Osaka-blockchain_test-SLOAD]: Osaka 310M cycles. Prague 311M cyclestest_storage.py::test_storage_access_warm[fork_Osaka-blockchain_test-SSTORE new value]: Osaka 908M cycles. Prague 909M cycles.test_storage.py::test_storage_access_warm[fork_Osaka-blockchain_test-SSTORE same value]: Osaka 636M cycles. Prague 637M cycles.Pretty brutal for only being 20M gas.
🔗 Related Issues or PRs
#1695
✅ Checklist
toxchecks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:uvx tox -e statictype(scope):.