Skip to content

Commit 012c855

Browse files
committed
feat: add SLOAD/SSTORE mixed benchmarks from execution-spec-tests
Migrated benchmark tests from ethereum/execution-spec-tests#2256: - Added test_mixed_sload_sstore with configurable SLOAD/SSTORE ratios (50-50, 70-30, 90-10) - Added test_sload_empty_erc20_balanceof for pure SLOAD benchmarks - Added test_sstore_erc20_approve for pure SSTORE benchmarks - Support for parametrized contract counts (1, 5, 10, 20, 100) - Uses pre-deployed ERC20 contracts via address stubs for execute mode These benchmarks stress test client implementations with state-heavy operations across multiple contracts to identify performance bottlenecks.
1 parent c3e5260 commit 012c855

3 files changed

Lines changed: 770 additions & 0 deletions

File tree

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# BloatNet Single-Opcode Benchmarks
2+
3+
This directory contains benchmarks for testing single EVM opcodes (SLOAD, SSTORE) under state-heavy conditions using pre-deployed contracts.
4+
5+
## Test Setup
6+
7+
### Prerequisites
8+
9+
1. Pre-deployed ERC20 contracts on the target network
10+
2. A JSON file containing contract addresses (stubs)
11+
12+
### Address Stubs Format
13+
14+
Create a JSON file (`stubs.json`) mapping test-specific stub names to deployed contract addresses:
15+
16+
```json
17+
{
18+
"test_sload_empty_erc20_balanceof_USDT": "0x1234567890123456789012345678901234567890",
19+
"test_sload_empty_erc20_balanceof_USDC": "0x2345678901234567890123456789012345678901",
20+
"test_sload_empty_erc20_balanceof_DAI": "0x3456789012345678901234567890123456789012",
21+
"test_sload_empty_erc20_balanceof_WETH": "0x4567890123456789012345678901234567890123",
22+
"test_sload_empty_erc20_balanceof_WBTC": "0x5678901234567890123456789012345678901234",
23+
24+
"test_sstore_erc20_approve_USDT": "0x1234567890123456789012345678901234567890",
25+
"test_sstore_erc20_approve_USDC": "0x2345678901234567890123456789012345678901",
26+
"test_sstore_erc20_approve_DAI": "0x3456789012345678901234567890123456789012",
27+
"test_sstore_erc20_approve_WETH": "0x4567890123456789012345678901234567890123",
28+
"test_sstore_erc20_approve_WBTC": "0x5678901234567890123456789012345678901234",
29+
30+
"test_mixed_sload_sstore_USDT": "0x1234567890123456789012345678901234567890",
31+
"test_mixed_sload_sstore_USDC": "0x2345678901234567890123456789012345678901",
32+
"test_mixed_sload_sstore_DAI": "0x3456789012345678901234567890123456789012",
33+
"test_mixed_sload_sstore_WETH": "0x4567890123456789012345678901234567890123",
34+
"test_mixed_sload_sstore_WBTC": "0x5678901234567890123456789012345678901234"
35+
}
36+
```
37+
38+
**Naming Convention:**
39+
- Stub names MUST start with the test function name
40+
- Format: `{test_function_name}_{identifier}`
41+
- Example: `test_sload_empty_erc20_balanceof_USDT`
42+
43+
44+
### Running the Tests
45+
46+
#### Execute Mode (Against Live Network)
47+
48+
```bash
49+
# Run with specific number of contracts (e.g., only the 5-contract variant)
50+
uv run execute remote \
51+
--rpc-endpoint http://localhost:8545 \
52+
--rpc-seed-key 0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80 \
53+
--rpc-chain-id 1337 \
54+
--address-stubs geth_stubs.json \
55+
--fork Prague \
56+
tests/benchmark/stateful/bloatnet/test_single_opcode.py::test_sload_empty_erc20_balanceof \
57+
-k "[5]" \
58+
59+
60+
61+
## Test Parametrization
62+
63+
Both single-opcode tests are parametrized with `num_contracts = [1, 5, 10, 20, 100]`, generating 5 test variants each:
64+
65+
- **1 contract**: Baseline single-contract performance
66+
- **5 contracts**: Small-scale multi-contract scenario
67+
- **10 contracts**: Medium-scale multi-contract scenario
68+
- **20 contracts**: Large-scale multi-contract scenario
69+
- **100 contracts**: Very large-scale multi-contract stress test
70+
71+
The mixed SLOAD/SSTORE test additionally parametrizes operation ratios:
72+
73+
- **50-50**: Equal mix of SLOAD and SSTORE operations
74+
- **70-30**: 70% SLOAD, 30% SSTORE operations
75+
- **90-10**: 90% SLOAD, 10% SSTORE operations
76+
77+
### How Stub Filtering Works
78+
79+
1. Test extracts its function name (e.g., `test_sload_empty_erc20_balanceof`)
80+
2. Filters stubs starting with that name from `stubs.json`
81+
3. Selects the **first N** matching stubs based on `num_contracts` parameter
82+
4. Errors if insufficient matching stubs found
83+
84+
85+
## Benchmark Descriptions
86+
87+
### test_sload_empty_erc20_balanceof
88+
Tests SLOAD operations by calling `balanceOf()` on ERC20 contracts with random addresses, forcing cold storage reads of likely-empty slots.
89+
90+
### test_sstore_erc20_approve
91+
Tests SSTORE operations by calling `approve()` on ERC20 contracts with incrementing spender addresses, forcing cold storage writes to new allowance slots.
92+
93+
### test_mixed_sload_sstore
94+
Tests mixed SLOAD/SSTORE workloads with configurable ratios, simulating realistic DeFi application patterns with combined read/write operations.

tests/benchmark/stateful/bloatnet/test_multi_opcode.py

Lines changed: 256 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@
66
operations.
77
"""
88

9+
from typing import TYPE_CHECKING, Any
10+
911
import pytest
1012
from execution_testing import (
1113
Account,
@@ -503,3 +505,257 @@ def test_bloatnet_balance_extcodehash(
503505
blocks=[Block(txs=[attack_tx])],
504506
post=post,
505507
)
508+
509+
510+
# ERC20 function selectors
511+
BALANCEOF_SELECTOR = 0x70A08231 # balanceOf(address)
512+
APPROVE_SELECTOR = 0x095EA7B3 # approve(address,uint256)
513+
514+
515+
@pytest.mark.valid_from("Prague")
516+
@pytest.mark.parametrize("num_contracts", [1, 5, 10, 20, 100])
517+
@pytest.mark.parametrize(
518+
"sload_percent,sstore_percent",
519+
[
520+
pytest.param(50, 50, id="50-50"),
521+
pytest.param(70, 30, id="70-30"),
522+
pytest.param(90, 10, id="90-10"),
523+
],
524+
)
525+
def test_mixed_sload_sstore(
526+
blockchain_test: BlockchainTestFiller,
527+
pre: Alloc,
528+
fork: Fork,
529+
gas_benchmark_value: int,
530+
address_stubs, # Type provided by pytest fixture
531+
num_contracts: int,
532+
sload_percent: int,
533+
sstore_percent: int,
534+
request: pytest.FixtureRequest,
535+
) -> None:
536+
"""
537+
BloatNet mixed SLOAD/SSTORE benchmark with configurable operation ratios.
538+
539+
This test:
540+
1. Filters stubs matching test name prefix
541+
(e.g., test_mixed_sload_sstore_*)
542+
2. Uses first N contracts based on num_contracts parameter
543+
3. Divides gas budget evenly across all selected contracts
544+
4. For each contract, divides gas into SLOAD and SSTORE portions by
545+
percentage
546+
5. Executes balanceOf (SLOAD) and approve (SSTORE) calls per the ratio
547+
6. Stresses clients with combined read/write operations on large
548+
contracts
549+
"""
550+
# Extract test function name for stub filtering
551+
test_name = request.node.name.split("[")[0] # Remove parametrization suffix
552+
553+
# Filter stubs that match the test name prefix
554+
matching_stubs = [
555+
stub_name for stub_name in address_stubs.root.keys() if stub_name.startswith(test_name)
556+
]
557+
558+
# Validate we have enough stubs
559+
if len(matching_stubs) < num_contracts:
560+
pytest.fail(
561+
f"Not enough matching stubs for test '{test_name}'. "
562+
f"Required: {num_contracts}, Found: {len(matching_stubs)}. "
563+
f"Matching stubs: {matching_stubs}"
564+
)
565+
566+
# Select first N stubs
567+
selected_stubs = matching_stubs[:num_contracts]
568+
gas_costs = fork.gas_costs()
569+
570+
# Calculate gas costs
571+
intrinsic_gas = fork.transaction_intrinsic_cost_calculator()(calldata=b"")
572+
573+
# Fixed overhead for SLOAD loop
574+
sload_loop_overhead = (
575+
# Attack contract loop overhead
576+
gas_costs.G_VERY_LOW * 2 # MLOAD counter (3*2)
577+
+ gas_costs.G_VERY_LOW * 2 # MSTORE selector (3*2)
578+
+ gas_costs.G_VERY_LOW * 3 # MLOAD + MSTORE address (3*3)
579+
+ gas_costs.G_BASE # POP (2)
580+
+ gas_costs.G_BASE * 3 # SUB + MLOAD + MSTORE for counter decrement (2*3)
581+
+ gas_costs.G_BASE * 2 # ISZERO * 2 for loop condition (2*2)
582+
+ gas_costs.G_MID # JUMPI (8)
583+
)
584+
585+
# ERC20 balanceOf internal gas
586+
sload_erc20_internal = (
587+
gas_costs.G_VERY_LOW # PUSH4 selector (3)
588+
+ gas_costs.G_BASE # EQ selector match (2)
589+
+ gas_costs.G_MID # JUMPI to function (8)
590+
+ gas_costs.G_JUMPDEST # JUMPDEST at function start (1)
591+
+ gas_costs.G_VERY_LOW * 2 # CALLDATALOAD arg (3*2)
592+
+ gas_costs.G_KECCAK_256 # keccak256 static (30)
593+
+ gas_costs.G_KECCAK_256_WORD * 2 # keccak256 dynamic for 64 bytes (2*6)
594+
+ gas_costs.G_COLD_SLOAD # Cold SLOAD - always cold for random addresses (2100)
595+
+ gas_costs.G_VERY_LOW * 3 # MSTORE result + RETURN setup (3*3)
596+
)
597+
598+
# Fixed overhead for SSTORE loop
599+
sstore_loop_overhead = (
600+
# Attack contract loop body operations
601+
gas_costs.G_VERY_LOW # MSTORE selector at memory[32] (3)
602+
+ gas_costs.G_LOW # MLOAD counter (5)
603+
+ gas_costs.G_VERY_LOW # MSTORE spender at memory[64] (3)
604+
+ gas_costs.G_BASE # POP call result (2)
605+
# Counter decrement
606+
+ gas_costs.G_LOW # MLOAD counter (5)
607+
+ gas_costs.G_VERY_LOW # PUSH1 1 (3)
608+
+ gas_costs.G_VERY_LOW # SUB (3)
609+
+ gas_costs.G_VERY_LOW # MSTORE counter back (3)
610+
# While loop condition check
611+
+ gas_costs.G_LOW # MLOAD counter (5)
612+
+ gas_costs.G_BASE # ISZERO (2)
613+
+ gas_costs.G_BASE # ISZERO (2)
614+
+ gas_costs.G_MID # JUMPI back to loop start (8)
615+
)
616+
617+
# ERC20 approve internal gas
618+
# Cold SSTORE: 22100 = 20000 base + 2100 cold access
619+
sstore_erc20_internal = (
620+
gas_costs.G_VERY_LOW # PUSH4 selector (3)
621+
+ gas_costs.G_BASE # EQ selector match (2)
622+
+ gas_costs.G_MID # JUMPI to function (8)
623+
+ gas_costs.G_JUMPDEST # JUMPDEST at function start (1)
624+
+ gas_costs.G_VERY_LOW # CALLDATALOAD spender (3)
625+
+ gas_costs.G_VERY_LOW # CALLDATALOAD amount (3)
626+
+ gas_costs.G_KECCAK_256 # keccak256 static (30)
627+
+ gas_costs.G_KECCAK_256_WORD * 2 # keccak256 dynamic for 64 bytes (12)
628+
+ gas_costs.G_COLD_SLOAD # Cold SLOAD for allowance check (2100)
629+
+ gas_costs.G_STORAGE_SET # SSTORE base cost (20000)
630+
+ gas_costs.G_COLD_SLOAD # Additional cold storage access (2100)
631+
+ gas_costs.G_VERY_LOW # PUSH1 1 for return value (3)
632+
+ gas_costs.G_VERY_LOW # MSTORE return value (3)
633+
+ gas_costs.G_VERY_LOW # PUSH1 32 for return size (3)
634+
+ gas_costs.G_VERY_LOW # PUSH1 0 for return offset (3)
635+
)
636+
637+
# Calculate gas budget per contract
638+
available_gas = gas_benchmark_value - intrinsic_gas
639+
gas_per_contract = available_gas // num_contracts
640+
641+
# For each contract, split gas by percentage
642+
sload_gas_per_contract = (gas_per_contract * sload_percent) // 100
643+
sstore_gas_per_contract = (gas_per_contract * sstore_percent) // 100
644+
645+
# Account for cold/warm transitions in CALL costs
646+
# First SLOAD call is COLD (2600), rest are WARM (100)
647+
sload_warm_cost = sload_loop_overhead + gas_costs.G_WARM_ACCOUNT_ACCESS + sload_erc20_internal
648+
cold_warm_diff = gas_costs.G_COLD_ACCOUNT_ACCESS - gas_costs.G_WARM_ACCOUNT_ACCESS
649+
sload_calls_per_contract = int((sload_gas_per_contract - cold_warm_diff) // sload_warm_cost)
650+
651+
# First SSTORE call is COLD (2600), rest are WARM (100)
652+
sstore_warm_cost = (
653+
sstore_loop_overhead + gas_costs.G_WARM_ACCOUNT_ACCESS + sstore_erc20_internal
654+
)
655+
sstore_calls_per_contract = int((sstore_gas_per_contract - cold_warm_diff) // sstore_warm_cost)
656+
657+
# Deploy selected ERC20 contracts using stubs
658+
erc20_addresses = []
659+
for stub_name in selected_stubs:
660+
addr = pre.deploy_contract(
661+
code=Bytecode(),
662+
stub=stub_name,
663+
)
664+
erc20_addresses.append(addr)
665+
666+
# Log test requirements
667+
print(
668+
f"Total gas budget: {gas_benchmark_value / 1_000_000:.1f}M gas. "
669+
f"~{gas_per_contract / 1_000_000:.1f}M gas per contract "
670+
f"({sload_percent}% SLOAD, {sstore_percent}% SSTORE). "
671+
f"Per contract: {sload_calls_per_contract} balanceOf calls, "
672+
f"{sstore_calls_per_contract} approve calls."
673+
)
674+
675+
# Build attack code that loops through each contract
676+
attack_code: Bytecode = (
677+
Op.JUMPDEST # Entry point
678+
+ Op.MSTORE(offset=0, value=BALANCEOF_SELECTOR) # Store selector once for all contracts
679+
)
680+
681+
for erc20_address in erc20_addresses:
682+
# For each contract, execute SLOAD operations (balanceOf)
683+
attack_code += (
684+
# Initialize counter in memory[32] = number of balanceOf calls
685+
Op.MSTORE(offset=32, value=sload_calls_per_contract)
686+
# Loop for balanceOf calls
687+
+ While(
688+
condition=Op.MLOAD(32) + Op.ISZERO + Op.ISZERO,
689+
body=(
690+
# Call balanceOf(address) on ERC20 contract
691+
# args_offset=28 reads: selector from MEM[28:32] + address
692+
# from MEM[32:64]
693+
Op.CALL(
694+
address=erc20_address,
695+
value=0,
696+
args_offset=28,
697+
args_size=36,
698+
ret_offset=0,
699+
ret_size=0,
700+
)
701+
+ Op.POP # Discard CALL success status
702+
# Decrement counter
703+
+ Op.MSTORE(offset=32, value=Op.SUB(Op.MLOAD(32), 1))
704+
),
705+
)
706+
)
707+
708+
# For each contract, execute SSTORE operations (approve)
709+
# Reuse the same memory layout as balanceOf
710+
attack_code += (
711+
# Store approve selector at memory[0] (reusing same slot)
712+
Op.MSTORE(offset=0, value=APPROVE_SELECTOR)
713+
# Initialize counter in memory[32] = number of approve calls
714+
# (reusing same slot)
715+
+ Op.MSTORE(offset=32, value=sstore_calls_per_contract)
716+
# Loop for approve calls
717+
+ While(
718+
condition=Op.MLOAD(32) + Op.ISZERO + Op.ISZERO,
719+
body=(
720+
# Store spender at memory[64] (counter as spender/amount)
721+
Op.MSTORE(offset=64, value=Op.MLOAD(32))
722+
# Call approve(spender, amount) on ERC20 contract
723+
# args_offset=28 reads: selector from MEM[28:32] +
724+
# spender from MEM[32:64] + amount from MEM[64:96]
725+
# Note: counter at MEM[32:64] is reused as spender,
726+
# and value at MEM[64:96] serves as the amount
727+
+ Op.CALL(
728+
address=erc20_address,
729+
value=0,
730+
args_offset=28,
731+
args_size=68,
732+
ret_offset=0,
733+
ret_size=0,
734+
)
735+
+ Op.POP # Discard CALL success status
736+
# Decrement counter
737+
+ Op.MSTORE(offset=32, value=Op.SUB(Op.MLOAD(32), 1))
738+
),
739+
)
740+
)
741+
742+
# Deploy attack contract
743+
attack_address = pre.deploy_contract(code=attack_code)
744+
745+
# Run the attack
746+
attack_tx = Transaction(
747+
to=attack_address,
748+
gas_limit=gas_benchmark_value,
749+
sender=pre.fund_eoa(),
750+
)
751+
752+
# Post-state
753+
post = {
754+
attack_address: Account(storage={}),
755+
}
756+
757+
blockchain_test(
758+
pre=pre,
759+
blocks=[Block(txs=[attack_tx])],
760+
post=post,
761+
)

0 commit comments

Comments
 (0)