Commit 260d068
authored
perf(storage): tune RocksDB options (lambdaclass#380)
## Summary
Replaces bare `Options::default()` on every column family with a tuned
config.
Most options that were no-ops at the RocksDB default were dropped; the
meaningful changes are:
| Option | RocksDB default | This PR |
|---|---|---|
| `max_background_jobs` | `2` | `8` |
| `max_subcompactions` | `1` | `2` |
| `compaction_readahead_size` | `2 MB` | `4 MB` |
| `bytes_per_sync` | `0` (disabled) | `32 MB` |
| `wal_bytes_per_sync` | `0` (disabled) | `32 MB` |
| `max_total_wal_size` | `0` (auto: ~5 GB at our buffer sizes) | `256
MB` |
| `enable_pipelined_write` | `false` | `true` |
| Block cache | per-CF 32 MB auto | shared `128 MB` LRU across all CFs |
Reference: [RocksDB Setup & Basic
Tuning](https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning)
and ethrex's
[`crates/storage/backend/rocksdb.rs`](https://github.com/lambdaclass/ethrex/blob/main/crates/storage/backend/rocksdb.rs).
## Why
We had some errors due to files being kept open during devnet runs. We
think it might be due to compaction being slow. The changes should make
compaction faster.
## Test plan
- [x] `cargo build -p ethlambda-storage`
- [x] `cargo test -p ethlambda-storage --release` (35 tests pass)
- [x] `cargo fmt --all`
- [x] `cargo clippy -p ethlambda-storage -- -D warnings`
- [ ] Devnet run to confirm no regression in startup, finality, or
memory footprint1 parent 3e81771 commit 260d068
1 file changed
Lines changed: 23 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
7 | | - | |
| 7 | + | |
| 8 | + | |
8 | 9 | | |
9 | 10 | | |
10 | 11 | | |
| |||
34 | 35 | | |
35 | 36 | | |
36 | 37 | | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
37 | 54 | | |
38 | 55 | | |
39 | | - | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
40 | 61 | | |
41 | 62 | | |
42 | 63 | | |
| |||
0 commit comments