Skip to content

Commit 260d068

Browse files
authored
perf(storage): tune RocksDB options (lambdaclass#380)
## Summary Replaces bare `Options::default()` on every column family with a tuned config. Most options that were no-ops at the RocksDB default were dropped; the meaningful changes are: | Option | RocksDB default | This PR | |---|---|---| | `max_background_jobs` | `2` | `8` | | `max_subcompactions` | `1` | `2` | | `compaction_readahead_size` | `2 MB` | `4 MB` | | `bytes_per_sync` | `0` (disabled) | `32 MB` | | `wal_bytes_per_sync` | `0` (disabled) | `32 MB` | | `max_total_wal_size` | `0` (auto: ~5 GB at our buffer sizes) | `256 MB` | | `enable_pipelined_write` | `false` | `true` | | Block cache | per-CF 32 MB auto | shared `128 MB` LRU across all CFs | Reference: [RocksDB Setup & Basic Tuning](https://github.com/facebook/rocksdb/wiki/Setup-Options-and-Basic-Tuning) and ethrex's [`crates/storage/backend/rocksdb.rs`](https://github.com/lambdaclass/ethrex/blob/main/crates/storage/backend/rocksdb.rs). ## Why We had some errors due to files being kept open during devnet runs. We think it might be due to compaction being slow. The changes should make compaction faster. ## Test plan - [x] `cargo build -p ethlambda-storage` - [x] `cargo test -p ethlambda-storage --release` (35 tests pass) - [x] `cargo fmt --all` - [x] `cargo clippy -p ethlambda-storage -- -D warnings` - [ ] Devnet run to confirm no regression in startup, finality, or memory footprint
1 parent 3e81771 commit 260d068

1 file changed

Lines changed: 23 additions & 2 deletions

File tree

crates/storage/src/backend/rocksdb.rs

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@ use crate::api::{
44
ALL_TABLES, Error, PrefixResult, StorageBackend, StorageReadView, StorageWriteBatch, Table,
55
};
66
use rocksdb::{
7-
ColumnFamilyDescriptor, DBWithThreadMode, MultiThreaded, Options, WriteBatch, WriteOptions,
7+
BlockBasedOptions, Cache, ColumnFamilyDescriptor, DBWithThreadMode, MultiThreaded, Options,
8+
WriteBatch, WriteOptions,
89
};
910
use std::path::Path;
1011
use std::sync::Arc;
@@ -34,9 +35,29 @@ impl RocksDBBackend {
3435
opts.create_if_missing(true);
3536
opts.create_missing_column_families(true);
3637

38+
opts.set_max_open_files(-1);
39+
opts.set_max_file_opening_threads(8);
40+
opts.set_max_background_jobs(8);
41+
opts.set_max_subcompactions(2);
42+
opts.set_compaction_readahead_size(4 * 1024 * 1024);
43+
opts.set_level_compaction_dynamic_level_bytes(true);
44+
opts.set_bytes_per_sync(32 * 1024 * 1024);
45+
opts.set_wal_bytes_per_sync(32 * 1024 * 1024);
46+
opts.set_max_total_wal_size(256 * 1024 * 1024);
47+
opts.set_use_fsync(false);
48+
opts.set_enable_pipelined_write(true);
49+
50+
let block_cache = Cache::new_lru_cache(128 * 1024 * 1024);
51+
let mut block_opts = BlockBasedOptions::default();
52+
block_opts.set_block_cache(&block_cache);
53+
3754
let cf_descriptors: Vec<_> = ALL_TABLES
3855
.iter()
39-
.map(|t| ColumnFamilyDescriptor::new(cf_name(*t), Options::default()))
56+
.map(|t| {
57+
let mut cf_opts = Options::default();
58+
cf_opts.set_block_based_table_factory(&block_opts);
59+
ColumnFamilyDescriptor::new(cf_name(*t), cf_opts)
60+
})
4061
.collect();
4162

4263
let db =

0 commit comments

Comments
 (0)