Skip to content

perf(l1): add binaryandhash data block index to all rocksdb CFs for faster point lookups #6529

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dicethedev wants to merge 2 commits into
base: main
Choose a base branch
from
Open

perf(l1): add binaryandhash data block index to all rocksdb CFs for faster point lookups #6529

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
34839c0
add BinaryAndHash data block index to all RocksDB CFs for faster poin…
dicethedev Apr 25, 2026
5109299
fix(l1): remove misleading comments and restrict binaryandhash to poi…
dicethedev Apr 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions crates/storage/backend/rocksdb.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,8 @@ impl RocksDBBackend {
let mut block_opts = BlockBasedOptions::default();
block_opts.set_block_size(16 * 1024); // 16KB
block_opts.set_bloom_filter(10.0, false);
block_opts.set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash);
Copy link
Copy Markdown
Contributor

@ElFantasma ElFantasma May 4, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Direct conflict point with #6530 — that PR adds the identical set_data_block_index_type(BinaryAndHash) + set_data_block_hash_ratio(0.75) calls to the same six BlockBasedOptions blocks. The textual merge resolves cleanly to one copy.

The overlap suggests this PR's content is a subset of #6530, which also adds LZ4 compression and broadens point_lookup_cfs. If the team wants the binary-hash index landed independently of the broader perf changes (smaller surface area, easier to revert if it ever needs to go), this PR is the cleaner unit. If the team is happy to land both together, #6530 is the strictly-larger superset and this PR can close.

block_opts.set_data_block_hash_ratio(0.75);
Copy link
Copy Markdown
Contributor

@ElFantasma ElFantasma May 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: trailing whitespace on this line (and lines 139-140, 154-155, 171, 183-184, 196). cargo fmt would clean these up — worth running before merge.

block_opts.set_block_cache(&block_cache);
cf_opts.set_block_based_table_factory(&block_opts);
}
Expand All @@ -134,6 +136,8 @@ impl RocksDBBackend {
let mut block_opts = BlockBasedOptions::default();
block_opts.set_block_size(16 * 1024); // 16KB
block_opts.set_bloom_filter(10.0, false); // 10 bits per key
block_opts.set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash);
block_opts.set_data_block_hash_ratio(0.75);
block_opts.set_block_cache(&block_cache);
cf_opts.set_block_based_table_factory(&block_opts);
}
Expand All @@ -147,6 +151,8 @@ impl RocksDBBackend {
let mut block_opts = BlockBasedOptions::default();
block_opts.set_block_size(16 * 1024); // 16KB
block_opts.set_bloom_filter(10.0, false); // 10 bits per key
block_opts.set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash);
block_opts.set_data_block_hash_ratio(0.75);
block_opts.set_block_cache(&block_cache);
cf_opts.set_block_based_table_factory(&block_opts);
}
Expand All @@ -162,6 +168,8 @@ impl RocksDBBackend {

let mut block_opts = BlockBasedOptions::default();
block_opts.set_block_size(32 * 1024); // 32KB
block_opts.set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash);
block_opts.set_data_block_hash_ratio(0.75);
block_opts.set_block_cache(&block_cache);
cf_opts.set_block_based_table_factory(&block_opts);
}
Expand All @@ -172,6 +180,8 @@ impl RocksDBBackend {

let mut block_opts = BlockBasedOptions::default();
block_opts.set_block_size(32 * 1024); // 32KB
block_opts.set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash);
block_opts.set_data_block_hash_ratio(0.75);
block_opts.set_block_cache(&block_cache);
cf_opts.set_block_based_table_factory(&block_opts);
}
Expand All @@ -183,6 +193,8 @@ impl RocksDBBackend {

let mut block_opts = BlockBasedOptions::default();
block_opts.set_block_size(16 * 1024);
block_opts.set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash);
block_opts.set_data_block_hash_ratio(0.75);
block_opts.set_block_cache(&block_cache);
cf_opts.set_block_based_table_factory(&block_opts);
}
Expand Down
Loading