perf(l1): enable lz4 compression for trie and flat key-value tables #6530
+22
−0
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -71,6 +71,10 @@ impl RocksDBBackend { | |
| RECEIPTS, | ||
| TRANSACTION_LOCATIONS, | ||
| FULLSYNC_HEADERS, | ||
| ACCOUNT_TRIE_NODES, | ||
| STORAGE_TRIE_NODES, | ||
| ACCOUNT_FLATKEYVALUE, | ||
| STORAGE_FLATKEYVALUE, | ||
| ]; | ||
|
|
||
| // opts.enable_statistics(); | ||
|
|
@@ -121,6 +125,9 @@ impl RocksDBBackend { | |
| let mut block_opts = BlockBasedOptions::default(); | ||
| block_opts.set_block_size(16 * 1024); // 16KB | ||
| block_opts.set_bloom_filter(10.0, false); | ||
| block_opts | ||
| .set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash); | ||
|
Contributor
There was a problem hiding this comment. Direct conflict point with #6529 — both add No functional disagreement to resolve — just merge order. The author teams (you and @dicethedev) might want to pick one PR as the canonical landing for this index change and close the other, instead of letting the merge race decide which becomes the no-op rebase. |
||
| block_opts.set_data_block_hash_ratio(0.75); | ||
| block_opts.set_block_cache(&block_cache); | ||
| cf_opts.set_block_based_table_factory(&block_opts); | ||
| } | ||
|
|
@@ -134,6 +141,9 @@ impl RocksDBBackend { | |
| let mut block_opts = BlockBasedOptions::default(); | ||
| block_opts.set_block_size(16 * 1024); // 16KB | ||
| block_opts.set_bloom_filter(10.0, false); // 10 bits per key | ||
| block_opts | ||
| .set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash); | ||
| block_opts.set_data_block_hash_ratio(0.75); | ||
| block_opts.set_block_cache(&block_cache); | ||
| cf_opts.set_block_based_table_factory(&block_opts); | ||
| } | ||
|
|
@@ -147,6 +157,9 @@ impl RocksDBBackend { | |
| let mut block_opts = BlockBasedOptions::default(); | ||
| block_opts.set_block_size(16 * 1024); // 16KB | ||
| block_opts.set_bloom_filter(10.0, false); // 10 bits per key | ||
| block_opts | ||
| .set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash); | ||
| block_opts.set_data_block_hash_ratio(0.75); | ||
| block_opts.set_block_cache(&block_cache); | ||
| cf_opts.set_block_based_table_factory(&block_opts); | ||
| } | ||
|
|
@@ -162,6 +175,9 @@ impl RocksDBBackend { | |
|
|
||
| let mut block_opts = BlockBasedOptions::default(); | ||
| block_opts.set_block_size(32 * 1024); // 32KB | ||
| block_opts | ||
| .set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash); | ||
| block_opts.set_data_block_hash_ratio(0.75); | ||
| block_opts.set_block_cache(&block_cache); | ||
| cf_opts.set_block_based_table_factory(&block_opts); | ||
| } | ||
|
|
@@ -172,6 +188,9 @@ impl RocksDBBackend { | |
|
|
||
| let mut block_opts = BlockBasedOptions::default(); | ||
| block_opts.set_block_size(32 * 1024); // 32KB | ||
| block_opts | ||
| .set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash); | ||
| block_opts.set_data_block_hash_ratio(0.75); | ||
| block_opts.set_block_cache(&block_cache); | ||
| cf_opts.set_block_based_table_factory(&block_opts); | ||
| } | ||
|
|
@@ -183,6 +202,9 @@ impl RocksDBBackend { | |
|
|
||
| let mut block_opts = BlockBasedOptions::default(); | ||
| block_opts.set_block_size(16 * 1024); | ||
| block_opts | ||
| .set_data_block_index_type(rocksdb::DataBlockIndexType::BinaryAndHash); | ||
| block_opts.set_data_block_hash_ratio(0.75); | ||
| block_opts.set_block_cache(&block_cache); | ||
| cf_opts.set_block_based_table_factory(&block_opts); | ||
| } | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Scope difference vs #6529 worth surfacing: this PR expands
point_lookup_cfsto includeACCOUNT_TRIE_NODES,STORAGE_TRIE_NODES,ACCOUNT_FLATKEYVALUE,STORAGE_FLATKEYVALUE. #6529 doesn't touch this array. Thepoint_lookup_cfsset drives other optimizations (theoptimize_for_point_lookuppath elsewhere in this file) — adding trie/FKV CFs here changes their broader RocksDB tuning, not just the binary-hash index.Worth a brief justification in the PR description for why these four CFs belong in the point-lookup set. Trie nodes and flat KV are both keyed lookups (32-byte keys), so the choice is sound, but it's a non-trivial config change that's separate from the binary-hash index addition. Could be split into its own PR if the team wants surgical perf work.