Skip to content

[ISSUE #10270] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig#10271

Merged
lizhimins merged 2 commits intoapache:developfrom
lizhimins:zhimin/issue-10270
Apr 22, 2026
Merged

[ISSUE #10270] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig#10271
lizhimins merged 2 commits intoapache:developfrom
lizhimins:zhimin/issue-10270

Conversation

@lizhimins
Copy link
Copy Markdown
Member

@lizhimins lizhimins commented Apr 21, 2026

Add popRocksdbBlockCacheSize to MessageStoreConfig with a default of 256MB, replacing the hardcoded value in RocksDBOptionsFactory.createPopCFOptions(). This allows operators to tune Pop RocksDB memory usage based on container memory budgets.

Which Issue(s) This PR Fixes

Brief Description

How Did You Test This Change?

…a MessageStoreConfig

Add popRocksdbBlockCacheSize to MessageStoreConfig with a default of 256MB,
replacing the hardcoded value in RocksDBOptionsFactory.createPopCFOptions().
Default ColumnFamily uses a minimal 16MB cache since it stores no meaningful data.
This allows operators to tune Pop RocksDB memory usage based on container
memory budgets.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 21, 2026

Codecov Report

❌ Patch coverage is 52.94118% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.90%. Comparing base (f04eafc) to head (ea67835).

Files with missing lines Patch % Lines
...ache/rocketmq/store/config/MessageStoreConfig.java 25.00% 6 Missing ⚠️
.../rocketmq/store/rocksdb/RocksDBOptionsFactory.java 0.00% 2 Missing ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop   #10271      +/-   ##
=============================================
- Coverage      48.96%   48.90%   -0.06%     
+ Complexity     13461    13450      -11     
=============================================
  Files           1375     1375              
  Lines         100436   100448      +12     
  Branches       12971    12971              
=============================================
- Hits           49178    49127      -51     
- Misses         45266    45322      +56     
- Partials        5992     5999       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@qianye1001
Copy link
Copy Markdown
Contributor

Thanks for making the Pop RocksDB BlockCache size configurable — this is a good step toward better memory tuning in container environments.

One thing worth considering: blockCacheSize is only part of the memory footprint. In createPopCFOptions(), writeBufferSize (128 MB) and maxWriteBufferNumber (4) are still hardcoded, which means the memtable alone can consume up to 512 MB. For memory-constrained containers, this is just as significant as the block cache.

It would be great to also expose writeBufferSize and maxWriteBufferNumber (and possibly minWriteBufferNumberToMerge) as configurable fields in MessageStoreConfig, so operators can holistically tune the Pop RocksDB memory budget. Otherwise, someone who carefully tunes the block cache down may still be surprised by the fixed memtable overhead.

Happy to discuss further if needed.

… properties

Add writeBufferSize to MessageStoreConfig with a default of 32MB, replacing
the hardcoded value in createPopCFOptions(). maxWriteBufferNumber and
minWriteBufferNumberToMerge remain hardcoded as they rarely need tuning.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@qianye1001
Copy link
Copy Markdown
Contributor

LGTM, thanks for the quick update!

Copy link
Copy Markdown
Contributor

@lollipopjin lollipopjin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lizhimins lizhimins merged commit 0752ddf into apache:develop Apr 22, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] Make Pop RocksDB BlockCache size configurable via MessageStoreConfig

4 participants