Skip to content

Make RocksDB ConsumeQueue size amplification configurable #10344

@RongtongJin

Description

@RongtongJin

Before Creating the Enhancement Request

  • I have confirmed that this should be classified as an enhancement rather than a bug/feature.

Summary

Make RocksDB ConsumeQueue universal compaction size amplification configurable. This gives operators a CQ only tuning option while keeping the current default behavior.

Motivation

RocksDB ConsumeQueue currently uses a fixed universal compaction max size amplification percent of 25. In environments with high compaction pressure operators may need to allow more temporary space amplification in exchange for fewer compactions and lower CPU pressure. A broker side config makes this tunable without changing the default.

Describe the Solution You'd Like

Add a MessageStoreConfig option rocksdbMaxSizeAmplificationPercent with default value 25. Use it only when building the RocksDB ConsumeQueue column family universal compaction options. Do not change POP Index Trans Timer or RocksDB DBOptions defaults.

Describe Alternatives You've Considered

Keep the value hard coded which requires a code change for deployments that need a different compaction tradeoff. Broader RocksDB option parameterization was also considered but it is not needed for this scoped CQ improvement.

Additional Context

Scope is intentionally limited to RocksDB ConsumeQueue. Related PR will follow after this issue is created.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions