Skip to content

Add redis allocation size limit#3035

Merged
chenBright merged 3 commits intoapache:masterfrom
wwbmmm:limit-redis-alloc-size
Jul 22, 2025
Merged

Add redis allocation size limit#3035
chenBright merged 3 commits intoapache:masterfrom
wwbmmm:limit-redis-alloc-size

Conversation

@wwbmmm
Copy link
Copy Markdown
Contributor

@wwbmmm wwbmmm commented Jul 22, 2025

What problem does this PR solve?

Issue Number: null

Problem Summary: redis request or response may have a too large size that cause the server to consume too much memory.

What is changed and the side effects?

Changed: Add a limit for allocation size

Side effects:

  • Performance effects: no

  • Breaking backward compatibility: Some redis request/response may fail to parse if the data size exceed the default value of redis_max_allocation_size (64M). User can adjust the flag redis_max_allocation_size for their senarios.


Check List:

Comment thread src/brpc/redis_reply.cpp
if (count > (int64_t)std::numeric_limits<uint32_t>::max()) {
LOG(ERROR) << "Too many sub replies! max count=2^32-1,"
" actually=" << count;
int64_t array_size = sizeof(RedisReply) * count;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use array_size instead of sizeof(RedisReply) * count in line 243.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

array_size should better be modified with "const"

Comment thread src/brpc/redis_command.cpp Outdated
Comment on lines +411 to +412
<< (FLAGS_redis_max_allocation_size / sizeof(butil::StringPiece))
<< ", actually=" << value;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Format.

Comment thread src/brpc/redis_command.cpp Outdated
" actually=" << len;
if (len > FLAGS_redis_max_allocation_size) {
LOG(ERROR) << "command string exceeds max allocation size! max="
<< FLAGS_redis_max_allocation_size << ", actually=" << len;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Format

Comment thread src/brpc/redis_reply.cpp Outdated
" actually=" << len;
if (len > FLAGS_redis_max_allocation_size) {
LOG(ERROR) << "bulk string exceeds max allocation size! max="
<< FLAGS_redis_max_allocation_size << ", actually=" << len;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Format

Comment thread src/brpc/redis_reply.cpp Outdated
int64_t array_size = sizeof(RedisReply) * count;
if (array_size > FLAGS_redis_max_allocation_size) {
LOG(ERROR) << "array allocation exceeds max allocation size! max="
<< FLAGS_redis_max_allocation_size << ", actually=" << array_size;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Format

Copy link
Copy Markdown
Contributor

@chenBright chenBright left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chenBright chenBright merged commit 75341f8 into apache:master Jul 22, 2025
15 checks passed
@wwbmmm wwbmmm deleted the limit-redis-alloc-size branch July 23, 2025 06:26
@wwbmmm wwbmmm restored the limit-redis-alloc-size branch July 31, 2025 02:09
wwbmmm added a commit that referenced this pull request Jul 31, 2025
wwbmmm added a commit that referenced this pull request Jul 31, 2025
wwbmmm added a commit to wwbmmm/incubator-brpc that referenced this pull request Jul 31, 2025
wwbmmm added a commit that referenced this pull request Aug 2, 2025
* Reapply "Add redis allocation size limit (#3035)"

This reverts commit d0f8f8f.

* Fix int overflow

* Update log
wwbmmm added a commit that referenced this pull request Aug 2, 2025
wwbmmm added a commit that referenced this pull request Aug 2, 2025
* Reapply "Add redis allocation size limit (#3035)"

This reverts commit d0f8f8f.

* Fix int overflow

* Update log
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants