Support dynamic expansion of RDMA block pool#3155
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR enhances the RDMA block pool to support dynamic memory expansion when multiple buckets are configured. Previously, dynamic expansion was restricted when FLAGS_rdma_memory_pool_buckets > 1, limiting scalability.
Key Changes:
- Introduces
expansion_listandexpansion_sizearrays to stage newly allocated memory before distributing it to buckets - Removes the restriction preventing dynamic expansion with multiple buckets
- Refactors memory management to use scope guards for cleaner lock handling
- Deprecates
FLAGS_rdma_memory_pool_user_specified_memoryflag while maintaining backward compatibility
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| src/butil/memory/scope_guard.h | Adds BUTIL_SCOPE_EXIT macro as an alias to BRPC_SCOPE_EXIT for consistent naming across the codebase |
| src/brpc/rdma/block_pool.cpp | Implements expansion_list mechanism for thread-safe dynamic memory expansion, refactors lock management using scope guards, and updates memory pool extension logic |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
When FLAGS_rdma_memory_pool_buckets is greater than 1, a bug may occur due to lock granularity issues. |
This PR can solve this problem. |
5eb557b to
48a20bb
Compare
|
FLAGS_rdma_memory_pool_user_specified_memory is used to allow users to register specific memory, such as SPDK. In these scenarios, memory needs to be allocated by SPDK and then registered in RDMA, which can achieve zero copy. |
Ok, I will revert it. When the idle_list is out of memory in AllocBlockFrom function, it seems that the user cannot immediately dynamically expand specific memory. Is that right? |
|
In this scenario, the memory should be pre-registered, and dynamic allocation requires the user to provide memory too. |
48a20bb to
3a03c02
Compare
@yanglimingcn done |
|
LGTM |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 10 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
What problem does this PR solve?
Issue Number:
Problem Summary:
When
FLAGS_rdma_memory_pool_buckets > 1, the block pool cannot be dynamically expanded.What is changed and the side effects?
Changed:
Dynamically expanded memory is stored in an
expansion_listprotected byextend_lock, and each bucket only retrieves memory from theexpansion_listat the corresponding index.Side effects:
Performance effects:
Breaking backward compatibility:
Check List: