Skip to content

[ISSUE #9926] Avoid redundant list allocation in parsePublishMessageQueues#10380

Open
wushiyuanmaimob wants to merge 1 commit into
apache:developfrom
wushiyuanmaimob:perf/reduce-alloc-parsePublishMessageQueues
Open

[ISSUE #9926] Avoid redundant list allocation in parsePublishMessageQueues#10380
wushiyuanmaimob wants to merge 1 commit into
apache:developfrom
wushiyuanmaimob:perf/reduce-alloc-parsePublishMessageQueues

Conversation

@wushiyuanmaimob
Copy link
Copy Markdown

Which Issue(s) This PR Fixes

Brief Description

MQAdminImpl.parsePublishMessageQueues() is called on every message send (hot path). Under high throughput (50K+ TPS), it creates significant GC pressure by allocating a new ArrayList and N new MessageQueue objects per call — even when namespace is empty (the common case for most users), where the copied data is identical to the original.

Changes:

  1. When namespace is null or empty, return the original list directly (zero allocation)
  2. When namespace stripping is needed, pre-allocate ArrayList with exact capacity to avoid internal array resizing
  3. Cache the namespace string to avoid repeated getNamespace() calls in the loop

Impact: Eliminates per-message object allocation on the send path for the majority of users who don't use namespaces. For users with namespaces, reduces allocation by pre-sizing the list.

How Did You Test This Change?

  • Added unit test assertParsePublishMessageQueuesReturnsOriginalListWhenNoNamespace — verifies same list instance is returned when namespace is empty
  • Added unit test assertParsePublishMessageQueuesStripsNamespace — verifies namespace stripping still works correctly
  • Ran full client module test suite: 566 tests passed, 0 failures

…ssageQueues

When namespace is empty (the common case for most users), the method
previously allocated a new ArrayList and N new MessageQueue objects on
every send call, copying identical data. Under high throughput (50K+ TPS),
this creates significant GC pressure from short-lived objects.

This change:
- Returns the original list directly when namespace is empty (zero alloc)
- Pre-allocates ArrayList capacity when namespace stripping is needed

Signed-off-by: sywu14 <wushiyuanwork@outlook.com>
List<MessageQueue> resultQueues = new ArrayList<>();
String namespace = this.mQClientFactory.getClientConfig().getNamespace();
if (namespace == null || namespace.isEmpty()) {
return messageQueueList;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One concern about this change is that the method semantics change from “always returning a new list” to “returning the original list when namespace is empty”.

If any caller mutates the returned list, this could accidentally mutate the original message queue list, especially if that list is reused or cached elsewhere in the send path.

Could you confirm that the return value of parsePublishMessageQueues is treated as read-only by all callers, or that the original list is not reused as mutable shared state?

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 48.83%. Comparing base (051ba27) to head (cf749c4).
⚠️ Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
...a/org/apache/rocketmq/client/impl/MQAdminImpl.java 80.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@              Coverage Diff              @@
##             develop   #10380      +/-   ##
=============================================
- Coverage      48.89%   48.83%   -0.07%     
+ Complexity     13452    13439      -13     
=============================================
  Files           1376     1376              
  Lines         100527   100530       +3     
  Branches       12983    12984       +1     
=============================================
- Hits           49154    49090      -64     
- Misses         45383    45425      +42     
- Partials        5990     6015      +25     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Enhancement] massive memory allocation in the parsePublishMessageQueues

3 participants