[ISSUE #10373] Quarantine flaky tests and add detection plan docs#10374
Merged
RongtongJin merged 3 commits intoMay 25, 2026
Conversation
Ran all RocketMQ module tests 100x across 10 ECS nodes to identify non-deterministic failures. Quarantined methods with @ignore across broker, client, filter, and tieredstore modules. Flaky tests quarantined: - broker: LiteLifecycleManagerTest#testCleanByParentTopic (2%) - broker: ConsumerOrderInfoManagerLockFreeNotifyTest#testRecover (2%) - broker: TransactionalMessageServiceImplTest#testDeletePrepareMessage_maxSize (1%) - client: DefaultMQConsumerWithTraceTest#testPullMessage_WithTrace_Success (1%) - client: DefaultMQLitePullConsumerWithTraceTest#testSubscribe_PollMessageSuccess_WithCustomizedTraceTopic (5%) - client: DefaultMQLitePullConsumerWithTraceTest#testSubscribe_PollMessageSuccess_WithDefaultTraceTopic (6%) - filter: BloomFilterTest#testCheckFalseHit (1%) - tieredstore: IndexStoreServiceTest#queryCrossFileBoundaryTest (35%) - tieredstore: IndexStoreServiceTest#concurrentGetTest (1.5%) Additional changes: - LiteLifecycleManagerTest: Switch to MockitoJUnitRunner.Silent - Add flaky test detection plan docs (CN + EN) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #10374 +/- ##
=============================================
- Coverage 48.98% 48.79% -0.19%
+ Complexity 13482 13430 -52
=============================================
Files 1376 1376
Lines 100539 100539
Branches 12983 12983
=============================================
- Hits 49244 49057 -187
- Misses 45287 45471 +184
- Partials 6008 6011 +3 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- Quarantine PopPriorityIT at class level (multiple methods fail intermittently with 'expected:<8> but was:<2>' due to async race) - Fix ConsumerOrderInfoManagerLockFreeNotifyTest - Fix IndexStoreServiceTest Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
lizhimins
force-pushed
the
zhimin/quarantine-flaky-tests
branch
from
5633427 to
4ae4e39
Compare
Move English doc from docs/cn/ to docs/en/ and rename both files to match existing docs naming convention (underscore + PascalCase). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
RongtongJin
approved these changes
May 25, 2026
4 tasks
lizhimins
added a commit
to lizhimins/rocketmq
that referenced
this pull request
May 26, 2026
…workflow Fix root causes of flaky tests quarantined in apache#10374: - BloomFilterTest#testCheckFalseHit: use single seeded Random instance instead of per-character Random(System.nanoTime()) which produced duplicate strings in tight loops - TransactionalMessageServiceImplTest#testDeletePrepareMessage_maxSize: increase verify timeout from 50ms to 3000ms to accommodate slow thread scheduling - DefaultMQConsumerWithTraceTest#testPullMessage_WithTrace_Success: call pullMessage directly instead of async PullMessageService to eliminate race condition - DefaultMQLitePullConsumerWithTraceTest: set RebalanceService.waitInterval as static field in @before to avoid instance-level race condition Also remove rerun-workflow.yml to stop masking flaky tests with automatic CI retries.
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#10373
@Ignoreannotations across broker, client, filter, and tieredstore modulesdocs/cn/LiteLifecycleManagerTesttoMockitoJUnitRunner.Silentto fix unnecessary stubbing error after@IgnoreMotivation
Flaky tests cause intermittent CI failures that erode developer trust in build signals. These methods were identified by running the full test suite 100× across 10 ECS nodes using a three-layer funnel (module → class → method). Methods with ≥1% failure rate are quarantined.
Test plan
@Ignoreannotations include failure rate metadata