Skip to content

test: fix flaky SpringAnnotationCompatibilityTest.shouldSupportAnnotationAndMultipleConfig#142

Open
Shawyeok wants to merge 1 commit into
apolloconfig:mainfrom
Shawyeok:fix/flaky-spring-annotation-compat-test
Open

test: fix flaky SpringAnnotationCompatibilityTest.shouldSupportAnnotationAndMultipleConfig#142
Shawyeok wants to merge 1 commit into
apolloconfig:mainfrom
Shawyeok:fix/flaky-spring-annotation-compat-test

Conversation

@Shawyeok

@Shawyeok Shawyeok commented Jun 11, 2026

Copy link
Copy Markdown

What

Fixes a flaky test that sometimes fails CI with:

SpringAnnotationCompatibilityTest.shouldSupportAnnotationAndMultipleConfig:126 expected:<application[]> but was:<application[.yaml]>

Failed runs:

Why it was flaky

The test changes config for several namespaces. Each change sends a Spring event, and all events go into one shared queue. The test then checks that the first event in the queue is for namespace application.

But Apollo sends these events on background threads (AbstractConfig#notifyAsync uses a thread pool), so the order they arrive is random. Most of the time application arrives first and the test passes. Sometimes application.yaml wins the race and the test fails.

I confirmed this by logging the events: even when the application.yaml change was fired first, the application event could still arrive first — the order depends only on thread scheduling.

Fix

Stop checking the order. Instead, wait until all expected namespaces (application, TEST1.apollo, application.yaml) have been seen, in any order. This is the same approach already used in ApolloSpringBootCompatibilityTest.

Testing

  • Ran the fixed suite 15 times with Spring 3.1.1 + JDK 8 (the failing CI matrix) — all passed
  • Ran 5 times with the event order deliberately reversed — all passed
  • Ran once with Spring 6.1.18 + JDK 17 — passed

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests
    • Improved Spring compatibility tests to accept namespace update events arriving in any order across multiple configurations.
    • Updated test probe to track which namespaces have been seen and to provide non-blocking checks for namespace arrival (removed reliance on blocking/polling semantics).

@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8a2978aa-4a6c-4b65-8d86-29eb7be2f2df

📥 Commits

Reviewing files that changed from the base of the PR and between d07caef and f9acc43.

📒 Files selected for processing (2)
  • apollo-compat-tests/apollo-spring-compat-it/src/test/java/com/ctrip/framework/apollo/compat/spring/SpringAnnotationCompatibilityTest.java
  • apollo-compat-tests/apollo-spring-compat-it/src/test/java/com/ctrip/framework/apollo/compat/spring/SpringApolloEventListenerProbe.java
🚧 Files skipped from review as they are similar to previous changes (2)
  • apollo-compat-tests/apollo-spring-compat-it/src/test/java/com/ctrip/framework/apollo/compat/spring/SpringAnnotationCompatibilityTest.java
  • apollo-compat-tests/apollo-spring-compat-it/src/test/java/com/ctrip/framework/apollo/compat/spring/SpringApolloEventListenerProbe.java

📝 Walkthrough

Walkthrough

The PR replaces queue-based, blocking namespace polling with set-based tracking in SpringApolloEventListenerProbe and updates the test to asynchronously wait until the probe has observed the three configured namespaces in any order.

Changes

Event listener probe refactoring for async event ordering

Layer / File(s) Summary
Event listener probe storage and API
apollo-compat-tests/apollo-spring-compat-it/src/test/java/com/ctrip/framework/apollo/compat/spring/SpringApolloEventListenerProbe.java
namespaces changes from BlockingQueue<String> to a synchronized Set<String>. The event handler adds namespace strings to the set. pollNamespace(long timeout, TimeUnit) is removed and replaced with hasNamespace(String) for non-blocking checks.
Test async event verification
apollo-compat-tests/apollo-spring-compat-it/src/test/java/com/ctrip/framework/apollo/compat/spring/SpringAnnotationCompatibilityTest.java
shouldSupportAnnotationAndMultipleConfig replaces blocking queue polling with waitForCondition that confirms the probe has received application, TEST1.apollo, and application.yaml, allowing any arrival order.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

I’m a rabbit in the test suite burrow,
I hop and catch namespaces in a row,
No blocking queues to slow my pace,
Sets record each name in any case,
Async green lights make the CI glow. 🐇✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: fixing a flaky test by removing order-dependent assertions and implementing order-agnostic event verification.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.16%. Comparing base (d4b76f8) to head (f9acc43).
⚠️ Report is 27 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main     #142      +/-   ##
============================================
+ Coverage     68.68%   71.16%   +2.48%     
- Complexity     1503     1639     +136     
============================================
  Files           212      224      +12     
  Lines          6396     6732     +336     
  Branches        647      680      +33     
============================================
+ Hits           4393     4791     +398     
+ Misses         1673     1592      -81     
- Partials        330      349      +19     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Shawyeok Shawyeok changed the title test: fix flaky SpringAnnotationCompatibilityTest test: fix flaky SpringAnnotationCompatibilityTest.shouldSupportAnnotationAndMultipleConfig Jun 11, 2026
The test asserted that the first ApolloConfigChangeEvent received by the
ApplicationListener probe is for namespace 'application'. However, config
change listeners are notified asynchronously on a shared thread pool
(AbstractConfig#notifyAsync), so events from different namespaces may
arrive in any order, occasionally failing CI with:

  expected:<application[]> but was:<application[.yaml]>

Replace the order-sensitive FIFO assertion with an order-independent
check that all expected namespaces are eventually observed, matching the
approach already used in ApolloSpringBootCompatibilityTest.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Shawyeok Shawyeok force-pushed the fix/flaky-spring-annotation-compat-test branch from d07caef to f9acc43 Compare June 11, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant