Skip to content

Reduce flakiness in io.opentelemetry.instrumentation.jmx.rules.WildflyTest.testWildflyMetrics(String)[2]#18512

Closed
trask wants to merge 1 commit into
open-telemetry:mainfrom
trask:otelbot/flaky-test-remediation-io-opentelemetry-instrumentation-jmx-rules-WildflyTest-testW-20260502165717
Closed

Reduce flakiness in io.opentelemetry.instrumentation.jmx.rules.WildflyTest.testWildflyMetrics(String)[2]#18512
trask wants to merge 1 commit into
open-telemetry:mainfrom
trask:otelbot/flaky-test-remediation-io-opentelemetry-instrumentation-jmx-rules-WildflyTest-testW-20260502165717

Conversation

@trask

@trask trask commented May 2, 2026

Copy link
Copy Markdown
Member

Automated attempt at fixing flakiness in io.opentelemetry.instrumentation.jmx.rules.WildflyTest.testWildflyMetrics(String)[2].

Flake history (per UTC day)

Day flaky failed passed
2026-04-25 1 0 347
2026-04-26 0 0 534
2026-04-27 1 0 505
2026-04-28 2 0 695
2026-04-29 0 0 706
2026-04-30 5 0 723
2026-05-01 2 0 418
2026-05-02 1 0 236

Sample failure (from Develocity)

(no failure message captured)

Copilot diagnosis

Root cause

No failure payload or build scan was available, but the flaky parameter is the current WildFly image and the test source showed that container readiness only waited for port 8080 to listen. WildFly can bind the listener before the copied testapp.war has finished deploying and before the deployment-scoped Undertow MBeans used by the assertions are available. Under slower container starts, the JMX exporter can begin collecting before those MBeans exist, making the later metric polling race application deployment.

Fix

  • Replaced the WildFly wait strategy from a listening-port check to an HTTP readiness check against the deployed test application.
  • Added a small helper that selects /testapp/javax/ for the legacy WildFly 10 image and /testapp/jakarta/ for the current WildFly image, preserving coverage for both servlet API generations.
  • Kept the existing metric verifier and assertions unchanged.

Why this addresses the root cause

The test now starts metric verification only after WildFly has deployed the WAR and can serve the servlet endpoint, which is the same application deployment that creates the session-related MBeans asserted by the test. This removes the race between early port binding and delayed deployment while preserving the original end-to-end JMX metric coverage.

Risks / follow-ups

  • If WildFly legitimately fails to deploy the test WAR, the failure will now surface during container readiness instead of later as missing metrics.
  • Maintainers should watch for any future image layout or context-root changes that would require updating the readiness endpoint.

Review the diagnosis and the diff carefully before merging - automated fixes can mask flakiness instead of addressing the root cause.

…yTest.testWildflyMetrics(String)[2]

Automated fix attempt based on Develocity flaky-test analysis.
@trask

trask commented May 2, 2026

Copy link
Copy Markdown
Member Author

Closing as duplicate of #18509

@trask trask closed this May 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants