Skip to content

Reduce flakiness in WildflyTest.testWildflyMetrics(String)#18521

Closed
trask wants to merge 1 commit into
open-telemetry:mainfrom
trask:otelbot/flaky-test-remediation-io-opentelemetry-instrumentation-jmx-rules-WildflyTest-testW-20260502212858
Closed

Reduce flakiness in WildflyTest.testWildflyMetrics(String)#18521
trask wants to merge 1 commit into
open-telemetry:mainfrom
trask:otelbot/flaky-test-remediation-io-opentelemetry-instrumentation-jmx-rules-WildflyTest-testW-20260502212858

Conversation

@trask

@trask trask commented May 2, 2026

Copy link
Copy Markdown
Member

Automated attempt at fixing flakiness in io.opentelemetry.instrumentation.jmx.rules.WildflyTest.testWildflyMetrics(String)[2].

Flake history (per UTC day)

Day flaky failed passed
2026-04-25 1 0 347
2026-04-26 0 0 534
2026-04-27 1 0 505
2026-04-28 2 0 695
2026-04-29 0 0 706
2026-04-30 5 0 723
2026-05-01 2 0 418
2026-05-02 1 0 236

Sample failure (from Develocity)

(no failure message captured)

Copilot diagnosis

Root cause

WildflyTest waited only for port 8080 to become available before starting metric verification. Local container logs show WildFly can open the Undertow listener before testapp.war is deployed and before the server emits WFLYSRV0025, while several asserted JMX metrics depend on the deployed web application and fully initialized WildFly subsystems. On slower or more loaded CI workers, that readiness gap can leave the JMX metric registration/export cycle racing the application deployment.

Fix

  • Changed the WildFly container wait strategy from a generic listening-port check to WildFly's WFLYSRV0025 server-started log message.
  • Kept the existing startup timeout, exposed port, metric assertions, and both WildFly image parameters unchanged.

Why this addresses the root cause

The WFLYSRV0025 line is emitted after WildFly has deployed testapp.war, resumed the server, and completed startup, which is the state the metric assertions require. Waiting for this application-level readiness signal removes the race where verification starts while deployment-specific MBeans are still appearing, without relaxing or deleting any metric coverage.

Risks / follow-ups

  • If WildFly changes the startup log code/message format in a future image, this test will fail at container startup instead of timing out during metric verification.
  • Maintainers may want to validate this on CI workers with slower Docker startup, since the new wait exposes genuine WildFly startup hangs more directly.

Review the diagnosis and the diff carefully before merging - automated fixes can mask flakiness instead of addressing the root cause.

…yTest.testWildflyMetrics(String)[2]

Automated fix attempt based on Develocity flaky-test analysis.
@trask

trask commented May 2, 2026

Copy link
Copy Markdown
Member Author

Closing, duplicate

@trask trask closed this May 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant