Skip to content

Reduce flakiness in WildflyTest.testWildflyMetrics(String)#18523

Closed
trask wants to merge 1 commit into
open-telemetry:mainfrom
trask:otelbot/flaky-test-remediation-io-opentelemetry-instrumentation-jmx-rules-WildflyTest-testW-20260502215239
Closed

Reduce flakiness in WildflyTest.testWildflyMetrics(String)#18523
trask wants to merge 1 commit into
open-telemetry:mainfrom
trask:otelbot/flaky-test-remediation-io-opentelemetry-instrumentation-jmx-rules-WildflyTest-testW-20260502215239

Conversation

@trask

@trask trask commented May 2, 2026

Copy link
Copy Markdown
Member

Automated attempt at fixing flakiness in io.opentelemetry.instrumentation.jmx.rules.WildflyTest.testWildflyMetrics(String)[2].

Flake history (per UTC day)

Day flaky failed passed
2026-04-25 1 0 347
2026-04-26 0 0 534
2026-04-27 1 0 505
2026-04-28 2 0 695
2026-04-29 0 0 706
2026-04-30 5 0 723
2026-05-01 2 0 418
2026-05-02 1 0 236

Sample failure (from Develocity)

(no failure message captured)

Copilot diagnosis

Root cause

The WildFly JMX test used Wait.forListeningPorts(8080), which can mark the container as ready as soon as the socket is open. WildFly can open the HTTP port before server startup, deployment processing, and JMX metric MBean registration have fully completed. On slower CI runs, the test could then spend its 60-second metric polling window observing incomplete JMX data and time out without a more specific failure message.

Fix

  • Changed WildflyTest to wait for WildFly's WFLYSRV0025: ... started in ... startup log message.
  • Kept the existing exposed port, startup timeout, metric verifier, YAML validation, deployed test webapp, and assertions unchanged.

Why this addresses the root cause

Waiting for WildFly's server-started log line aligns Testcontainers readiness with the app server lifecycle event that follows deployment and service initialization, rather than with the earlier TCP bind. This gives the existing Awaitility-based metric verifier a stable starting point while preserving the original metric coverage.

Risks / follow-ups

  • If WildFly changes the startup log format in a future image, the test may fail during container startup instead of during metric verification.
  • Maintainers should confirm the log wait remains compatible with both the legacy jboss/wildfly:10.1.0.Final image and the maintained WildFly image when those images are refreshed.

Review the diagnosis and the diff carefully before merging - automated fixes can mask flakiness instead of addressing the root cause.

…yTest.testWildflyMetrics(String)[2]

Automated fix attempt based on Develocity flaky-test analysis.
@trask

trask commented May 2, 2026

Copy link
Copy Markdown
Member Author

Closing as a duplicate automated flaky-remediation PR. WildflyTest is now recorded as a class-level skip in otelbot/flaky-test-remediation-progress.

@trask trask closed this May 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant