Retry Tests for Transient External Dependency / Infrastructure Failures

[Subhanshu20135]

Summary

Introduce a Retry Test feature aimed at test failures caused by transient infrastructure or external dependency issues (e.g., network hiccups, temporary service unavailability, throttling), rather than application logic defects. The goal is to reduce flaky CI outcomes while keeping true failures visible and actionable.

Problem / Motivation

Some test cases rely on external systems such as databases, message brokers, HTTP services, or cloud resources. These dependencies can intermittently fail due to infrastructure conditions, causing:

• False-negative test failures in CI
• Manual re-runs of entire test suites
• Reduced confidence in test signal and slower feedback loops

We need a controlled retry mechanism that:

• Retries only when the failure is likely transient
• Does not mask genuine logic or assertion failures
• Clearly reports when a test required retrying to pass

Goals

• Reduce CI flakiness caused by transient external dependency failures.
• Preserve correctness by avoiding retries for deterministic logic/assertion failures.
• Improve observability by recording retry attempts, reason, and final outcome.

Non-Goals

• Blindly retrying all test failures.
• Hiding flaky tests or dependency instability (retries must remain visible).
• Replacing proper test isolation, cleanup, and resilience improvements.

Proposed Solution

Implement a retry policy that applies to tests interacting with external dependencies, I would be happy to provide my solution if it aligns with the Project Direction

[mpkorstanje]

Based on #5397 I assume your proposed solution would be implemented as an extension. Have you considered that such an extension already exists in the form of JUnit Pioneer - Retry Tests?

Alternatively, for more complex decisions and reporting have you considered using the JUnit Platform Launcher API to programmatically decide which tests to re-run and how to report on them.

[mpkorstanje]

And I see we already have this on the backlog in the form of #1558. Currently waiting for interest.
Please express your interest there. Specifically describing your exact usecase would be helpful.

[Subhanshu20135]

Hi @mpkorstanje , I see you have mentioned that there exists an Issue for the Feature Request, and there has been some work done, but when I was going through the framework I did not find any implementation which implements this.

As per the discussion which is there I am assuming the use of Repeated Tests, which runs a single test n number of times pre-defined. My Implementation is for running the test more than once if it fails.

Also my use case is when I write test cases which connects to external APIs, this might be useful as there can be network/database/api hiccups which would lead to failure of test cases not the core business logic.

Hope this might have helped!