Skip to content

FIX : undeterministic tests #495

@bedaHovorka

Description

@bedaHovorka

The failure is in :desktop-ui:test, not the workflow itself. The workflow step simply runs:

  • .github/workflows/gradle-java21.yml at ee6ba87ede892ab9c978b43f03c4d82783cf132f, lines 78–83
    ./gradlew test --no-daemon --warning-mode=all --stacktrace

Code link:

- name: Run unit tests
timeout-minutes: 10
env:
GITHUB_ACTOR: ${{ github.actor }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: ./gradlew test --no-daemon --warning-mode=all --stacktrace

What failed

The logs show:

  • 563 tests completed, 1 failed, 1 skipped
  • Gradle then aborts with:
    Execution failed for task ':desktop-ui:test'.
  • The detailed failing test name is not present in the retrieved log excerpt, and the generated HTML/XML test report was not available from the repository snapshot.

Focused solution

This should be treated as a single failing desktop UI test, likely caused by:

  1. a brittle test depending on ordering/timing/state leakage, or
  2. a recent PR change affecting desktop-ui behavior while most of the suite still passes.

Since the failure is isolated to desktop-ui:test, the right fix is to stabilize or update the failing test rather than changing CI.

Recommended code-level fixes

Apply these patterns to the failing desktop UI test once identified from the local/CI test report:

1. Remove order dependence

If the test assumes collection iteration order, compare deterministically:

assertEquals(
    expected.sortedBy { it.id },
    actual.sortedBy { it.id }
)

or for strings:

assertContentEquals(
    expected.sorted(),
    actual.sorted()
)

2. Eliminate shared mutable state between tests

If the desktop UI tests reuse singleton/application state, reset setup per test:

@BeforeTest
fun setUp() {
    // create fresh presenter/view-model/controller for each test
}

@AfterTest
fun tearDown() {
    // clear observers, listeners, temp files, static/shared state
}

If a singleton or DI container is involved, avoid reusing it across tests unless explicitly reset.

3. Avoid timing-sensitive assertions

If the failing test depends on async UI updates, replace immediate assertions like:

controller.trigger()
assertEquals("Ready", statusBar.text)

with a deterministic synchronization point, e.g. inject a test dispatcher / run queued events before asserting.

For coroutine-based code:

runTest {
    controller.trigger()
    advanceUntilIdle()
    assertEquals("Ready", statusBar.text)
}

4. Avoid environment-specific path/text assertions

Linux CI often exposes separator, locale, or rendering differences. Prefer normalized assertions:

assertEquals(
    expected.trim().replace("\r\n", "\n"),
    actual.trim().replace("\r\n", "\n")
)

For file paths:

assertEquals(expected.normalize(), actual.normalize())

5. If the test checks exception/error text, assert key fragments only

Exact parser/UI messages can vary slightly:

val ex = assertFailsWith<IllegalArgumentException> {
    subject.doThing()
}
assertTrue(ex.message!!.contains("missing X attribute"))

instead of exact full-string equality.

What not to change

Do not change the workflow timeout or Gradle command. The workflow definition is already straightforward and the logs clearly indicate a normal test failure, not an infrastructure issue.

Best concrete patch direction

Once you open the failing desktop-ui test report, update the offending test along one of these lines:

  • sort expected/actual collections before equality checks
  • reinitialize UI/controller state in @BeforeTest
  • flush async work before asserting
  • normalize line endings / paths / messages
  • loosen overly strict exact-string assertions

If you share the failing test name or the corresponding desktop-ui/build/test-results/test/TEST-*.xml content, I can turn this into an exact patch.

Metadata

Metadata

Labels

bugSomething isn't working

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions