Skip to content

Ranged Ingredient Test Failures#4884

Open
DilithiumThoride wants to merge 7 commits into
1.20.1from
dt/deranged-failures
Open

Ranged Ingredient Test Failures#4884
DilithiumThoride wants to merge 7 commits into
1.20.1from
dt/deranged-failures

Conversation

@DilithiumThoride
Copy link
Copy Markdown
Contributor

@DilithiumThoride DilithiumThoride commented May 29, 2026

What

Batched and Parallel Ranged Ingredient Tests sometimes fail for no apparent reason. Trying to replicate these failures locally has proved unreliable at best, with the failures being extremely inconsistent, and refusing to even log what they were.
We believe that the logging failures come from #4538 and the attempts param; running tests with higher attempts count does not print the assert log entries on failure.

In an effort to find more information I've added an extra step of logging to the suspect tests based on This Failed Test Report as an exemplar.

Implementation Details

  • attempts param removed from ranged test cases, reverting Re-run ranged input tests multiple times #4538
  • Batched and Parallel Ranged Ingredient tests now print a quite lengthy log message if they fail to complete a recipe, listing the controller's current state and last set of recipe failures and reasons
  • B/P RI tests now have a much longer Timeout clock on them, a few ticks buffer between checks, and run higher iteration counts
  • Also fixed an unrelated logging error in the Assembly Line tests, which was encountered on accident while trying to make the Ranged tests fail.

AI Usage

  • [ X ] Yes AI driven tools were used for this pull request.

Agent Used

Claude, no further details as Jurre ran it on his end.

Agent Usage Description

Jurre fed the test code into Claude and asked it to figure out why it was generating so many silent failures. Claude and Jurre came back with the suspect conclusions of the attempts param, and overly tight timings on test assertions. This was taken into advisement when making subsequent code changes, however No machine generated code was used in creating the PR.

Outcome

Fewer spurious automated test failures, and/or more informative ones when they do fail?

How Was This Tested

Multiple hours of trying to make the tests fail locally (Gametests run, Gametests debug, deleting world between cycles, not deleting world between cycles) which failed to fail

@DilithiumThoride DilithiumThoride requested a review from a team as a code owner May 29, 2026 07:40
@DilithiumThoride DilithiumThoride added ignore changelog PR should not be added to the changelog. 1.20.1 type: tests Test related changes Release: Patch - 0.0.X Smaller changes that either are bug fixes or very minor tweaks. labels May 29, 2026
@github-actions github-actions Bot added the Tests: Failed Game Tests have failed on this PR label May 29, 2026
…ing failure logging. Increase multiblock run count to compensate. Extend output check delay slightly.
@github-actions github-actions Bot added Tests: Passed Game Tests have passed on this PR and removed Tests: Failed Game Tests have failed on this PR labels May 30, 2026
Copy link
Copy Markdown
Contributor

@jurrejelle jurrejelle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hoping we get some clarity as to why, if this doesn't solve it 🫡

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.20.1 ignore changelog PR should not be added to the changelog. Release: Patch - 0.0.X Smaller changes that either are bug fixes or very minor tweaks. Tests: Passed Game Tests have passed on this PR type: tests Test related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants