Skip to content

fix(e2e): dedupe automine test cmds causing mass container-kill flakes#24547

Merged
vezenovm merged 1 commit into
mv/ci-docker-name-race-retryfrom
mv/fix-automine-flakes
Jul 6, 2026
Merged

fix(e2e): dedupe automine test cmds causing mass container-kill flakes#24547
vezenovm merged 1 commit into
mv/ci-docker-name-race-retryfrom
mv/fix-automine-flakes

Conversation

@vezenovm

@vezenovm vezenovm commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Stacked on #24477. Fixes the 49-per-run automine FLAKEs reported there and on this PR's first run.

Root cause (yarn-project/end-to-end/bootstrap.sh): the tests array listed the automine subdirectories explicitly and also kept the src/automine/!(simulation)/**/*.test.ts catch-all (a merge artifact from merging merge-train/fairies-v5 into mv/e2e-onchain-delivery-harness), so every nested automine test was emitted as two byte-identical cmds (268 jobs for 145 tests). Both copies run under the same docker container name. When the two copies overlap in time, the later one's docker_isolate force-removes the earlier one's live container (or wins the name-conflict retry added in #24477), the earlier test fails, and its automatic retry (which gets a fresh NAME_POSTFIX) passes: a deterministic FLAKED-with-code-0 for every overlapping pair, which is why the count sits near 49 on every run and why only src/automine/** flakes. When the copies do not overlap, the second is skipped by the test cache, hiding the duplication. The parent branch's "docker container name conflicts" were the same duplication seen from the other side.

@vezenovm vezenovm added ci-full Run all master checks. ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure labels Jul 6, 2026
@vezenovm vezenovm changed the title fix(pxe): retry settled-read witness fetches and self-diagnose misses in kernel reset fix(e2e): dedupe automine test cmds causing mass container-kill flakes Jul 6, 2026
The tests array listed the automine subdirectories explicitly and also kept the
src/automine/!(simulation)/**/*.test.ts catch-all (a merge artifact), so every nested automine test
was emitted as two identical cmds. Both copies run under the same docker container name; when they
overlap, one docker_isolate force-removes the other's live container, failing that test, and the
fresh-named retry then passes. That is the mass automine FLAKED pattern (49 per run) on this stack.
The deduplicated unique cmd set is unchanged (145 automine jobs).
@vezenovm
vezenovm force-pushed the mv/fix-automine-flakes branch from 458e0dc to a7e6a40 Compare July 6, 2026 17:10
@vezenovm
vezenovm merged commit f80d7e4 into mv/ci-docker-name-race-retry Jul 6, 2026
10 of 12 checks passed
@vezenovm
vezenovm deleted the mv/fix-automine-flakes branch July 6, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-full Run all master checks. ci-no-fail-fast Sets NO_FAIL_FAST in the CI so the run is not aborted on the first failure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant