fix(ci): pass --shard to vitest in the windows e2e matrix [RED-738] [ship]#1415
Merged
Conversation
The `e2e - checkly - windows` job declares a 4-way shard matrix, but all four shards ran the entire 27-file e2e suite. pnpm forwards a literal `--` verbatim into the script (npm strips it), where vitest's parser collects everything after it into an argument list it never reads — silently discarding --shard. Dropping the separator is sufficient: pnpm appends trailing args to the script command as-is, so --shard reaches vitest without one. The compound `test:e2e` script is not the cause. Args appended to `A && B` land at the end of B, and B is the vitest invocation. Verified locally that the four shards now partition the suite 7/7/7/6 into 27 unique files. A green run is not evidence on its own — the broken form passed too, running the full suite four times over. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linear: RED-738
The
e2e - checkly - windowsjob declares a 4-way shard matrix (1/4…4/4), but all four shards ran the entire 27-file e2e suite — the matrix bought nothing.Root cause
Not the compound
test:e2escript, which is what the symptom suggests. Args appended toA && Bland at the end ofB, andBis the vitest invocation, so they reach vitest fine.The sole cause is the literal
--. pnpm forwards it verbatim into the script (npm strips it), where vitest's parser collects everything after it into an argument list it never reads — so--shardis silently discarded rather than reinterpreted. That is exactly why the broken form ran green and complete instead of failing.Verified on pnpm 10.34.1, the version CI uses:
pnpm --filter thing run compound -- --shard=1/4["--", "--shard=1/4"]pnpm --filter thing run compound --shard=1/4["--shard=1/4"]pnpm does not intercept flags after the script name — even
--reporter, which pnpm itself defines — so no separator is needed.Also ruled out: passing the shard through an env var read by the e2e vitest config. Vitest's
--shardis CLI-only and has no config-file equivalent.Verification
A green run proves nothing here — the broken form passed too, running the full suite four times over. Two checks were done locally:
pnpm pack && cross-env NODE_CONFIG_DIR=./e2e/config vitest --run -c ./vitest.config.e2e.mts --shard=1/4, with no--before the flag.vitest listper shard yields a true partition — 27 unique files, no overlap, no omission:test.spec.tsdeploy.spec.tsTo confirm on this PR: the four
e2e - checkly - windows (n/4)logs should report file counts smaller than 27 summing to 27 (7/7/7/6). Shards 1–3 all reporting 7 is expected and correct — the discriminator is 7+7+7+6=27 against the broken 27+27+27+27=108.What this does and does not buy
The ticket originally claimed ~4x on both time and load. More precisely:
fileParallelism/maxWorkers, so ~4 files already run at once per job: peak is ~4 shards × ~4 forks either way. If the intermittent HTTP 500s from the test-session endpoint persist, that is not evidence this flag failed to land.deploy.spec.tsconcurrently, each doing an account-widecleanupProjects()sweep. Now only one does — 5 concurrent copies (4 Windows + Ubuntu) down to 2.deploy.spec.tssets the floor, andglobalSetupdoesn't shard: every shard still payspnpm packplus three template installs regardless of how few files it runs.Risk
This is the first time the suite has actually been split, so latent cross-file coupling surfaces here for the first time.
No ordering coupling exists: every mutating spec is self-isolating via
uuidv4()/nanoid()IDs, and vitest already runs files in parallel forks with no ordering guarantee. Butchecks-list.spec.ts,checks-get.spec.tsandchecks-stats.spec.tsassert on account-wide check counts and now run in shards withoutdeploy.spec.tspopulating the account. Failures isolated to those three specs indicate that assumption, not the flag. Rollback is a one-character revert.Other changes
None — single-file change.