Skip to content

Commit 71be244

Browse files
os-zhuangclaude
andauthored
ci(test): shard the Vitest suite across 4 runners (#2641)
The Test job is the whole of CI's critical path: ~8.3 min of a ~9 min wall clock. Its cost is almost entirely fixed per-file overhead rather than the assertions — a recent PR run reported 495s of which only 178s was `tests`, against `setup` 831s, `import` 199s and `environment` 156s (cumulative across workers). Those are paid once per test file because `isolate: true` re-executes the module graph for every file, and isolation has to stay on (vitest.config.mts records the order-dependent failures that turning it off caused). So rather than touch isolation, split the 559 files across 4 runners: - test: a 4-shard matrix, PR-only, running `pnpm test --shard=N/4`. Fixed overhead per job here is only ~30s (checkout + Node + install off a warm pnpm store), so 4 shards trade ~26% more runner minutes for ~3x less wall clock. fail-fast: false so one broken shard doesn't mask the others. - test-coverage: the former push-only coverage path, split into its own unsharded job. Nothing blocks on it, so it keeps feeding Codecov one complete report instead of needing a blob-report merge. Vitest shards by hashing each test file's path, so unit/dom files interleave and the shards self-balance. Verified that `vitest run --shard` really splits and conserves the file/test totals across projects — note `vitest list` ignores --shard, so it cannot be used to check this. Nothing (branch protection, `needs:`, workflow_run) references the old `Test` check name, so the rename is safe. Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
1 parent 12e980c commit 71be244

1 file changed

Lines changed: 58 additions & 13 deletions

File tree

.github/workflows/ci.yml

Lines changed: 58 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,33 @@ jobs:
3535
- name: Verify all packages are in changeset fixed group
3636
run: node scripts/check-changeset-fixed.mjs
3737

38+
# PRs run the suite split across 4 runners. The suite is dominated by fixed
39+
# per-file cost rather than by the assertions: a PR run reported 495s wall
40+
# clock of which only 178s was `tests` — the rest was `setup` (831s
41+
# cumulative), `import` (199s) and `environment` (156s), each paid once per
42+
# test file because `isolate: true` re-executes the module graph per file
43+
# (vitest.config.mts explains why isolation has to stay on). That cost shards
44+
# near-linearly, and the fixed overhead of an extra runner here is only ~30s
45+
# (checkout + Node + a warm-pnpm-store install), so 4 shards cut wall clock
46+
# ~3x for ~26% more runner minutes.
47+
#
48+
# Vitest shards by hashing each test file's path, so `unit` and `dom` files
49+
# interleave across shards on their own — no manual balancing needed.
3850
test:
39-
name: Test
51+
name: Test (shard ${{ matrix.shard }}/4)
52+
# Coverage instrumentation (v8 adds 40-100% overhead) is skipped on PRs;
53+
# the `test-coverage` job below keeps Codecov current on push.
54+
if: github.event_name == 'pull_request'
4055
runs-on: ubuntu-latest
4156
# Bound the job so a stalled runner / non-exiting test worker fails fast and
42-
# is retryable, instead of hanging up to GitHub's 6h default. PR runs can
43-
# spend extra time rebuilding affected package graphs on cold caches.
44-
timeout-minutes: 40
57+
# is retryable, instead of hanging up to GitHub's 6h default.
58+
timeout-minutes: 20
59+
60+
strategy:
61+
# Report every shard's failures, not just the first one to break.
62+
fail-fast: false
63+
matrix:
64+
shard: [1, 2, 3, 4]
4565

4666
steps:
4767
- name: Checkout code
@@ -64,22 +84,47 @@ jobs:
6484
- name: Install dependencies
6585
run: pnpm install --frozen-lockfile
6686

67-
# PRs: skip coverage instrumentation (v8 adds 40-100% overhead) and
68-
# run the canonical root Vitest project once. Running `turbo run test`
87+
# Run the canonical root Vitest project once. Running `turbo run test`
6988
# here starts one Vitest process per package; several package configs
7089
# intentionally inherit the root monorepo project list, so CI ends up
7190
# repeating large chunks of the suite and can starve slower plugin tests.
72-
- name: Run tests (PR — fast, no coverage)
73-
if: github.event_name == 'pull_request'
74-
run: pnpm test
91+
- name: Run tests (shard ${{ matrix.shard }}/4)
92+
run: pnpm test --shard=${{ matrix.shard }}/4
93+
94+
# Push to main/develop: one unsharded run so Codecov still receives a single
95+
# complete report. Nothing blocks on this job, so it isn't worth the blob
96+
# report merge that sharding a coverage run would require.
97+
test-coverage:
98+
name: Test (coverage)
99+
if: github.event_name == 'push'
100+
runs-on: ubuntu-latest
101+
timeout-minutes: 40
102+
103+
steps:
104+
- name: Checkout code
105+
uses: actions/checkout@v7
106+
with:
107+
submodules: true
108+
109+
- name: Enable Corepack
110+
run: corepack enable
111+
112+
- name: Verify pnpm version
113+
run: pnpm --version
114+
115+
- name: Setup Node.js
116+
uses: actions/setup-node@v6
117+
with:
118+
node-version: '20.x'
119+
cache: 'pnpm'
120+
121+
- name: Install dependencies
122+
run: pnpm install --frozen-lockfile
75123

76-
# Push to main/develop: still run coverage so Codecov stays current.
77-
- name: Run tests with coverage (push)
78-
if: github.event_name == 'push'
124+
- name: Run tests with coverage
79125
run: pnpm test:coverage --coverage.reporter=json --coverage.reporter=text
80126

81127
- name: Upload coverage to Codecov
82-
if: github.event_name == 'push'
83128
uses: codecov/codecov-action@v7
84129
with:
85130
fail_ci_if_error: false

0 commit comments

Comments
 (0)