Skip to content

Commit 8201070

Browse files
fi3eworkfi3ework
authored
chore(eval): add migrate-to-rstest eval and report (#54)
1 parent 37e3921 commit 8201070

3 files changed

Lines changed: 257 additions & 1 deletion

File tree

‎skills-lock.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ skills:
1515
resolution:
1616
type: link
1717
path: dev-skills/rstack-skill-evaluator
18-
digest: sha256-75d5a6eca2ceabb492a4680ae05ef7fdb5b124c39c5d2007ea34cb8a314d7478
18+
digest: ""
1919
pr-creator:
2020
specifier: link:./skills/pr-creator
2121
resolution:

‎skills-test/migrate-to-rstest/evals/evals.json‎

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
{
2+
"skill_name": "migrate-to-rstest",
3+
"fixture_root": "/tmp/agent-skills-evals/migrate-to-rstest/fixtures",
4+
"runs_root": "/tmp/agent-skills-evals/migrate-to-rstest/runs",
5+
"runner_instructions": "Fixtures are intentionally NOT committed to the repo (5 fixtures including pnpm monorepos with React + MSW would balloon it) and are scratch-grade by design — fixture_root and runs_root must be absolute paths under /tmp (or an OS scratch dir). The runner agent's contract: (1) before running any eval, check whether fixture_root exists and contains a subdirectory per evals[].fixture; (2) if any fixture is missing, generate it from that eval's pre_migration_baseline (framework, test counts, coverage_provider, extra_patterns) plus the top-level notes — both fields together fully describe the fixture shape; (3) verify the generated fixture runs green pre-migration (tests_passed / coverage_threshold_met must hold) before grading; (4) reuse existing fixtures across runs — do NOT regenerate unless the user explicitly asks. The same applies to runs_root: each grading run writes a fresh subdirectory under it; never reuse a previous run's working tree.",
6+
"notes": "Fixtures use realistic production patterns: ts-jest, vi.mock with factories, vi.hoisted, @testing-library/jest-dom adapter, coverage thresholds, clearMocks/mockReset, path aliases. They also exercise patterns NOT covered by the official rstest migration docs: __mocks__ directory automock, CSS modules via identity-obj-proxy, React + @vitejs/plugin-react + MSW, workspace cross-package imports, Jest multi-project config with globalSetup/globalTeardown, project-scoped displayName + testEnvironmentOptions, polyfills via setupFiles split. Every fixture's test script runs with coverage and enforces thresholds — post-migration coverage must also meet those thresholds.",
7+
"evals": [
8+
{
9+
"id": 1,
10+
"eval_name": "jest-basic",
11+
"fixture": "jest-basic",
12+
"pre_migration_baseline": {
13+
"framework": "jest + ts-jest",
14+
"test_files": 5,
15+
"tests_passed": 12,
16+
"tests_failed": 0,
17+
"coverage_threshold_met": true,
18+
"coverage_provider": "istanbul (jest default)",
19+
"extra_patterns": [
20+
"__mocks__/logger.ts manual mock via bare jest.mock()",
21+
"CSS module imports via identity-obj-proxy in moduleNameMapper",
22+
"transformIgnorePatterns for ESM deps"
23+
]
24+
},
25+
"prompt": "Migrate the ts-jest project to Rstest. The project uses jest + ts-jest + @types/jest + TypeScript, has clearMocks, moduleNameMapper (including identity-obj-proxy for CSS modules), transformIgnorePatterns, and coverageThreshold set in jest.config.js. Tests include jest.fn, jest.spyOn, jest.mock with factory, a bare jest.mock() that pulls from a src/__mocks__/ manual-mock directory, and CSS module imports. The `test` script runs `jest --coverage` and enforces the thresholds. Migrate to Rstest preserving all behavior AND coverage threshold enforcement. Keep Jest deps until Rstest is green.",
26+
"assertions": [
27+
"rstest.config.ts (or equivalent) exists",
28+
"package.json replaces ts-jest/jest with @rstest/core (+ @rstest/coverage-istanbul) as devDeps",
29+
"scripts.test invokes rstest with coverage enabled",
30+
"moduleNameMapper alias translated to rstest alias",
31+
"CSS module moduleNameMapper translated (CSS imports still resolve to identity-like string map or equivalent)",
32+
"coverageThreshold translated to rstest coverage thresholds (lines>=80, functions>=80, branches>=70, statements>=80)",
33+
"jest.<api> usages literally replaced with rstest.<api> in test files (no globalThis.jest shim)",
34+
"jest.mock factory translated to rstest.mock equivalent",
35+
"Bare jest.mock('@app/logger') + src/__mocks__/logger.ts pattern still works (either preserved or equivalent auto-mock setup)",
36+
"jest deps and jest.config.js not deleted before Rstest is green",
37+
"Post-migration pnpm test: 12 tests pass across 5 files with coverage thresholds met"
38+
]
39+
},
40+
{
41+
"id": 2,
42+
"eval_name": "vitest-basic",
43+
"fixture": "vitest-basic",
44+
"pre_migration_baseline": {
45+
"framework": "vitest",
46+
"test_files": 3,
47+
"tests_passed": 6,
48+
"tests_failed": 0,
49+
"coverage_threshold_met": true,
50+
"coverage_provider": "v8"
51+
},
52+
"prompt": "Migrate the Vitest (globals mode) project to Rstest. The config has coverage (v8 provider with thresholds), setupFiles, clearMocks, mockReset, @app alias. Tests use vi.fn/spyOn/useFakeTimers and vi.mock with factory + vi.mocked for module mocking. `test` script runs `vitest run --coverage`. Migrate preserving behavior AND coverage thresholds. Rstest's only coverage provider is @rstest/coverage-istanbul. Keep Vitest deps until Rstest is green.",
53+
"assertions": [
54+
"rstest.config.ts exists",
55+
"package.json replaces @vitest/coverage-v8 with @rstest/coverage-istanbul and vitest with @rstest/core",
56+
"scripts.test invokes rstest with coverage enabled",
57+
"@app alias preserved in rstest config",
58+
"clearMocks/mockReset preserved",
59+
"coverage thresholds translated (istanbul provider)",
60+
"Global vi.<api> replaced with rs.<api> (no globalThis.vi shim)",
61+
"vi.mock / vi.mocked translated to rs.mock / rs.mocked correctly",
62+
"describe/it name strings preserved verbatim",
63+
"setupFiles path preserved",
64+
"vitest deps not deleted before Rstest is green",
65+
"Post-migration pnpm test: 6 tests pass across 3 files with coverage thresholds met"
66+
]
67+
},
68+
{
69+
"id": 3,
70+
"eval_name": "vitest-multiproject",
71+
"fixture": "vitest-multiproject",
72+
"pre_migration_baseline": {
73+
"framework": "vitest",
74+
"test_files": 8,
75+
"tests_passed": 16,
76+
"tests_failed": 0,
77+
"coverage_threshold_met": true,
78+
"coverage_provider": "v8",
79+
"projects": ["utils (node)", "web (jsdom + React + MSW + jest-dom)"],
80+
"extra_patterns": [
81+
"@fixture/shared workspace package via pnpm workspace:* (source and test-level imports)",
82+
"web uses @vitejs/plugin-react for React+TSX",
83+
"web setup: MSW setupServer with HTTP handlers + server.use override + @testing-library/jest-dom/vitest adapter + afterEach cleanup",
84+
"web tests: @testing-library/react render/findByTestId/waitFor, snapshot, vi.hoisted + vi.mock"
85+
]
86+
},
87+
"prompt": "Migrate this pnpm monorepo from Vitest to Rstest entirely. Root vitest.config.ts holds coverage config (v8 + thresholds); vitest.workspace.ts lists two test projects (utils/web). A third workspace package @fixture/shared is a library consumed by utils/ and web/ via pnpm workspace:* — its source (normalize) is imported by utils/slug.ts and web/id.ts, and one of its helpers (expectIdShape) is imported by web/tests/id.test.ts. packages/utils uses node env; packages/web uses jsdom + @vitejs/plugin-react for a React+TSX component, MSW setupServer (with server.use override in a test), @testing-library/jest-dom/vitest adapter in the setup file, @testing-library/react render/findByTestId/waitFor, vi.hoisted + vi.mock, snapshot. Per Rstest convention, `projects` goes at top level of defineConfig (not under test). @fixture/shared must continue to resolve correctly from both source and test files after migration. Coverage thresholds must still be enforced post-migration (use @rstest/coverage-istanbul). Replace the @testing-library/jest-dom/vitest adapter with the Rstest-compatible expect.extend pattern. SWC handles TSX so @vitejs/plugin-react is not needed. Keep vitest until Rstest is green across both projects.",
88+
"assertions": [
89+
"Root rstest.config.ts with projects at top level of defineConfig",
90+
"Per-package rstest.config.ts with correct environment (utils=node, web=jsdom)",
91+
"Root package.json replaces @vitest/coverage-v8 with @rstest/coverage-istanbul and vitest with @rstest/core",
92+
"scripts.test invokes rstest with coverage enabled",
93+
"coverage thresholds translated (istanbul provider)",
94+
"Test imports changed from 'vitest' to '@rstest/core'; vi -> rs (not rstest); no aliased shim (no `rs as vi`, no `globalThis.vi = rs`, no `const vi = rs`)",
95+
"vi.hoisted + vi.mock translated correctly",
96+
"describe/it names preserved verbatim",
97+
"@testing-library/jest-dom/vitest adapter replaced with expect.extend(matchers) pattern",
98+
"afterEach DOM cleanup preserved",
99+
"React + TSX component renders under jsdom in Rstest (SWC handles TSX; @vitejs/plugin-react removed)",
100+
"MSW setupServer lifecycle (beforeAll/afterEach/afterAll + server.use) still works",
101+
"snapshot file preserved; tests still match byte-for-byte",
102+
"@fixture/shared workspace package still resolves from source (utils/slug.ts, web/id.ts) and test (web/tests/id.test.ts) imports",
103+
"vitest deps/configs not deleted before Rstest is green; old vitest.config.ts / vitest.workspace.ts / per-package vitest.config.ts removed after green",
104+
"Post-migration pnpm test: 16 tests pass across 8 files covering both projects with coverage thresholds met"
105+
]
106+
},
107+
{
108+
"id": 4,
109+
"eval_name": "vitest-multiproject-partial",
110+
"fixture": "vitest-multiproject-partial",
111+
"pre_migration_baseline": {
112+
"framework": "vitest",
113+
"test_files": 6,
114+
"tests_passed": 12,
115+
"tests_failed": 0,
116+
"coverage_threshold_met": true,
117+
"coverage_provider": "v8",
118+
"packages": [
119+
"core (node, 2 tests)",
120+
"api (node, 4 tests, vi.mock in handler test)",
121+
"ui (jsdom, 2 tests)",
122+
"web (jsdom, 4 tests, vi.mock + jest-dom)"
123+
],
124+
"migration_target": "packages/web only"
125+
},
126+
"prompt": "Migrate ONLY packages/web of this pnpm monorepo from Vitest to Rstest. packages/core, packages/api, packages/ui must stay on Vitest. Root vitest.shared.ts is imported by every package via mergeConfig and contains include / testTimeout / globals / clearMocks / coverage (v8 with thresholds). Each package's test script runs with coverage. packages/web uses jsdom + @testing-library/jest-dom/vitest setup + vi.mock for id.ts. Translate web's configuration to a standalone packages/web/rstest.config.ts (should NOT import from vitest/config). Rstest needs @rstest/coverage-istanbul since Rstest doesn't support v8 coverage. Root vitest devDep and @vitest/coverage-v8 must stay (other packages still need them). Keep packages/web/vitest.config.ts and vitest.setup.ts until Rstest is green.",
127+
"assertions": [
128+
"packages/web/rstest.config.ts exists standalone (no import from vitest/config)",
129+
"packages/web has jsdom env, coverage thresholds (istanbul provider), setupFiles, clearMocks inlined from shared config",
130+
"packages/web/package.json test script uses rstest with coverage enabled",
131+
"packages/web adds @rstest/core and @rstest/coverage-istanbul devDeps",
132+
"packages/core, packages/api, packages/ui completely untouched (no file diff, excluding coverage/)",
133+
"Root vitest.shared.ts completely unchanged",
134+
"Root package.json still has vitest + @vitest/coverage-v8",
135+
"packages/web tests import '@rstest/core' (vi -> rs, no `rs as vi` alias), vi.mock -> rs.mock",
136+
"@testing-library/jest-dom/vitest adapter migrated to Rstest expect.extend pattern",
137+
"packages/web alone post-migration: 4 tests pass on Rstest with coverage thresholds met (istanbul)",
138+
"Root pnpm -r test: 12 tests pass across 6 files (web on Rstest + core/api/ui on Vitest) with all coverage thresholds met",
139+
"packages/web/vitest.config.ts and vitest.setup.ts removed after Rstest green"
140+
]
141+
},
142+
{
143+
"id": 5,
144+
"eval_name": "jest-multiproject",
145+
"fixture": "jest-multiproject",
146+
"pre_migration_baseline": {
147+
"framework": "jest + ts-jest",
148+
"test_files": 2,
149+
"tests_passed": 8,
150+
"tests_failed": 0,
151+
"coverage_threshold_met": true,
152+
"coverage_provider": "istanbul (jest default)",
153+
"projects": ["node-lib (node)", "dom-lib (jsdom)"],
154+
"extra_patterns": [
155+
"root jest.config.js uses `projects: [...]` to reference per-package configs",
156+
"root-level globalSetup / globalTeardown seeding process.env.__DB_SEED__",
157+
"per-project displayName (object form with color)",
158+
"per-project testEnvironmentOptions (url, userAgent, pretendToBeVisual) for dom-lib",
159+
"setupFiles (jest.polyfills.ts — matchMedia) split from setupFilesAfterEnv (jest.setup.ts — jest-dom matchers / custom matcher via expect.extend)"
160+
]
161+
},
162+
"prompt": "Migrate this pnpm monorepo Jest multi-project setup to Rstest. The root jest.config.js uses `projects: ['<rootDir>/packages/node-lib', '<rootDir>/packages/dom-lib']` plus `globalSetup` / `globalTeardown` (seeds and tears down `process.env.__DB_SEED__`) and a root `coverageThreshold`. Each project has its own jest.config.js with: displayName (object form, {name, color}), preset 'ts-jest', its own testEnvironment (node vs jsdom), setupFilesAfterEnv, clearMocks, collectCoverageFrom. dom-lib additionally sets testEnvironmentOptions (url: 'https://app.example.com/dashboard', userAgent: 'FixtureAgent/1.0', pretendToBeVisual: true), setupFiles (jest.polyfills.ts — polyfills matchMedia before framework) separate from setupFilesAfterEnv (jest.setup.ts — imports @testing-library/jest-dom). node-lib's jest.setup.ts both hydrates globalThis.__DB__ from process.env.__DB_SEED__ and registers a custom matcher via expect.extend (with declare global namespace jest for types). Migrate the whole setup to Rstest: projects top-level, globalSetup/globalTeardown preserved (Rstest globalSetup returns the teardown fn), jest-dom matchers via expect.extend, jest-specific types/namespaces replaced with Rstest equivalents, jest.<api> -> rstest.<api> if any appear. Coverage thresholds must still be enforced (use @rstest/coverage-istanbul). Keep Jest deps and configs until Rstest is green.",
163+
"assertions": [
164+
"Root rstest.config.ts with projects at top level of defineConfig",
165+
"Per-package rstest.config.ts with correct environment (node-lib=node, dom-lib=jsdom)",
166+
"Root package.json replaces jest + ts-jest + jest-environment-jsdom with @rstest/core + @rstest/coverage-istanbul",
167+
"scripts.test invokes rstest with coverage enabled",
168+
"coverage thresholds translated (istanbul provider)",
169+
"globalSetup + globalTeardown translated (seeding process.env.__DB_SEED__ still works; teardown deletes it)",
170+
"dom-lib jsdom url / userAgent / pretendToBeVisual options preserved in rstest config",
171+
"dom-lib setupFiles (matchMedia polyfill) runs before framework; setupFilesAfterEnv (jest-dom) runs after — both still wired",
172+
"node-lib setupFilesAfterEnv hydrates globalThis.__DB__ from env + registers custom matcher via expect.extend",
173+
"Jest types/namespaces for custom matcher (declare global namespace jest) migrated to Rstest-compatible declaration",
174+
"jest.<api> -> rstest.<api> literal replacement in any test code (no globalThis.jest shim, no `rstest as jest` alias)",
175+
"Jest deps/configs not deleted before Rstest is green; old jest.config.js / jest.global-setup.ts / jest.global-teardown.ts / per-package jest.config.js removed after green",
176+
"Post-migration pnpm test: 8 tests pass across 2 files covering both projects with coverage thresholds met"
177+
]
178+
}
179+
]
180+
}

0 commit comments

Comments
 (0)