feat(core): experimental softMode — reuse workers with per-file env reset#1289
Closed
pkasarda wants to merge 9 commits into
Closed
feat(core): experimental softMode — reuse workers with per-file env reset#1289pkasarda wants to merge 9 commits into
pkasarda wants to merge 9 commits into
Conversation
Adds a third value for `RstestConfig.isolate`: `'soft'`. Workers persist
across files (like `isolate: false`), and rstest's narrow per-file core
cache clean still runs (so each file starts with fresh rstest state), but
the worker process — and Node's module cache for vendor deps like `jsdom`
— stays warm. This skips the ~300-1000 ms cold load of `jsdom` and its
transitives on every file. Each file still gets a fresh `new JSDOM()` +
fresh setupFiles, so test environment correctness is preserved.
Bench on a mid-sized React + jsdom test suite (19 files, 191 tests),
3 cold runs each, same machine:
isolate: true (default) 6.6s median
isolate: 'soft' 5.0s median (-24%)
Per-lib win scales with how DOM-heavy + how many files. Small pure-node
libs (no jsdom dependency) see a slight regression because the per-file
worker-reuse bookkeeping isn't amortised — for those, leave `isolate: true`.
Diff is intentionally tiny: type widening + pool reuse path + treating
`'soft'` the same as `false` for the existing `__rstest_clean_core_cache__`
call. No env-side changes, no module-mock surgery, no DOM reset logic —
each file gets a fresh JSDOM via the existing setup/teardown cycle.
Trade-offs (documented in the type doc-comment):
- Tests that rely on `rstest.mock(...)` factories needing to be re-applied
per file should keep `isolate: true`. Module mocks are tied to cached
module instances; the narrow cache clean doesn't reset them.
- Setup files that mutate prototype methods directly (e.g.
`Element.prototype.foo = jest.fn()` at top level) leak across files
in the same worker. Move those into `beforeEach` to make them
re-runnable.
Hardens the previous minimal 'soft' diff (7c956db) so it actually amortizes the per-file cost without leaking state. Three architectural fixes: 1. Persist the test environment across files in soft mode. Per-file env teardown was racing pending async work (React commit phase accessing window after dom.window.close() ran), throwing "ReferenceError: window is not defined" from inside react-dom. Worker-scope `cachedEnv` is set once and held; `softResetEnv` clears body/head/URL between files in place. Strict isolate still tears down per file (process exits anyway). 2. Worker-scope spy restore via tinyspy.restoreAll(). Per-api `mocks` Set is closure-scoped: a spy installed by file A's setupFile is orphaned when file A's api is GC'd, and file B's `restoreAllMocks` only walks file B's mocks. tinyspy already exposes a module-level `spies` Set and `restoreAll()` — switching to that restores every spy in the worker regardless of which api created it. 3. Reset fake timers between files. Sinon's `install()` rejects a second install on the same global, so a fresh `FakeTimers` instance (created per file in soft mode) throws "Can't install fake timers twice on the same global object" on file 2's `useFakeTimers()`. Capture the previous file's api and call `useRealTimers()` in teardown before the next file starts. Also: catch `rpc is closed` errors in the silentConsole interceptor. Pending async logs (React commits, deferred microtasks) fire after the worker's rpc has been disposed; the rejection was surfacing as "unhandledRejection: [birpc] rpc is closed". Best-effort drop — applies to all isolation modes. Validated: - rstest core unit tests: 242/242 pass - e2e default isolate: 625 pass (1 unrelated webkit-binary failure) - e2e --isolate false: 472/472 pass - private bench (4 client libs): bank-rec -42%, shared/core -42%, journals-feature & payments-feature pass at parity-or-better Known limitation: heavy Taco component renders mutate HTMLElement.prototype.focus into a getter-only descriptor (origin unknown, not reproducible in standalone Node+JSDOM+taco+React). Bundle re-evaluation per file in soft mode then fails on file 2's taco `prototype.focus = newFn`. Affected client libs (workflows-feature, reports-module) stay on `isolate: true`. Investigation continues. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Unblocks libs whose tests import `@testing-library/user-event` (or any
vendor package that monkey-patches DOM prototypes) when running under
`isolate: 'soft'`.
## Root cause
`@testing-library/user-event` patches `HTMLElement.prototype.focus` and
`.blur` to wrapped getters on first `userEvent.click()` via
`Object.defineProperties(proto, { focus: { configurable: true, get } })`.
It guards against re-patching with a module-local `Symbol('patched...')`
marker, which is created FRESH every time the bundle is re-evaluated.
In soft mode the worker is reused but the rspack-bundled test entry is
re-evaluated per file (each file has its own `__webpack_module_cache__`).
File 2's bundle re-runs user-event's `patchFocus`, the symbol-marker
check misses (different Symbol identity), and the next call into Taco's
own focus monkey-patch — `windowObject.HTMLElement.prototype.focus = fn`
inside `setupGlobalFocusEvents` — throws "Cannot set property focus of
[object HTMLElement] which has only a getter" because user-event has
already replaced the descriptor with a getter-only shape.
The same pattern affects any vendor doing prototype-level mutation
keyed on a per-module-instance symbol: react-aria's focus-visible
polyfill, future user-event versions, etc.
## Fix
Capture descriptors of well-known DOM prototype methods at env-setup
time (when JSDOM/happyDom is first installed in the worker), then
restore them in `softResetEnv` between files. Limited to:
HTMLElement.prototype, Element.prototype, Node.prototype
× { focus, blur, click, scrollIntoView, getBoundingClientRect }
These are the keys vendor packages are known to mutate. Snapshotting
every property would be slow and surprising — most properties don't
need restoration.
Also reset `localStorage` / `sessionStorage` / `document.cookie` in
softResetEnv. These don't address the focus issue directly but are the
other obvious storage that leaks across files in a reused worker; the
cost is negligible and the alternative (silent state contamination) is
worse.
## Validated
- rstest e2e --isolate false: 472/472 pass (no regression)
- private client repo:
- bank-rec-feature: 7.3s → 5.4s (-26%)
- shared/core: 3.6s (parity, small lib)
- workflows-feature: 13.4s → 7.3s (-46%) — WAS BLOCKED, now works
- journals-feature & payments-feature: pass
- reports-module: 554/556 pass under soft (down from 0/556) — focus
fix is universal; the remaining 2 file-level failures are unrelated
msw / module-mock state leaks under investigation.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
API change (pre-stable): the worker-reuse-with-reset mode is now opted
into via `experiments.softMode: true` rather than a third value on
`isolate`. Two reasons:
1. `experiments.*` is the right shape for an unstable feature. Users
pin the rstest version if they depend on the exact semantics; the
shape can change before stabilising without churning every
consumer's `isolate: 'soft'` config.
2. The reuse-vs-reset and strict-vs-loose axes are conceptually
orthogonal even if they happen to share an internal mode value.
Splitting the public API makes that clear.
Internal `IsolateMode = boolean | 'soft'` and the runtime checks
(`isolate !== true`, `isolate === 'soft'`) are unchanged — the
config normalizer sets `isolate: 'soft'` internally when
`experiments.softMode === true`. Pool/worker code didn't need to move.
Also addresses code-review punchlist items:
H1 — beforeExit teardown for the worker-scope cachedEnv. Strict mode
already tore down via cleanupFns; soft mode didn't, leaking JSDOM
virtual console / timers / cookie jar on clean worker exit.
H2 — snapshot all own-property descriptors on HTMLElement/Element/Node
prototypes (skip `constructor`), not just a hand-picked 5-key
list. The previous list missed realistic vendor targets
(addEventListener, animate, matches, …). Captures + symbols too.
Cost: ~3 protos × ~50-100 keys × O(1) descriptor lookup ≈ <5 ms.
M2 — split silent `try { …all-resets… } catch` into per-step attempts
with debug logging gated on `DEBUG=rstest:soft-mode`. A failing
`localStorage.clear` no longer hides the cookie/scroll/storage
resets below it.
M3 — documented the cookie-wipe limitation (only `path=/` is wiped;
tests setting non-root-path cookies must clear them in afterEach).
L1 — removed obsolete `_resetActiveElement` from the `g.window` shim
type (was unused).
E2E coverage (C2 from the review): adds `e2e/soft-mode/` with a 2-file
fixture that asserts the soft-mode contract directly:
- same worker pid for file-a and file-b (worker reuse)
- DOM body cleared between files
- localStorage / sessionStorage cleared between files
- `HTMLElement.prototype.focus` descriptor restored to its original
value-descriptor shape (probes the snapshot/restore path; uses a
history array recorded by setup-spy.ts at module-evaluation time)
- `useFakeTimers()` succeeds in file-b after file-a installed and
never uninstalled (verifies the per-file `useRealTimers()` reset)
- tinyspy `restoreAll()` actually restores spies across files
Validated:
- rstest core unit: 242/242 pass
- e2e default (`pnpm test`): 625/626 pass — one pre-existing webkit
failure (missing playwright binary), unrelated
- e2e `--isolate false`: 473/473 pass (new soft-mode fixture is the
+1 vs the prior 472)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ners
Under `experiments.softMode` (or `isolate: false`), the worker survives
across files. If the previous file scheduled async work that's still in
flight when its slot closes — typically a `useEffect`-driven XHR or a
microtask chain — the work resolves into the NEXT file's slot. Its
`unhandledRejection` then bubbles into the next file's
`process.on('unhandledRejection')` handler and gets attributed to the
wrong file (surfacing as flaky "Network Error" or similar).
In `isolate: true` the process dies between files and the pending work
dies with it. Jest's per-file `vm.Context` gives the same guarantee.
Worker-reuse modes don't get this for free, so we recreate it: at the
very top of `preparePool`, BEFORE the per-file listeners install, drain
5 setImmediate cycles with a temporary listener that absorbs any
`unhandledRejection`/`uncaughtException` that fires during the drain
window. Errors absorbed here originate in the prior file and aren't
attributable to the new file in any useful way; surfacing them as the
new file's failure was worse than dropping them. Set
`DEBUG=rstest:soft-mode` to log the absorbed count.
Validated:
- rstest core unit: 242/242
- e2e --isolate false: 473/473
- private client repo workspace mode: 3348/3348 green in 2:38 wall
(faster than the prior 3:00 — the drain reduces flake-retry churn)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses high-severity items from code review on the prior commits:
**runInPool.ts:**
* Atomic per-file listener install. `globalCleanups.push(removeListener)`
is now ordered BEFORE the matching `process.on(...)` calls, so an
early throw between the two can't leak the listener across files. In
soft mode, accumulated handlers would fire N times and attribute the
same error to N earlier file slots — fixing it preemptively.
* Symmetric DOM prototype-snapshot diff. `restoreProtoSnapshot` now
ALSO deletes own-keys that were added since snapshot time, not just
restores mutated ones. Without this, a vendor that adds
`HTMLElement.prototype.newMethod = fn` in file N leaks the new
method into file N+1. Symbol keys are included — covers vendor
marker patterns like `Symbol('patched-focus')`.
**e2e/soft-mode/** (rewritten):
* File renames: `file-a` → `a-init`, `file-b` → `c-verify`, plus a
new `b-leak` in the middle. Alphabetical order pins the schedule
under `pool.maxWorkers: 1` so assertions about file sequence are
deterministic.
* `b-leak.test.ts` deliberately schedules a deferred
`Promise.reject(...)` AND a deferred `setTimeout(() => { throw })`
without awaiting. If `drainPendingAsyncFromPriorFile` regresses
(removed, drain count too low, listener order wrong), the leaked
errors bubble into the next file's slot and the suite fails. With
the absorber working, the leaks are silently absorbed and `c-verify`
runs cleanly. Without this fixture the absorber path is uncovered —
matching the C2 finding from code review.
* `c-verify` assertions tightened:
- `__soft_file_seq__` pinned to exact `['file-a', 'file-leaker',
'file-b']` (was `length >= 2` — too loose).
- `Element.prototype.getBoundingClientRect` mock check no longer
skips the assertion via `if (before)` — asserts the spy exists
AND has zero recorded calls.
- Focus-descriptor history asserts all 3 captures returned `'value'`
(proves softReset restored prior file's patch each time).
**e2e/soft-mode-reuse/** (new):
* 4 fixture files + `pool.maxWorkers: 2`. Each file records its
`process.pid` to a shared log via the `RSTEST_SOFT_REUSE_LOG` env
hand-off. The driver test reads the log and asserts:
- 4 records (all files ran)
- fewer than 4 unique pids (worker reuse actually happened)
Catches a regression where softMode silently falls back to
`isolate: true` (4 unique pids) or a single-worker bottleneck (1
unique pid) — the previous fixture used `maxWorkers: 1` so "same
pid across files" was tautological.
Validated:
- rstest core unit: 242/242
- e2e --isolate false: 474/474 (was 473; +1 from soft-mode-reuse)
- e2e default: 627/629 (2 pre-existing browser-mode failures
unrelated — missing playwright webkit binary + a snapshot
obsolete-removal flake; neither touches the soft-mode code)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two improvements from review follow-up:
**Config warning (M1):** `withDefaultConfig` now emits a single
`logger.warn` if the user sets BOTH `isolate: false` and
`experiments.softMode: true`. Soft mode wins (it's strictly stronger:
reuse + reset), but silent override is surprising. The warning is
one-shot per config load, includes the resolution, and points to the
fix ("remove one to silence").
**Happy-dom e2e coverage (L2):** mirror of the jsdom soft-mode fixture
under `e2e/soft-mode-happy-dom/`. Verifies that `softResetEnv` and
`captureProtoSnapshot` handle the `'happy-dom'` env-name branch
identically to `'jsdom'`. Without this, a regression that broke
happy-dom (e.g. someone adding `if (envName === 'jsdom')` instead of
`!== 'node'` in softResetEnv) would pass our existing e2e silently.
Validated: e2e --isolate false 475/475 (up from 474 after the +1
soft-mode-reuse and now +1 soft-mode-happy-dom).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Profiling reports-module (74 jsdom-heavy files) on a single worker
revealed catastrophic heap growth across the soft-mode worker's
lifetime:
file 1 → heap 71 MB, rss 254 MB
file 10 → heap 745 MB, rss 1.5 GB
file 50 → heap 3.5 GB, rss 4.0 GB
file 65 → heap 4.0 GB, rss 4.5 GB
By file 50 the worker was GC-thrashing — vendor module instances,
React fiber roots, JSDOM nodes, and accumulated closures monotonically
grew the old generation. The `--trace` data confirmed the symptom:
individual tests ran 2-3× slower under softMode vs `isolate: true`,
because of GC pressure (not the env-reset or drain overhead).
This is a hard limitation of sharing a JS heap across many React-
heavy test files. softMode CAN'T fully isolate without `vm.Context`
(which would be ~Jest semantics and a much larger change). Instead,
**bound the lifetime** so heap pressure can't accumulate past a known
ceiling.
## What changed
`PoolRunner` gains a `maxTasks` option + `recordTaskCompleted()` and
checks `tasksRun < maxTasks` in `isUsable()`. The pool calls
`recordTaskCompleted()` on each release; when the cap is reached
`isUsable()` flips false and the existing `releaseRunner` path
disposes the runner instead of returning it to the idle pool, so the
pool spawns a fresh worker on the next acquire.
Cap defaults to **20 tasks** for soft mode (`isolate !== true`).
Strict isolate runners are single-use anyway so the cap is irrelevant
there.
With recycle at 20:
- 4 fresh workers spawned for the 74-file reports-module run (vs 1
long-lived worker before)
- Peak heap per worker stays ~1.5-2 GB instead of 4+ GB
- Per-test GC pressure removed
## What's *not* changed
`maxTasks: 0` is the legacy unbounded behavior — preserved as the
default for `isolate: true` (single-use anyway). Soft mode opts in
via the constant; making it user-configurable is a follow-up.
## Why 20
Empirically chosen from the heap trace. heap was ~700 MB by file 10
and ~1.5 GB by file 20 in the worst case. 20 keeps peak heap under
~1.5 GB on the worst lib measured. Smaller libs may benefit from
higher caps; configurability is TBD.
## Validated
- rstest core unit: 242/242 pass
- e2e --isolate false: 475/475 pass (recycle threshold not reached on
any soft-mode fixture; legacy path exercised)
- Single-worker reports-module: 4 workers spawned for 74 files,
confirming recycle path fires correctly
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The 20-task recycle cap was hardcoded; lift it onto the public surface
as `experiments.softMode: { maxFilesPerWorker: N }`. Default stays 20
(empirically chosen against a 74-file jsdom-heavy lib), but downstream
configs can now dial it up for lighter test bodies or down for heavier
ones without forking core.
Plumbing: SoftModeOptions type → withDefaultConfig still collapses any
truthy softMode into the internal `isolate: 'soft'` → createPool reads
the tuning value off `experiments.softMode` and passes it through
PoolOptions → Pool uses it as PoolRunner.maxTasks (fallback to 20).
E2E: `e2e/soft-mode-recycle` asserts that `maxFilesPerWorker: 1` with
4 files yields 4 unique pids — i.e. the cap is honored end-to-end.
Contributor
Author
|
Superseded by smaller standalone PRs that don't bundle e-conomic-specific exhaust:
The full softMode story turned out to need vendor-specific reset axes (DOM proto snapshot for user-event, drain-pending-async for msw v1) that aren't defensible as upstream defaults. Keeping those in a private fork; what's defensible upstream is just the building blocks above. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft — opening for visibility, not for review yet. No CC.
What
Adds
experiments.softMode: boolean | { maxFilesPerWorker?: number }to
RstestConfig. When enabled, the worker pool reuses processesacross test files (like
isolate: false) but also resets per-file envstate between them — DOM, storage, fake timers, spies, prototype
mutations — so most tests written against Jest's per-file-context
isolation still pass without code changes.
Why
In our private workspace ~70-file jsdom-heavy libs were spending the
bulk of wall time on per-file
fork()+new JSDOM()+ setupFiles.With
isolate: true(default), each file pays that fixed cost. Withisolate: false, the workers are reused but ANY cross-file state(DOM body, prototype mutations from user-event/react-aria, fake timers,
spies) leaks — practically unusable.
experiments.softModeis the in-between: reuse the worker ANDmechanically clean the per-file-isolated state the worker level can
see.
What's reset between files
document.body/document.headinnerHTML,window.history, scrolllocalStorage,sessionStorage,document.cookie(path=/ only)HTMLElement,Element,Node(snapshotted at env-setup, restored before next file). Also drops
own-keys added since snapshot (e.g. vendor-installed
Symbol('patched-focus')markers from user-event'spatchFocus).useRealTimers()on the prior file'sRstestapi)restoreAll()of
preparePoolso leftoverunhandledRejections from the priorfile don't misattribute to the next file's slot
Heap pressure → worker recycling
--traceprofiling revealed individual tests running 2-3× slowerunder softMode vs
isolate: trueon a 74-file jsdom-heavy lib. Heapinstrumentation showed why:
Sharing a JS heap across many React-heavy files = monotonic growth
(vendor module instances, React fiber roots, JSDOM nodes, accumulated
closures). By file 50 the worker GC-thrashes at 4 GB. This is the
hard limit of soft-mode reuse without
vm.Contextisolation.Fix:
PoolRunnernow hasmaxTasks— when a soft-mode workerhits N tasks,
isUsable()flips false and the pool's existing disposepath spawns a fresh worker. Heap pressure can't accumulate past a
known ceiling. Default cap 20 for
isolate !== true; strictisolate is single-use and unaffected. The cap is user-configurable
via
experiments.softMode: { maxFilesPerWorker: N }— dial it up forlight-test libs, down for heavier ones. With recycle at 20, the
worst lib's reports-module went from individual tests being 2-3×
slower to roughly parity with
isolate: true, while still amortizingfork + jsdom-init across the 20-file window.
What doesn't change
isolate: truesemantics — unchanged.isolate: falsesemantics — unchanged (still no per-file reset).RstestConfig.isolatestaysboolean.Why
experiments.*The reset semantics are pragmatic (curated to what we found in
practice). I'd rather mark this as experimental and iterate than
freeze a contract that turns out incomplete. A
logger.warnfires ifa user sets both
isolate: falseANDexperiments.softModeenabled.Files
packages/core/src/types/config.ts— publicexperiments?: { softMode?: boolean | SoftModeOptions }, newSoftModeOptionsinterface, internalNormalizedConfig.isolate: boolean | 'soft'packages/core/src/config.ts— normalize any truthyexperiments.softModeto internalisolate: 'soft', warn onisolate: false+ softMode conflictpackages/core/src/pool/index.ts— extractmaxFilesPerWorkerfromexperiments.softModeand thread intoPoolOptionspackages/core/src/pool/pool.ts,pool/types.ts— reuse runners for'soft'in addition tofalse; passmaxTaskstoPoolRunner(default 20, override via config)packages/core/src/pool/poolRunner.ts—tasksRuncounter,recordTaskCompleted(),isUsable()cappackages/core/src/runtime/worker/runInPool.ts— env caching, prototype snapshot/restore, fake-timer reset, tinyspyrestoreAll, drain-pending-async absorber,beforeExitteardown, optionalRSTEST_HEAP_TRACE=1diagnostice2e/soft-mode/— jsdom fixture covering DOM reset, prototype restore, fake-timer reinstall, spy restore, and absorber path (3 files:a-init/b-leak/c-verify)e2e/soft-mode-reuse/—maxWorkers: 2× 4 files, asserts actual worker reuse via pid loge2e/soft-mode-happy-dom/— happy-dom env pathe2e/soft-mode-recycle/—maxFilesPerWorker: 1× 4 files, asserts unique pid per file (cap honored end-to-end)Testing
pnpm --filter @rstest/core test— 242/242e2e --isolate false— 475/475 (was 472 pre-changes, +3 new soft-mode fixtures)e2edefault — 627/629 (2 pre-existing browser-mode failures unrelated)Validated in a private 348-file workspace: workspace-mode green at
3348/3348. Bench on the worst-case lib (74-file reports-module):
softMode + recycle median 23 s wall vs vanilla
isolate: truemedian33 s wall (−30%). Workspace-mode delta is smaller (~5%) because
parallelism across the default 11 workers dilutes the per-worker
amortization.
Status
Marking draft. Not requesting review yet; just opening so I and a
colleague can see the cumulative diff in one place.
Followups before un-drafting:
task-count