⚡️ Speed up assert/property hot path ~5x#7017
Open
dubzzz wants to merge 4 commits into
Open
Conversation
For `assert(property(constant(1), () => {}))` (and similarly trivial
workloads dominated by framework overhead), the per-iteration cost drops
from ~820 ns to ~140 ns on Node 22 — about 5.5–6× faster end-to-end.
Concrete changes:
- Introduce `FastXorshift128Plus`, a drop-in replacement for pure-rand's
`xorshift128plus` that produces the identical value stream but runs
`jump()` via a precomputed 64 KB four-Russians table (16 byte-lookups
+ 16 four-word XORs) instead of 128 calls to `next()`. Wire it in as
the default and the `'xorshift128plus'` string randomType.
- Replace `runIdToFrequency`'s `Math.log` + `~~` with an integer digit
ladder. Stays on the Smi path and eliminates per-iter `DoubleToI`.
- Refactor `Tosser` from a generator into a `TossIterator` class that
reuses one `Random` wrapper across iterations, and, when the rng is
`FastXorshift128Plus`, syncs the work rng's state with four field
writes instead of allocating a fresh clone per call.
- Add a `pullNext` hot path to `SourceValuesIterator` and
`RunnerIterator` that returns the value (or `SENTINEL_DONE`) directly,
avoiding the `{value, done}` IteratorResult allocations the `for-of`
protocol would force on every iteration.
- Move arity-based predicate dispatch into `Property.run` itself. The
former `(t) => p(...t)` wrapper allocated a fresh closure per
`property()` call, giving each call site its own feedback cell and
causing repeated "wrong feedback cell" deopts on the hot path. The
dispatched call site is now stable across all `property()` calls.
- Inline the `noUndefinedAsContext` hot path in `Property.generate` and
`AlwaysShrinkableArbitrary.generate`: skip the helper call entirely
when `value.context !== undefined` (the common case).
Verified: the full fast-check test suite (2505 tests) still passes,
including the Poisoning suite (no reliance on `.bind` / direct
prototype-method calls on user buffers) and the NoRegression snapshot
suite (same value stream as before). `NoRegressionStack` snapshots are
updated to reflect the removal of the now-unused `Property.predicate`
wrapper frame.
- Drop now-unused `xorshift128plus` import from QualifiedParameters (the default and the 'xorshift128plus' string both resolve to fastXorshift128plus, which has the same value stream). - Widen `TossIterator.pullNext` return type to `Value<Ts> | SENTINEL_DONE` to satisfy the `PullableIterator` contract (the runtime never returns SENTINEL_DONE — this iterator is unbounded once examples are exhausted). - Extend the `fakeRandom` test helper's Omit to also exclude the new `unsafeReplaceInternal` / `unsafeGetInternal` internal methods.
|
@fast-check/ava
fast-check
@fast-check/jest
@fast-check/packaged
@fast-check/poisoning
@fast-check/vitest
@fast-check/worker
commit: |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #7017 +/- ##
==========================================
- Coverage 94.87% 94.65% -0.22%
==========================================
Files 212 213 +1
Lines 5888 6173 +285
Branches 1545 1587 +42
==========================================
+ Hits 5586 5843 +257
- Misses 294 317 +23
- Partials 8 13 +5
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Mirrors the harness from #7027: two minimal benches that run the full `assert(property(...))` flow with an empty always-successful predicate. Lets the CI bench job show the runner-overhead win this PR brings against `main` side-by-side.
Contributor
⏱️ Benchmark ResultsClick to expand |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
For trivial properties — anything dominated by fast-check's per-iteration plumbing rather than user predicates —
assert(property(...))is now ~5× faster on Node 22. On the minimal benchmarkassert(property(constant(1), () => {}))the call goes from ~81 µs to ~15 µs, and per-iteration steady-state cost drops from ~820 ns to ~140 ns.This matters for users who write many small property tests in their suite: today the runner overhead is comparable to or larger than the actual user predicate for cheap properties, and that overhead is fixed cost paid on every
assert(). For larger CI suites with thousands of property tests, this can shave seconds off wall-clock time. Heavier predicates see proportionally smaller wins (their own work dominates), but they still benefit from the cheaper jump and lower per-iteration allocation pressure.No observable behavior change. The default rng still produces the exact same value stream for any given seed, so existing tests (counterexamples, seeds, replay paths) reproduce bit-for-bit. The full test suite (2505 tests) passes, including the Poisoning suite (which verifies the framework still works under hostile global-prototype tampering) and the NoRegression snapshot suite. The only user-observable difference:
params.randomType === xorshift128plusidentity checks no longer match the default — code using the'xorshift128plus'string or default behavior is unaffected, but code that compares against the importedxorshift128plusfunction will see ourfastXorshift128plusinstead. Stack-trace snapshots also lose one (now-removed)Property.predicateframe.What changed
FastXorshift128Plus— drop-in replacement for pure-rand'sxorshift128plus(samenext()and samejump()end-state) whosejump()is implemented as a precomputed four-Russians lookup table (16 byte lookups + 16 four-word XORs) instead of 128 calls tonext(). Wired in as the default and as the'xorshift128plus'string randomType. Standalonejump()benchmark: ~21× faster (846 ns → 40 ns); this alone accounts for most of the end-to-end win, sincejump()was ~50% of CPU.runIdToFrequency— replaced~~(Math.log(runId + 1) * 1/log(10))with an integer digit-count ladder. Same numeric output, but stays on the Smi path so V8 no longer emits a per-iterationDoubleToI.Tosser— replaced the generator with aTossIteratorclass that reuses oneRandomwrapper across iterations. When both pool and work rng areFastXorshift128Plus(the default case), state is synced with four field writes per iteration instead of allocating a freshclone().SourceValuesIterator/RunnerIterator— added apullNext()hot path returning the value (or aSENTINEL_DONEsymbol) directly, so the runner loop no longer allocates an{ value, done }IteratorResult per iteration. The shrink-stream slot is held as a monomorphicPullableIteratorto keep V8's IC happy; the dynamic capability probe is done once at construction (and explicitly avoidsFunction.prototype.bindso the Poisoning suite still passes).Propertyarity dispatch — moved the(t) => p(...t)spread out of the per-property()closure and into aswitchinsideProperty.runkeyed on a stored arity. The old wrapper allocated a fresh closure per call, giving each call site its own feedback cell and triggering repeated "wrong feedback cell" deopts; the new dispatch is a single stable call site shared across everyproperty()call.Property.generate/AlwaysShrinkableArbitrary.generate— inlined the early-return for the common case wherevalue.context !== undefined, skipping thenoUndefinedAsContextfunction call entirely (it was a no-op for any tuple-wrapped arbitrary, which covers everything built viaproperty(...)).How to reproduce the benchmark
The repo ships
perf-bench/bench-suite.mjs(a self-contained Node script). For a quick standalone check:Indicative numbers on Node 22.22 (5000 asserts × 100 numRuns each, median of 9 samples):
assert(property(constant(1), () => {}))numRuns=1000(per-iter)numRuns=1(setup-bound)property(integer(), () => {})property(int, int, int) -> noopDesign notes / trade-offs
jump()to pure-rand? That's the right long-term home, but shipping a vendored implementation here unblocks the win without coupling fast-check releases to apure-randrelease. The 64 KB jump table is built lazily at module import (~1 ms one-shot) and is shared across allFastXorshift128Plusinstances.Propertyfor 1-arg properties (skip the tuple)? Considered, but it would change theTstyping contract (property(arb, p)would no longer beProperty<[T]>internally) and bleed into shrink semantics. The cheaper arity-dispatch + tuple wrapping already captures most of the win without breaking layering.(t) => p(...t)wrapper drops one frame from error stacks (Property.predicateno longer appears). This is captured in the updatedNoRegressionStacksnapshot — the new stack is strictly shorter, which is arguably an improvement for users debugging failures.Impact
Patch-level under the changeset definition: no API additions, no breaking behavior, identical value streams. The one observable change is the default rng's function identity, which is plausibly an implementation detail that nobody asserts against — but flagging it here so the maintainer can decide whether it warrants a minor bump instead.
Tests
NoRegressionStacksnapshot is updated to reflect the now-shorter stack (one fewer wrapper frame).QualifiedParameterstest that asserted the default rng function identity is updated to referencefastXorshift128plus.assert(property(...)); perf is verified via theperf-bench/scripts shipped in this PR, not run in CI).Fixes #issue-number
Checklist
— Don't delete this checklist and make sure you do the following before opening the PR
pnpm run bumpor by following the instructions from the changeset bot🐛(vitest) Something...) when the change targets a package other thanfast-checkGenerated by Claude Code