Skip to content

Commit 1abc957

Browse files
authored
Bench(reactivity): Add five hardening benches (#203)
1 parent 88684d9 commit 1abc957

3 files changed

Lines changed: 144 additions & 19 deletions

File tree

ai/plans/reactivity-hardening.md

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ for (const r of toRun) {
112112
}
113113
```
114114

115-
Benchmark (lands alongside in PR A): `flush-fanout-allocation-200x500`200 flush cycles, 500 invalidations each. Captures the fan-out shape. Expected: measurable improvement, allocation count drops sharply.
115+
Benchmark precursor: `flush-fanout-allocation-1000x500`1000 flush cycles, 500 invalidations each. Captures the fan-out shape. Expected: measurable improvement, allocation count drops sharply.
116116

117117
### Item 6: `boundRun` removal + shared `setContext` helper
118118

@@ -124,26 +124,26 @@ Benchmark (lands alongside in PR A): `flush-fanout-allocation-200x500` — 200 f
124124
- Drop `boundRun`. `Reaction.create` calls `reaction.run()` directly.
125125
- Extract shared `mergeContext(target, additional, defaults)` helper. Each class passes its own seed values (`{ value }` for Signal, `{ firstRun }` for Reaction, raw bag for Dependency).
126126

127-
Benchmark addition: `reaction-create-stop-200kx10` — page-render shape with many short-lived reactions. Expected: small but measurable improvement in `sub-unsub-100k`.
127+
Benchmark: existing `sub-unsub-100k` measures this directly. Expected: small but measurable improvement. (Currently runs at 22ms on CI — near the noise floor; if the win is small the bench may need amplification to clear σ.)
128128

129129
### Item 7: Benchmark additions
130130

131-
These workloads aren't gating Items 1-6 (those land on correctness merits), but they baseline the perf claims and gate the larger Item 9 rewrite. Each follows the existing pattern in `bench-signal.js``performance.mark` + `performance.measure`, sink-anchored, iteration counts sized to clear the σ-floor.
131+
These workloads aren't gating Items 1-6 (those land on correctness merits), but they baseline the perf claims and gate the larger Item 9 rewrite. Each follows the existing pattern in `bench-signal.js``performance.mark` + `performance.measure`, sink-anchored, iteration counts grounded in actual CI durations of existing benches to clear the σ-floor with headroom.
132+
133+
Ships as a precursor PR to main so the `tip-of-tree` side of tachometer-CI emits the same measurements as `this-change` when the hardening PR runs.
132134

133135
Stable-dependency churn (gates Item 9):
134-
- `reactive-stable-fanout-5000x500` — 5000 reactions each reading the same single signal, 500 invalidations
135-
- `reactive-stable-deps-3reads-50kx200` — 50k reactions × 3 signals × 200 cycles (median templating shape)
136-
- `reaction-stable-deps-10kx1k` — 10k reactions × 1 signal × 1000 invalidations
136+
- `reactive-stable-fanout-5000x100` — 5000 reactions each reading the same single signal, 100 invalidations
137+
- `reactive-stable-deps-3reads-5000x100` — 5000 reactions × 3 signals × 100 cycles (median templating shape)
137138

138139
Computed lifecycle (informs Item 8):
139-
- `computed-unobserved-1000x1000`1000 computed signals derived from a root, no external subscriber, root updated 1000 times
140-
- `computed-subscribe-unsubscribe-50k` — create computed, attach subscriber, detach, repeat
140+
- `computed-unobserved-200x500`200 computed signals derived from a root, no external subscriber, root updated 500 times
141+
- `computed-subscribe-unsubscribe-10k` — create computed, attach subscriber, detach, repeat 10k times
141142

142143
Scheduler allocation (verifies Item 5):
143-
- `flush-fanout-allocation-200x500`already specified under Item 5
144+
- `flush-fanout-allocation-1000x500`1000 flush cycles, 500-subscriber fanout each
144145

145-
Reaction lifecycle (verifies Item 6):
146-
- `reaction-create-stop-200kx10` — already specified under Item 6
146+
**Dropped from original list:** `reaction-stable-deps-10kx1k` (companion measurement redundant with the wide-fan + median-shape pair) and `reaction-create-stop-200kx10` (overlaps existing `sub-unsub-100k`). Five new benches; Item 6 cites existing `sub-unsub-100k` directly.
147147

148148
### Item 8: Unify `derive` / `computed` with lazy reference counting
149149

@@ -165,8 +165,8 @@ Tests:
165165
- Existing `computed-chain-10x60k` benchmark stays flat or improves (subscribers exist throughout the run)
166166

167167
Acceptance criteria (vs Item 7's baselines):
168-
- `computed-unobserved-1000x1000` improves dramatically — near-zero work for the unobserved case
169-
- `computed-subscribe-unsubscribe-50k` shows acceptable reference-counting overhead
168+
- `computed-unobserved-200x500` improves dramatically — near-zero work for the unobserved case
169+
- `computed-subscribe-unsubscribe-10k` shows acceptable reference-counting overhead
170170

171171
### Item 9: Dependency-tracking rewrite (gated)
172172

@@ -175,8 +175,8 @@ Acceptance criteria (vs Item 7's baselines):
175175
**Counter:** `Set.delete` + `Set.add` on small sets is fast and allocation-free on modern V8. The existing `reaction-dep-diff-45k` benchmark measures the changing-dependency case; nothing measures stable-deps churn today. The hypothesis may not survive measurement.
176176

177177
**Gating:** Item 7's stable-dep benchmarks must show meaningful headroom before this PR lands. Specific thresholds:
178-
- `reactive-stable-fanout-5000x500` shows ≥2× headroom attributable to Set churn
179-
- `reactive-stable-deps-3reads-50kx200` confirms in the median shape
178+
- `reactive-stable-fanout-5000x100` shows ≥2× headroom attributable to Set churn
179+
- `reactive-stable-deps-3reads-5000x100` confirms in the median shape
180180

181181
**If proceeding — versioned mark-and-sweep edges:**
182182
- Each reaction has an iteration counter, incremented per run
@@ -187,8 +187,8 @@ Acceptance criteria (vs Item 7's baselines):
187187
Side benefit: this gives natural transactional error recovery. If the callback throws, the partial sweep is skipped and dependencies remain intact for the next run.
188188

189189
**Acceptance criteria:**
190-
- `reactive-stable-fanout-5000x500` improves ≥2×
191-
- `reactive-stable-deps-3reads-50kx200` improves ≥1.5×
190+
- `reactive-stable-fanout-5000x100` improves ≥2×
191+
- `reactive-stable-deps-3reads-5000x100` improves ≥1.5×
192192
- `reaction-dep-diff-45k` flat or better (changing-set case must not regress)
193193
- `sub-unsub-100k` flat (creation/teardown path unchanged)
194194

packages/reactivity/bench/tachometer/bench-signal.js

Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -325,6 +325,121 @@ let sink = null;
325325
r.stop();
326326
}
327327

328+
/*******************************
329+
Reactivity Hardening — Items 5 / 8 / 9
330+
*******************************/
331+
332+
// Stable-dependency churn — gates Item 9 dep-tracking rewrite.
333+
// N reactions × stable deps × M invalidations. Existing reaction-dep-diff-45k
334+
// measures the changing-dependency case; these isolate the stable-set churn
335+
// hypothesized to dominate per-expression workloads.
336+
337+
// reactive-stable-fanout-5000x100 — wide-fan case, single stable dep per reaction
338+
{
339+
const sig = new Signal(0);
340+
const reactions = new Array(5000);
341+
for (let i = 0; i < 5000; i++) {
342+
reactions[i] = Reaction.create(() => {
343+
sink = sig.get();
344+
});
345+
}
346+
// purpose: 5000 reactions × 1 signal × 100 invalidations. Per-run Set.delete + add on a stable dep edge.
347+
performance.mark(startMark('reactive-stable-fanout-5000x100'));
348+
for (let i = 0; i < 100; i++) {
349+
sig.set(i + 1);
350+
Reaction.flush();
351+
}
352+
performance.measure('reactive-stable-fanout-5000x100', startMark('reactive-stable-fanout-5000x100'));
353+
for (let i = 0; i < 5000; i++) { reactions[i].stop(); }
354+
}
355+
356+
// reactive-stable-deps-3reads-5000x100 — median templating shape, 3 stable deps per reaction
357+
{
358+
const sigA = new Signal(0);
359+
const sigB = new Signal(0);
360+
const sigC = new Signal(0);
361+
const reactions = new Array(5000);
362+
for (let i = 0; i < 5000; i++) {
363+
reactions[i] = Reaction.create(() => {
364+
sink = sigA.get() + sigB.get() + sigC.get();
365+
});
366+
}
367+
// purpose: 5000 reactions × 3 signals × 100 cycles. Each run clears + re-adds 3 stable dep edges.
368+
performance.mark(startMark('reactive-stable-deps-3reads-5000x100'));
369+
for (let i = 0; i < 100; i++) {
370+
sigA.set(i + 1);
371+
Reaction.flush();
372+
}
373+
performance.measure('reactive-stable-deps-3reads-5000x100', startMark('reactive-stable-deps-3reads-5000x100'));
374+
for (let i = 0; i < 5000; i++) { reactions[i].stop(); }
375+
}
376+
377+
// Computed lifecycle — informs Item 8 lazy refcounted computed.
378+
379+
// computed-unobserved-200x500 — eager-recompute baseline.
380+
// Under the current code, computeds re-run on every source change regardless
381+
// of whether anyone observes them. Post-Item-8 this drops to near-zero — the
382+
// computed stays dormant without subscribers.
383+
{
384+
const root = new Signal(0);
385+
const computeds = new Array(200);
386+
for (let i = 0; i < 200; i++) {
387+
computeds[i] = Signal.computed(() => root.get() + i);
388+
}
389+
// anchor to keep the array live for DCE; values read outside any reaction so no subscriber attaches
390+
let preamble = 0;
391+
for (let i = 0; i < 200; i++) { preamble += computeds[i].get(); }
392+
sink = preamble;
393+
// purpose: 200 unobserved computed signals, root updated 500 times. Measures the eager-recompute cost the refcount removes.
394+
performance.mark(startMark('computed-unobserved-200x500'));
395+
for (let i = 0; i < 500; i++) {
396+
root.set(i + 1);
397+
Reaction.flush();
398+
}
399+
performance.measure('computed-unobserved-200x500', startMark('computed-unobserved-200x500'));
400+
}
401+
402+
// computed-subscribe-unsubscribe-10k — refcount machinery overhead on the subscribe/unsubscribe path.
403+
// Function-scoped fixture so each cycle's computed + observer are GC-eligible.
404+
{
405+
const root = new Signal(0);
406+
const cycle = () => {
407+
const c = Signal.computed(() => root.get() + 1);
408+
const r = Reaction.create(() => {
409+
sink = c.get();
410+
});
411+
r.stop();
412+
};
413+
// purpose: 10000 create-computed + attach-observer + detach cycles. Lifecycle cost the refcount path must keep acceptable.
414+
performance.mark(startMark('computed-subscribe-unsubscribe-10k'));
415+
for (let i = 0; i < 10_000; i++) { cycle(); }
416+
performance.measure('computed-subscribe-unsubscribe-10k', startMark('computed-subscribe-unsubscribe-10k'));
417+
}
418+
419+
// Scheduler allocation — verifies Item 5 set-swap.
420+
421+
// flush-fanout-allocation-1000x500 — amplified per-flush spread cost.
422+
// Existing reactive-fanout-500x1200 measures the same shape at 90ms; this
423+
// runs more flushes (1000) so per-flush array allocation is proportionally
424+
// more of the total. Set-swap eliminates the spread.
425+
{
426+
const sig = new Signal(0);
427+
const reactions = new Array(500);
428+
for (let i = 0; i < 500; i++) {
429+
reactions[i] = Reaction.create(() => {
430+
sink = sig.get();
431+
});
432+
}
433+
// purpose: 500 subscribers fanout across 1000 flush cycles. Each flush spreads pendingReactions; tests per-flush allocation churn.
434+
performance.mark(startMark('flush-fanout-allocation-1000x500'));
435+
for (let i = 0; i < 1000; i++) {
436+
sig.set(i + 1);
437+
Reaction.flush();
438+
}
439+
performance.measure('flush-fanout-allocation-1000x500', startMark('flush-fanout-allocation-1000x500'));
440+
for (let i = 0; i < 500; i++) { reactions[i].stop(); }
441+
}
442+
328443
/*******************************
329444
Results
330445
*******************************/

packages/reactivity/bench/tachometer/tachometer-ci-signal.json

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,12 @@
2222
{ "mode": "performance", "entryName": "sub-unsub-100k" },
2323
{ "mode": "performance", "entryName": "reaction-flush-noop-5m" },
2424
{ "mode": "performance", "entryName": "reaction-coalesce-400x100" },
25-
{ "mode": "performance", "entryName": "reaction-dep-diff-45k" }
25+
{ "mode": "performance", "entryName": "reaction-dep-diff-45k" },
26+
{ "mode": "performance", "entryName": "reactive-stable-fanout-5000x100" },
27+
{ "mode": "performance", "entryName": "reactive-stable-deps-3reads-5000x100" },
28+
{ "mode": "performance", "entryName": "computed-unobserved-200x500" },
29+
{ "mode": "performance", "entryName": "computed-subscribe-unsubscribe-10k" },
30+
{ "mode": "performance", "entryName": "flush-fanout-allocation-1000x500" }
2631
]
2732
},
2833
{
@@ -42,7 +47,12 @@
4247
{ "mode": "performance", "entryName": "sub-unsub-100k" },
4348
{ "mode": "performance", "entryName": "reaction-flush-noop-5m" },
4449
{ "mode": "performance", "entryName": "reaction-coalesce-400x100" },
45-
{ "mode": "performance", "entryName": "reaction-dep-diff-45k" }
50+
{ "mode": "performance", "entryName": "reaction-dep-diff-45k" },
51+
{ "mode": "performance", "entryName": "reactive-stable-fanout-5000x100" },
52+
{ "mode": "performance", "entryName": "reactive-stable-deps-3reads-5000x100" },
53+
{ "mode": "performance", "entryName": "computed-unobserved-200x500" },
54+
{ "mode": "performance", "entryName": "computed-subscribe-unsubscribe-10k" },
55+
{ "mode": "performance", "entryName": "flush-fanout-allocation-1000x500" }
4656
]
4757
}
4858
]

0 commit comments

Comments
 (0)