bench(wa-sqlite): measure scheduler contention

[marcus-pousette]

Purpose

Experimental raw-browser harness used to evaluate #167's worker RPC scheduler. It measures background remote ingest while foreground reads and normal local writes share the same wa-sqlite client.

The experiment has finished. Its useful evidence is preserved here and in #167; this PR is being closed rather than merged as permanent infrastructure.

What is measured

• Direct, dedicated-worker, and shared-worker runtimes.
• Memory and OPFS storage.
• Empty and 5,000-operation prefilled trees.
• Remote append batches at normal and background priority in the same build.
• Foreground read latency, normal local-write latency, remote-batch latency, total duration, and exact final tree count.

Normal omits the priority option and preserves FIFO barriers. Background exercises #167's foreground bypass. Direct memory is the no-scheduler control.

Avoiding misleading fast paths

• Four trials per scenario, with priority order alternating normal/background and background/normal.
• Trial identity is recorded in workload metadata and output filenames.
• Reads and writes are released across remote batch-start barriers instead of only during the opening batch.
• Every run uses an isolated database and verifies the expected final child count.
• Physical SQLite filenames are deliberately short and opaque; descriptive labels remain in result metadata. This avoids confusing OPFS pathname capacity with scheduler behavior.

The first run exposed the independent OPFS path-capacity bug fixed by #213: wa-sqlite reported 64 bytes, while one filename plus SQLite's -journal suffix was 66 bytes. It was deterministic SQLITE_CANTOPEN, not a teardown race. The false cooldown/workaround was removed.

Results

Focused run 29203415286 completed all 64/64 runtime rows. Values below are background-vs-normal deltas between four-trial medians; negative latency is better.

Runtime / prefill	Read p95	Local-write p95	Remote-batch p95
Memory direct / 0	+0.7%	-2.6%	+1.4%
Memory direct / 5k	+0.1%	-1.2%	+0.8%
Memory worker / 0	-12.2%	+1.3%	+4.0%
Memory worker / 5k	+0.4%	+0.0%	+1.0%
OPFS worker / 0	-16.1%	+1.1%	+0.9%
OPFS worker / 5k	-2.6%	-2.9%	-2.4%
OPFS shared / 0	+1.0%	+1.4%	+1.5%
OPFS shared / 5k	+0.4%	+0.6%	+0.7%

The useful result is narrow and repeatable: every empty-tree trial improved dedicated-worker read p95 (memory: 11.3–14.6%; OPFS: 14.4–17.9%). Populated cases stayed near neutral. The largest median latency regression was 4.0% remote-batch p95.

The shared-worker case used one client/port, so it did not exercise the distinct cross-tab/global scheduler. That finding led #167 to remove shared-worker scheduling and retain only the validated dedicated-worker path. Four trials are directional evidence, not a claim of statistical certainty.

durationMs was captured after the final integrity read, so it dilutes the derived throughput comparison. The latency samples and exact final-count validation are unaffected; #167 deliberately does not use the throughput field.

Scope and validation

• One commit; 4 files, +443/-1.
• Reuses shared benchmark helpers and the existing Playwright invocation.
• Prettier, diff checks, Vite production build, Playwright discovery, and normal GitHub CI pass.
• fix(ci): make benchmark dispatch build and run measurements #212's focused web dispatch completed both sides, uploaded the artifact, and published the result comment.

Why close instead of merge

• The harness adds +443/-1 and increased the web benchmark from about 9.9 to 16.4 minutes per checkout (roughly 66%).
• No threshold or regression gate consumes these scenario pairs automatically.
• Keeping it would impose recurring code and runner cost after the decision it was built for.
• The raw artifact, methodology, and decision-relevant results remain linked above.

[github-actions]

Benchmarks

Triggered by workflow_dispatch - Run: https://github.com/cybersemics/treecrdt/actions/runs/29203415286
Suite: web - Base command: success - Head command: success
Base: fix/shared-worker-priority-scheduler (5e8e544) - Head: codex/runtime-perf-benchmark (33ebd8d)
Compared entries: 96 - Improved: 17 - Regressed: 15

Impl	Storage	Workload	Base ops/s	Head ops/s	Δ ops/s	Base p50 ms	Head p50 ms	Δ p50	Status
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-background-prefill-0-trial-1	-	3076.57	-	-	656.57	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-background-prefill-0-trial-2	-	3085.57	-	-	654.66	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-background-prefill-0-trial-3	-	3056.11	-	-	660.97	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-background-prefill-0-trial-4	-	3073.74	-	-	657.18	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-background-prefill-5000-trial-1	-	1272.05	-	-	1587.99	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-background-prefill-5000-trial-2	-	1287.66	-	-	1568.73	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-background-prefill-5000-trial-3	-	1248.42	-	-	1618.05	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-background-prefill-5000-trial-4	-	1255.93	-	-	1608.36	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-normal-prefill-0-trial-1	-	3057.09	-	-	660.76	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-normal-prefill-0-trial-2	-	3055.05	-	-	661.20	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-normal-prefill-0-trial-3	-	3059.63	-	-	660.21	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-normal-prefill-0-trial-4	-	3019.64	-	-	668.95	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-normal-prefill-5000-trial-1	-	1240.68	-	-	1628.15	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-normal-prefill-5000-trial-2	-	1279.53	-	-	1578.71	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-normal-prefill-5000-trial-3	-	1281.51	-	-	1576.27	-	✅
wa-sqlite	browser-memory-dedicated-worker	runtime-mixed-sync-ingest-local-writes-memory-dedicated-worker-normal-prefill-5000-trial-4	-	1280.59	-	-	1577.40	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-background-prefill-0-trial-1	-	2936.35	-	-	687.93	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-background-prefill-0-trial-2	-	2977.81	-	-	678.35	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-background-prefill-0-trial-3	-	2986.64	-	-	676.34	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-background-prefill-0-trial-4	-	2986.29	-	-	676.42	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-background-prefill-5000-trial-1	-	1230.51	-	-	1641.59	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-background-prefill-5000-trial-2	-	1230.81	-	-	1641.20	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-background-prefill-5000-trial-3	-	1211.07	-	-	1667.94	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-background-prefill-5000-trial-4	-	1239.98	-	-	1629.06	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-normal-prefill-0-trial-1	-	1996.88	-	-	1011.58	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-normal-prefill-0-trial-2	-	2998.68	-	-	673.63	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-normal-prefill-0-trial-3	-	2942.57	-	-	686.48	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-normal-prefill-0-trial-4	-	3022.28	-	-	668.37	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-normal-prefill-5000-trial-1	-	1224.12	-	-	1650.16	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-normal-prefill-5000-trial-2	-	1227.02	-	-	1646.26	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-normal-prefill-5000-trial-3	-	1216.72	-	-	1660.20	-	✅
wa-sqlite	browser-memory-direct	runtime-mixed-sync-ingest-local-writes-memory-direct-normal-prefill-5000-trial-4	-	1233.83	-	-	1637.18	-	✅
wa-sqlite	browser-memory	insert-chain-100	2694.69	2863.07	+6.25%	37.11	34.93	-5.88%	🟡
wa-sqlite	browser-memory	insert-chain-1000	407.85	411.11	+0.80%	2451.90	2432.45	-0.79%	✅
wa-sqlite	browser-memory	insert-move-100	4069.18	4105.09	+0.88%	49.15	48.72	-0.87%	✅
wa-sqlite	browser-memory	insert-move-1000	6645.40	6780.58	+2.03%	300.96	294.96	-1.99%	✅
wa-sqlite	browser-memory	replay-log-100	2855.51	2557.54	-10.43%	35.02	39.10	+11.65%	🟠
wa-sqlite	browser-memory	replay-log-1000	406.84	409.68	+0.70%	2457.98	2440.91	-0.69%	✅
wa-sqlite	browser-memory	sync-balanced-children-cold-start-fanout10-100	250.63	260.33	+3.87%	39.90	38.41	-3.73%	✅
wa-sqlite	browser-memory	sync-balanced-children-cold-start-fanout10-1000	258.90	256.95	-0.75%	38.63	38.92	+0.76%	✅
wa-sqlite	browser-memory	sync-balanced-children-cold-start-fanout10-10000	249.80	247.63	-0.87%	40.03	40.38	+0.87%	✅
wa-sqlite	browser-memory	sync-balanced-children-payloads-cold-start-fanout10-100	510.98	515.81	+0.95%	41.10	40.71	-0.94%	✅
wa-sqlite	browser-memory	sync-balanced-children-payloads-cold-start-fanout10-1000	501.04	507.46	+1.28%	41.91	41.38	-1.26%	✅
wa-sqlite	browser-memory	sync-balanced-children-payloads-cold-start-fanout10-10000	432.03	430.44	-0.37%	48.61	48.79	+0.37%	✅
wa-sqlite	browser-memory	sync-one-missing-100	15.91	15.37	-3.42%	62.83	65.06	+3.54%	✅
wa-sqlite	browser-memory	sync-one-missing-1000	5.10	5.06	-0.73%	196.06	197.49	+0.73%	✅
wa-sqlite	browser-memory	sync-one-missing-10000	0.30	0.30	-0.08%	3368.33	3370.98	+0.08%	✅
wa-sqlite	browser-memory	sync-root-children-fanout10-1110	49.72	48.66	-2.14%	40.22	41.10	+2.18%	✅
wa-sqlite	browser-opfs-coop-sync	insert-chain-100	1114.33	1225.98	+10.02%	89.74	81.57	-9.11%	🟠
wa-sqlite	browser-opfs-coop-sync	insert-chain-1000	384.42	387.55	+0.82%	2601.35	2580.32	-0.81%	✅
wa-sqlite	browser-opfs-coop-sync	insert-move-100	1940.52	1940.81	+0.01%	103.06	103.05	-0.01%	✅
wa-sqlite	browser-opfs-coop-sync	insert-move-1000	3267.79	3748.41	+14.71%	612.04	533.56	-12.82%	🟠
wa-sqlite	browser-opfs-coop-sync	replay-log-100	1240.46	1239.58	-0.07%	80.61	80.67	+0.07%	✅
wa-sqlite	browser-opfs-coop-sync	replay-log-1000	387.32	389.68	+0.61%	2581.87	2566.23	-0.61%	✅
wa-sqlite	browser-opfs-coop-sync	sync-balanced-children-cold-start-fanout10-100	138.09	140.00	+1.38%	72.42	71.43	-1.36%	✅
wa-sqlite	browser-opfs-coop-sync	sync-balanced-children-cold-start-fanout10-1000	143.16	145.61	+1.71%	69.85	68.68	-1.68%	✅
wa-sqlite	browser-opfs-coop-sync	sync-balanced-children-cold-start-fanout10-10000	140.41	133.68	-4.79%	71.22	74.81	+5.03%	🟡
wa-sqlite	browser-opfs-coop-sync	sync-balanced-children-payloads-cold-start-fanout10-100	249.39	243.73	-2.27%	84.20	86.16	+2.32%	✅
wa-sqlite	browser-opfs-coop-sync	sync-balanced-children-payloads-cold-start-fanout10-1000	244.51	235.54	-3.66%	85.89	89.16	+3.80%	✅
wa-sqlite	browser-opfs-coop-sync	sync-balanced-children-payloads-cold-start-fanout10-10000	184.58	185.55	+0.53%	113.77	113.18	-0.52%	✅
wa-sqlite	browser-opfs-coop-sync	sync-one-missing-100	9.43	9.03	-4.19%	106.04	110.68	+4.37%	✅
wa-sqlite	browser-opfs-coop-sync	sync-one-missing-1000	2.46	2.46	-0.04%	407.17	407.33	+0.04%	✅
wa-sqlite	browser-opfs-coop-sync	sync-one-missing-10000	0.16	0.16	+0.22%	6187.21	6173.66	-0.22%	✅
wa-sqlite	browser-opfs-coop-sync	sync-root-children-fanout10-1110	23.21	22.91	-1.33%	86.15	87.31	+1.35%	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-background-prefill-0-trial-1	-	941.09	-	-	2146.45	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-background-prefill-0-trial-2	-	943.46	-	-	2141.05	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-background-prefill-0-trial-3	-	920.66	-	-	2194.09	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-background-prefill-0-trial-4	-	924.39	-	-	2185.23	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-background-prefill-5000-trial-1	-	182.66	-	-	11058.95	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-background-prefill-5000-trial-2	-	181.48	-	-	11130.41	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-background-prefill-5000-trial-3	-	184.12	-	-	10971.11	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-background-prefill-5000-trial-4	-	184.31	-	-	10959.92	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-normal-prefill-0-trial-1	-	943.57	-	-	2140.81	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-normal-prefill-0-trial-2	-	947.50	-	-	2131.92	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-normal-prefill-0-trial-3	-	952.71	-	-	2120.27	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-normal-prefill-0-trial-4	-	934.42	-	-	2161.77	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-normal-prefill-5000-trial-1	-	180.11	-	-	11215.57	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-normal-prefill-5000-trial-2	-	177.89	-	-	11355.08	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-normal-prefill-5000-trial-3	-	181.74	-	-	11114.81	-	✅
wa-sqlite	browser-opfs-dedicated-worker	runtime-mixed-sync-ingest-local-writes-opfs-dedicated-worker-normal-prefill-5000-trial-4	-	182.40	-	-	11074.77	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-background-prefill-0-trial-1	-	631.79	-	-	3197.26	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-background-prefill-0-trial-2	-	627.35	-	-	3219.87	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-background-prefill-0-trial-3	-	628.13	-	-	3215.90	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-background-prefill-0-trial-4	-	620.65	-	-	3254.67	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-background-prefill-5000-trial-1	-	95.35	-	-	21184.93	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-background-prefill-5000-trial-2	-	95.93	-	-	21057.89	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-background-prefill-5000-trial-3	-	95.61	-	-	21127.54	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-background-prefill-5000-trial-4	-	94.11	-	-	21464.98	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-normal-prefill-0-trial-1	-	638.22	-	-	3165.06	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-normal-prefill-0-trial-2	-	634.24	-	-	3184.93	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-normal-prefill-0-trial-3	-	632.92	-	-	3191.56	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-normal-prefill-0-trial-4	-	621.08	-	-	3252.38	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-normal-prefill-5000-trial-1	-	95.39	-	-	21177.19	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-normal-prefill-5000-trial-2	-	95.44	-	-	21164.27	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-normal-prefill-5000-trial-3	-	95.88	-	-	21068.47	-	✅
wa-sqlite	browser-opfs-shared-worker	runtime-mixed-sync-ingest-local-writes-opfs-shared-worker-normal-prefill-5000-trial-4	-	93.93	-	-	21506.46	-	✅

Status: ✅ <5% change | 🟡 5–10% | 🟠 10–30% | 🔴 >30%

Artifacts:

• base summary: bench-base/benchmarks/summary.json
• head summary: bench-head/benchmarks/summary.json
• command logs: bench-base/benchmark.log, bench-head/benchmark.log

[marcus-pousette]

The experiment is complete and its decision-relevant results are preserved in this description and #167. Closing instead of merging avoids adding 443 lines and roughly 66% to every web benchmark checkout; the full artifact remains linked for future reference.