Give the SRI weak-order study enough trajectories for its own tolerance - #4096
Merged
ChrisRackauckas merged 1 commit intoAug 2, 2026
Conversation
`iip_weak.jl`'s SRI study asserts `abs(𝒪est[:weak_final] - 2) < 0.5` from 2e4 trajectories. Sampling the estimator over 12 independent seeds shows that tolerance is too tight for that sample size: trajectories 𝒪 range mean sd max |𝒪-2| 2e4 [1.822, 2.277] 2.006 0.172 0.277 1e5 [1.892, 2.214] 2.008 0.100 0.214 4e5 [1.910, 2.078] 1.972 0.049 0.090 The mean sits on 2 at every count and `sd` falls as `1/sqrt(n)` to within noise, so the estimator is unbiased and the only problem is spread. The study's own RNG state — seeded once at the top of the file and advanced by the fifteen studies before it — lands about 3σ out, at `0.5533865655544976`, and fails. This has not been seen because the group never reached line 186: the file was OOM-killed at the SROCK2 call on line 71 until SciML#4060. Raise the count to 4e5 rather than widen the tolerance. At 4e5 the worst of 12 seeds is 0.09 from 2, more than five times inside the existing bound, so the assertion still catches a real order regression; widening it to admit a bad estimate would not. The cost is ~4 minutes for that one study. Verified with the whole group, 16 GiB cgroup, JULIA_NUM_THREADS=2: GROUP=IIPWeakConvergence -> tests passed, 17m00s, 1.55 GiB peak which is the first end-to-end run of this file — everything after line 186 had never executed before. Fixes SciML#4061 Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UwLXp5WY1uiqPun7qhwxeU
ChrisRackauckas
marked this pull request as ready for review
August 2, 2026 20:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ignore until reviewed by @ChrisRackauckas.
Fixes #4061.
The problem
iip_weak.jl's SRI study assertsabs(sim.𝒪est[:weak_final] - 2) < 0.5from 2e4 trajectories, and fails:It is deterministic — it reproduces to the last digit across runs — and it is not new. The group simply never got this far: the file was OOM-killed at the
SROCK2call on line 71, 115 lines earlier, until #4060 landed. Everything after line 186 had never executed.Why more trajectories rather than a wider tolerance
Sampling the estimator over 12 independent seeds:
< 0.5The mean sits on 2 at every count and
sdfalls as1/sqrt(n)to within noise — 1e5 → 4e5 is a 4× increase in samples andsdgoes 0.100 → 0.050 predicted, 0.049 observed. So the estimator is unbiased and the sole problem is spread: 2e4 gives it a standard deviation of 0.172 against a 0.5 tolerance, and the file's own RNG state (seeded once at the top, then advanced by the fifteen studies before it) lands about 3σ out.At 4e5 the worst of 12 seeds is 0.09 from 2, more than five times inside the existing bound, so the assertion still has the resolution to catch a genuine order regression. Widening the tolerance to admit a 0.553 estimate would not — it would remove exactly the signal the test exists to provide. The cost is ~4 minutes for that one study (16.3 s → 254.1 s here).
Verification
Whole group through the normal harness, in a 16 GiB cgroup (
systemd-run --scope -p MemoryMax=16G -p MemorySwapMax=0) atJULIA_NUM_THREADS=2, matching the self-hosted pool and itsnum-threads: autodefault:The log reaches
SRIandSRIW1, i.e. this is the first end-to-end run of the file.