Skip to content

Give the SRI weak-order study enough trajectories for its own tolerance - #4096

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:fix-sri-weak-convergence-trajectories
Aug 2, 2026
Merged

Give the SRI weak-order study enough trajectories for its own tolerance#4096
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:fix-sri-weak-convergence-trajectories

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

Please ignore until reviewed by @ChrisRackauckas.

Fixes #4061.

The problem

iip_weak.jl's SRI study asserts abs(sim.𝒪est[:weak_final] - 2) < 0.5 from 2e4 trajectories, and fails:

Test Failed at lib/StochasticDiffEq/test/weak_convergence/iip_weak.jl:186
  Expression: abs(sim.𝒪est[:weak_final] - 2) < 0.5
   Evaluated: 0.5533865655544976 < 0.5

It is deterministic — it reproduces to the last digit across runs — and it is not new. The group simply never got this far: the file was OOM-killed at the SROCK2 call on line 71, 115 lines earlier, until #4060 landed. Everything after line 186 had never executed.

Why more trajectories rather than a wider tolerance

Sampling the estimator over 12 independent seeds:

trajectories 𝒪 range mean sd max |𝒪−2| fails < 0.5
2e4 (current) [1.822, 2.277] 2.006 0.172 0.277 0/12
1e5 [1.892, 2.214] 2.008 0.100 0.214 0/12
4e5 [1.910, 2.078] 1.972 0.049 0.090 0/12

The mean sits on 2 at every count and sd falls as 1/sqrt(n) to within noise — 1e5 → 4e5 is a 4× increase in samples and sd goes 0.100 → 0.050 predicted, 0.049 observed. So the estimator is unbiased and the sole problem is spread: 2e4 gives it a standard deviation of 0.172 against a 0.5 tolerance, and the file's own RNG state (seeded once at the top, then advanced by the fifteen studies before it) lands about 3σ out.

At 4e5 the worst of 12 seeds is 0.09 from 2, more than five times inside the existing bound, so the assertion still has the resolution to catch a genuine order regression. Widening the tolerance to admit a 0.553 estimate would not — it would remove exactly the signal the test exists to provide. The cost is ~4 minutes for that one study (16.3 s → 254.1 s here).

Verification

Whole group through the normal harness, in a 16 GiB cgroup (systemd-run --scope -p MemoryMax=16G -p MemorySwapMax=0) at JULIA_NUM_THREADS=2, matching the self-hosted pool and its num-threads: auto default:

GROUP=IIPWeakConvergence julia --project=. -e 'using Pkg; Pkg.test()'

     Testing StochasticDiffEq tests passed
        Elapsed (wall clock) time (h:mm:ss or m:ss): 17:00.35
        Maximum resident set size (kbytes): 1623164

The log reaches SRI and SRIW1, i.e. this is the first end-to-end run of the file.

`iip_weak.jl`'s SRI study asserts `abs(𝒪est[:weak_final] - 2) < 0.5` from 2e4
trajectories. Sampling the estimator over 12 independent seeds shows that
tolerance is too tight for that sample size:

  trajectories   𝒪 range           mean    sd      max |𝒪-2|
  2e4            [1.822, 2.277]    2.006   0.172   0.277
  1e5            [1.892, 2.214]    2.008   0.100   0.214
  4e5            [1.910, 2.078]    1.972   0.049   0.090

The mean sits on 2 at every count and `sd` falls as `1/sqrt(n)` to within noise,
so the estimator is unbiased and the only problem is spread. The study's own RNG
state — seeded once at the top of the file and advanced by the fifteen studies
before it — lands about 3σ out, at `0.5533865655544976`, and fails.

This has not been seen because the group never reached line 186: the file was
OOM-killed at the SROCK2 call on line 71 until SciML#4060.

Raise the count to 4e5 rather than widen the tolerance. At 4e5 the worst of 12
seeds is 0.09 from 2, more than five times inside the existing bound, so the
assertion still catches a real order regression; widening it to admit a bad
estimate would not. The cost is ~4 minutes for that one study.

Verified with the whole group, 16 GiB cgroup, JULIA_NUM_THREADS=2:

  GROUP=IIPWeakConvergence  ->  tests passed, 17m00s, 1.55 GiB peak

which is the first end-to-end run of this file — everything after line 186 had
never executed before.

Fixes SciML#4061

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UwLXp5WY1uiqPun7qhwxeU
@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review August 2, 2026 20:55
@ChrisRackauckas
ChrisRackauckas merged commit de431f1 into SciML:master Aug 2, 2026
137 of 147 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

iip_weak.jl SRI weak-order assertion fails deterministically (was masked by an OOM 115 lines earlier)

2 participants