Skip to content

Report the top error-estimate contributors on solver exit - #4057

Draft
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:diagnostics/error-estimate-contributors
Draft

Report the top error-estimate contributors on solver exit#4057
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:diagnostics/error-estimate-contributors

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

Please ignore until reviewed by @ChrisRackauckas.

Motivation

From a Slack thread with @MasonProtter: a large ODE run repeatedly with updated parameters suddenly starts hitting maxiters. Saving every step to see what happened OOMs a 256GB machine, and with save_everystep=false the surviving output is ten time points and a stats block — no NaNs, no large values, nothing to go on.

The solver knew which equation was forcing the step size down. It just never said so. Quoting the thread:

What we want isn't just the EEst, which is the time stepping error estimate, it's EEst = ||atmp||. And EEst > 1 means error too high, reject. What you care about is atmp — because that vector tells you "I have to reject because atmp[i] says the time stepper has too high of an error on equation i".

What this does

Implements SciMLBase.log_error_estimate (added in SciML/SciMLBase.jl#1478) for ODEIntegrator. On any failing exit, the log now carries the scalar estimate plus the top-3 contributing state components:

┌ Warning: Verbosity toggle: max_iters
│  Interrupted. Larger maxiters is needed. [...] Reached t=0.01712924006064487 with dt=3.5068470420252215e-5.
│
│ Error estimate diagnostics:
│ step error estimate EEst = 0.1727 (a step is accepted when EEst <= 1) after 498 accepted and 2 rejected steps
│
│ largest contributors to EEst = internalnorm(atmp), where atmp is the tolerance-weighted local error per state component:
│   atmp[7] = -0.5463, u[7] = 8.311e-07, uprev[7] = 8.311e-07
│   atmp[1] = 2.141e-19, u[1] = 0.9983, uprev[1] = 0.9983
│   atmp[2] = 2.141e-19, u[2] = 0.9983, uprev[2] = 0.9983
└

That is a 10-equation system where only component 7 is stiff — the diagnostic names it, 15 orders of magnitude clear of everything else.

Details:

  • atmp is read straight off the cache. Mutable caches keep the residuals after the step, so there is no recomputation and no extra f evaluation; the numbers reported are exactly the ones the controller rejected on.
  • Composite and default caches are peeled by cache.current, so the cache that actually ran is the one reported, not the one that would run next.
  • Out-of-place caches build the residuals in a local and drop them. Those report the scalar estimate alone rather than guessing.
  • Non-finite residuals are counted separately and sort ahead of finite ones, so a single NaN component is named rather than buried.
  • Non-adaptive integration returns "" — there is no error estimate to report.
  • All of it is behind the exit's verbosity toggle; nothing is computed when the toggle is off.

Printf is added to OrdinaryDiffEqCore for %.4g formatting (also added by #3731; trivial conflict if that lands first).

Tests

test/InterfaceIII/exit_diagnostics.jl covers the unit behavior of top_error_contributors (including the NaN ordering), and asserts through solve that the named component appears for a maxiters exit, an instability exit under both Tsit5 and FBDF, a NaN-producing RHS, and a composite algorithm; that a silent verbosity emits nothing; and that an out-of-place cache degrades to the scalar estimate.

Run locally on Julia 1.11:

Test Summary:          | Pass  Total  Time
top_error_contributors |    4      4  0.1s
Test Summary:                                 | Pass  Total  Time
maxiters names the dominant error contributor |    6      6  1.4s
Test Summary:                                   | Pass  Total  Time
instability exits report the error estimate too |    4      4  1.4s
Test Summary:                       | Pass  Total  Time
non-finite residuals are called out |    3      3  0.3s
Test Summary:                                        | Pass  Total  Time
composite and default caches report the active cache |    2      2  3.2s
Test Summary:                  | Pass  Total  Time
silent verbosity emits nothing |    2      2  1.6s
Test Summary:                                 | Pass  Total  Time
degrades gracefully without a stored residual |    4      4  2.6s
Test Summary:                                            | Pass  Total  Time
non-adaptive integration has no error estimate to report |    1      1  0.3s

Checklist

  • Appropriate tests were added
  • Any code changes were done in a way that does not break public API
  • All documentation related to code changes were updated
  • The new code follows the contributor guidelines, in particular the SciML Style Guide and COLPRAC
  • Any new documentation only uses public API

Additional context

Depends on SciML/SciMLBase.jl#1478, which defines the log_error_estimate hook and is what makes these diagnostics fire on maxiters at all. CI here will not resolve until that is merged and released — the SciMLBase = "3.41" compat bound points at the unreleased version.

Composes with #3731: that PR's log_numerical_instability reports the Jacobian rows/columns involved, this one reports which components the step controller is choking on, and #1478 makes both run on every failing exit rather than only on dtmin/unstable.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DKB6v14nZHZPYtKiLsPLDT

Implements SciMLBase's `log_error_estimate` for `ODEIntegrator`, so a solve
that gives up reports the scalar error estimate along with the state components
that produced it.

`EEst = internalnorm(atmp)` only says the step was rejected. `atmp` says which
equation rejected it, which is the thing a user has to look at when one
component of a large system quietly forces the step size down until the solve
hits maxiters. The residuals survive on mutable caches after the step, so the
breakdown needs no recomputation; out-of-place caches build them in a local, so
those report the scalar estimate alone.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DKB6v14nZHZPYtKiLsPLDT
@ChrisRackauckas-Claude
ChrisRackauckas-Claude force-pushed the diagnostics/error-estimate-contributors branch from caf1509 to 6c2817c Compare July 29, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants