Skip to content

Keep Newton-Krylov Auto warm starts cold - #4040

Draft
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/fix-4034-warmstart
Draft

Keep Newton-Krylov Auto warm starts cold#4040
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/fix-4034-warmstart

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jul 27, 2026

Copy link
Copy Markdown
Member

This PR should be ignored until reviewed by @ChrisRackauckas.

Fixes #4034. Companion LinearSolve guard: SciML/LinearSolve.jl#1123.

What changed

This is now a plain revert of the behavioral part of #3991:

  • Newton-based integrators no longer rewrite LinearSolve.WarmStart.Auto to
    WarmStart.Hegedus.
  • Auto therefore remains LinearSolve's standalone cold-start behavior.
  • Explicit WarmStart.None, Previous, and Hegedus choices still pass through
    unchanged.
  • The now-unused cross-sublibrary resolver, its feature-specific unit test, and
    its ExplicitImports exceptions are removed.
  • A one-ulp index-1 DAE regression pins round-off reproducibility for Hairer4 and
    Hairer42.
  • OrdinaryDiffEqNonlinearSolve receives a patch bump to 2.5.5.

The companion LinearSolve change makes an explicitly requested Hegedüs warm
start fall back to cold unless the projected previous direction reduces the raw
residual by at least a factor of two. This PR deliberately does not detect
that guard and re-enable Auto: even guarded Hegedüs remains slower than cold
starts on the representative matrix-free Newton-Krylov workload below.

Root cause

Hegedüs rescales the previous solution by

$$\xi = \frac{\langle Au,b\rangle}{\lVert Au\rVert^2}.$$

That scalar is optimal along the previous direction, but an optimal projection
onto a nearly orthogonal direction is still non-predictive. In the issue
reproducer, instrumentation over 260 Newton-Krylov solves found:

r0/||b||:        min=3.82e-16  median=2.30e-02  max=1.00e+00
|cos(Ax0,b)|:    min=1.99e-16  median=1.00e+00  max=1.00e+00
|xi|:            min=3.95e-32  median=6.93e-02  max=5.07e+31
iteration-0 acceptance fraction: 0.373

The previous Newton increment had converged to roughly 1e-19; rescaling it by
roughly 1e31 produced an order-one vector of round-off noise. Because the
inner solve uses the integrator tolerance (1e-3 by default), GMRES sometimes
accepted that vector at iteration zero. Each update then depended on the full
history of prior solves instead of only the current (W, b), and the adaptive
controller amplified a one-ulp input difference.

The suspected dz === linsolve.u alias is real, but not causal. Giving
LinearSolve an independent previous-solution buffer produced identical results
to every printed digit because the Hegedüs scalar is invariant to rescaling of
that buffer.

Robustness evidence

Julia 1.12.6; one-ulp perturbation of u0[1]; maximum solution difference:

solver #3991 unguarded Hegedüs cold Auto explicit guarded Hegedüs
Hairer4 3.73e-3 6.64e-8 1.33e-15
Hairer42 5.67e-4 1.33e-15 1.33e-15
Cash4 4.62e-6 1.33e-15 2.77e-9
ImplicitEuler 3.11e-9 1.33e-15 1.33e-15
KenCarp4 4.50e-3 1.90e-8 1.90e-8
FBDF 1.51e-8 7.71e-9 6.78e-9

Local clean-master reproduction on 938dae56bf, released LinearSolve 5.3.0:

Hairer4 drift = 0.003734637491723536

The new <1e-5 regression fails there exactly as intended.

Historical GPU CI identifies #3991 as the first bad change:

There is no local NVIDIA GPU on this machine, so the GPU result is reported from
the exact CI jobs; the CPU regression is reproduced locally above.

Performance evidence

nf / best-of-N wall time in milliseconds.

Small index-1 DAE (n=4, reltol=1e-3)

configuration Hairer4 Hairer42 Cash4 FBDF
cold 842 / 0.89 865 / 0.89 1127 / 1.28 191 / 0.33
unguarded Hegedüs (#3991) 670 / 1.00 664 / 0.95 775 / 1.14 160 / 0.36
guarded Hegedüs 617 / 0.87 625 / 0.89 740 / 1.05 160 / 0.38

The guarded explicit mode is both stable and iteration-efficient on this tiny,
highly correlated workload.

Matrix-free 1D Brusselator (n=800, reltol=1e-6)

Default diagonal preconditioners; identical nonlinear-solve counts in every row:

configuration KenCarp4 TRBDF2 FBDF
cold 116405 / 5945 90170 / 3176 40582 / 1660
unguarded Hegedüs (#3991) 146946 / 7754 142007 / 6553 46941 / 2003
guarded Hegedüs 131218 / 6209 117236 / 4423 43606 / 1847

Against cold, guarded Hegedüs is still +12.7% / +4.4% for KenCarp4,
+30.0% / +39.3% for TRBDF2, and +7.5% / +11.3% for FBDF (nf / wall
time). The guard removes part of #3991's regression but does not turn it into a
default-worthy speedup.

Sparse AD + ILU(τ=50), same Brusselator

configuration KenCarp47 TRBDF2 FBDF
cold 706 / 82.7 780 / 95.7 427 / 60.0
unguarded Hegedüs (#3991) 607 / 62.3 767 / 110.1 385 / 48.4
guarded Hegedüs 688 / 90.9 780 / 102.2 411 / 47.7

At roughly one Krylov iteration per solve, even weakly predictive guesses can
save the single iteration, while the guard rejects them. Results are mixed:
Hegedüs can still be a useful explicit, measured choice, but neither guarded nor
unguarded behavior is a sound universal default.

Local verification

  • Clean-master standalone regression: fails at Hairer4 drift
    0.003734637491723536 as expected.
  • GROUP=Regression_I: passed, including the new regression (6/6).
  • GROUP=OrdinaryDiffEqNonlinearSolve_Core: passed, including Homotopy (25/25)
    and Homotopy init/default_nlsolve (16/16).
  • GROUP=OrdinaryDiffEqNonlinearSolve_QA: passed (JET and Aqua).
  • GROUP=OrdinaryDiffEqDifferentiation_Core: passed.
  • Repository-wide Runic 1.7 --check: passed.
  • Companion LinearSolve PR: focused warm-start tests 41/41, GROUP=Core,
    GROUP=QA, and repository-wide Runic all passed.
  • Exact explicit guarded-Hegedüs rerun: Hairer4 drift 1.33e-15 (nf=617),
    Hairer42 drift 1.33e-15 (nf=625), Cash4 drift 2.77e-9 (nf=740).

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Full local test results (all run after the change, on Julia 1.12.6):

suite result
lib/OrdinaryDiffEqDifferentiation Pkg.test() exit 0 — incl. Krylov warm_start default 7/7
lib/OrdinaryDiffEqSDIRK Pkg.test() exit 0 — Tableau 57/57, Stage predictors 115/115, Convergence 102 pass / 2 pre-existing broken, DAE 8/8
lib/OrdinaryDiffEqBDF Pkg.test() exit 0
lib/OrdinaryDiffEqNonlinearSolve Pkg.test() exit 0
umbrella GROUP=Regression_I exit 0 — Dense 4104/4104, Special Interp 30/30, Inplace 2/2, Adaptive 46/46, Hard DAE 36/36, Newton-Krylov Round-off 6/6
LinearSolve GROUP=Core (against the patched LinearSolve) exit 0 — Krylov warm start 40/40

The new regression test was additionally run standalone under both configurations it has to
survive — released LinearSolve 5.2.0 (cold start) and the locally patched LinearSolve
(guarded Hegedüs) — 3/3 for each of Hairer4 and Hairer42 in both.

🤖 Generated with Claude Code

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude ChrisRackauckas-Claude changed the title Require a guarded Hegedüs warm start before defaulting to it (fixes #4034) Keep Newton-Krylov Auto warm starts cold Jul 29, 2026

Copy link
Copy Markdown
Member Author

The branch has been rewritten on current clean master (938dae56bf) after the full comparison. The earlier self-rearming/guard-detection design is superseded.

Final decision:

The reason is performance, not only robustness: on the matrix-free n=800 Brusselator, guarded Hegedüs still regressed versus cold by +12.7% nf / +4.4% time (KenCarp4), +30.0% / +39.3% (TRBDF2), and +7.5% / +11.3% (FBDF). It remains useful on some correlated systems, but is not a sound automatic default.

Exact final local verification on Julia 1.12.6:

group / check result
umbrella GROUP=Regression_I passed; new one-ulp test 6/6
OrdinaryDiffEqNonlinearSolve_Core passed; Homotopy 25/25, Homotopy init/default 16/16
OrdinaryDiffEqNonlinearSolve_QA passed; JET + Aqua
OrdinaryDiffEqDifferentiation_Core passed
repository-wide Runic 1.7 --check passed
companion LinearSolve Core + QA passed, including Allocation QA 48/48 and SupernodalLU Allocation QA 8/8

The clean-master standalone regression was also run and failed at Hairer4 drift 0.003734637491723536, while the final cold-Auto branch passed the <1e-5 regression. An exact explicit guarded-Hegedüs rerun against #1123 produced Hairer4 drift 1.33e-15 (nf=617), Hairer42 1.33e-15 (nf=625), and Cash4 2.77e-9 (nf=740).

Copy link
Copy Markdown
Member Author

CI note: the immediate documentation, downstream, and ImplicitDiscreteSolve downgrade failures are the clean-master NonlinearSolveBase compatibility conflict fixed independently in #4051. The failing resolver requires NonlinearSolveBase >=2.40 from ImplicitDiscreteSolve while OrdinaryDiffEqNonlinearSolve currently caps the locally available range at 2.38. These setup failures occur before this PR's tests run and are unrelated to the warm-start diff; #4040 remains intentionally unstacked on the small prerequisite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Hegedüs warm-start default (#3991) amplifies round-off ~1e13x on the Newton-Krylov path (GPU DAE test failure)

2 participants