Keep Newton-Krylov Auto warm starts cold - #4040
Conversation
f465cf3 to
9f91980
Compare
|
Full local test results (all run after the change, on Julia 1.12.6):
The new regression test was additionally run standalone under both configurations it has to 🤖 Generated with Claude Code |
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
9f91980 to
be825ff
Compare
|
The branch has been rewritten on current clean Final decision:
The reason is performance, not only robustness: on the matrix-free Exact final local verification on Julia 1.12.6:
The clean-master standalone regression was also run and failed at Hairer4 drift |
|
CI note: the immediate documentation, downstream, and ImplicitDiscreteSolve downgrade failures are the clean- |
This PR should be ignored until reviewed by @ChrisRackauckas.
Fixes #4034. Companion LinearSolve guard: SciML/LinearSolve.jl#1123.
What changed
This is now a plain revert of the behavioral part of #3991:
LinearSolve.WarmStart.AutotoWarmStart.Hegedus.Autotherefore remains LinearSolve's standalone cold-start behavior.WarmStart.None,Previous, andHegeduschoices still pass throughunchanged.
its ExplicitImports exceptions are removed.
Hairer42.
OrdinaryDiffEqNonlinearSolvereceives a patch bump to 2.5.5.The companion LinearSolve change makes an explicitly requested Hegedüs warm
start fall back to cold unless the projected previous direction reduces the raw
residual by at least a factor of two. This PR deliberately does not detect
that guard and re-enable
Auto: even guarded Hegedüs remains slower than coldstarts on the representative matrix-free Newton-Krylov workload below.
Root cause
Hegedüs rescales the previous solution by
That scalar is optimal along the previous direction, but an optimal projection
onto a nearly orthogonal direction is still non-predictive. In the issue
reproducer, instrumentation over 260 Newton-Krylov solves found:
The previous Newton increment had converged to roughly
1e-19; rescaling it byroughly
1e31produced an order-one vector of round-off noise. Because theinner solve uses the integrator tolerance (
1e-3by default), GMRES sometimesaccepted that vector at iteration zero. Each update then depended on the full
history of prior solves instead of only the current
(W, b), and the adaptivecontroller amplified a one-ulp input difference.
The suspected
dz === linsolve.ualias is real, but not causal. GivingLinearSolve an independent previous-solution buffer produced identical results
to every printed digit because the Hegedüs scalar is invariant to rescaling of
that buffer.
Robustness evidence
Julia 1.12.6; one-ulp perturbation of
u0[1]; maximum solution difference:AutoLocal clean-master reproduction on
938dae56bf, released LinearSolve 5.3.0:The new
<1e-5regression fails there exactly as intended.Historical GPU CI identifies #3991 as the first bad change:
gpu_abs=4.44e-9,gpu_rel=9.73e-9;gpu_abs=3.62e-3,gpu_rel=7.02e-3;3.62e-3/7.02e-3;passed and Hairer4 returned to
4.44e-9/9.73e-9.There is no local NVIDIA GPU on this machine, so the GPU result is reported from
the exact CI jobs; the CPU regression is reproduced locally above.
Performance evidence
nf/ best-of-N wall time in milliseconds.Small index-1 DAE (
n=4,reltol=1e-3)The guarded explicit mode is both stable and iteration-efficient on this tiny,
highly correlated workload.
Matrix-free 1D Brusselator (
n=800,reltol=1e-6)Default diagonal preconditioners; identical nonlinear-solve counts in every row:
Against cold, guarded Hegedüs is still +12.7% / +4.4% for KenCarp4,
+30.0% / +39.3% for TRBDF2, and +7.5% / +11.3% for FBDF (
nf/ walltime). The guard removes part of #3991's regression but does not turn it into a
default-worthy speedup.
Sparse AD + ILU(τ=50), same Brusselator
At roughly one Krylov iteration per solve, even weakly predictive guesses can
save the single iteration, while the guard rejects them. Results are mixed:
Hegedüs can still be a useful explicit, measured choice, but neither guarded nor
unguarded behavior is a sound universal default.
Local verification
0.003734637491723536as expected.GROUP=Regression_I: passed, including the new regression (6/6).GROUP=OrdinaryDiffEqNonlinearSolve_Core: passed, including Homotopy (25/25)and Homotopy
init/default_nlsolve(16/16).GROUP=OrdinaryDiffEqNonlinearSolve_QA: passed (JET and Aqua).GROUP=OrdinaryDiffEqDifferentiation_Core: passed.--check: passed.GROUP=Core,GROUP=QA, and repository-wide Runic all passed.1.33e-15(nf=617),Hairer42 drift
1.33e-15(nf=625), Cash4 drift2.77e-9(nf=740).