Commit 689d0b7
Add Jacobian reuse for Rosenbrock-W methods (#3075)
* Add Jacobian reuse for Rosenbrock-W methods (rebased onto master)
Implements CVODE-inspired Jacobian reuse for Rosenbrock-W methods.
W-methods guarantee correctness with a stale Jacobian, so we skip
expensive J recomputations when conditions allow:
- Reuse J but always rebuild W (cheap LU vs expensive AD/finite-diff)
- Recompute J on: first iter, step rejection, callback, resize,
gamma ratio change >30%, every 20 accepted steps, algorithm switch
- Disabled for: strict Rosenbrock, DAEs, linear problems, CompositeAlgorithm
Squashed and rebased from PR #3075 (7 commits) onto current master
after substantial upstream restructuring (cache consolidation,
generic_rosenbrock.jl deletion, RodasTableau unification).
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* Fix AD dual propagation and OOP numerical consistency in Jacobian reuse
- Strip ForwardDiff.Dual from dtgamma before storing in JacReuseState
(these values are heuristic-only and don't need to carry derivatives)
- When new_jac=true in OOP path, delegate to standard calc_W instead of
custom W construction to ensure numerical consistency with IIP path
(fixes regression in special_interps test for Rosenbrock23)
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* Disable Jacobian reuse for non-adaptive solves
Non-adaptive solves (adaptive=false) with prescribed timesteps don't
benefit meaningfully from J reuse, and it causes IIP/OOP inconsistency:
adaptive solves have step rejections (EEst > 1) that trigger fresh J
recomputation and reset reuse state, while non-adaptive solves following
the same timesteps never reject and thus evolve different reuse state.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* Relax special_interps tolerance for Rosenbrock-W methods
Rosenbrock-W methods with Jacobian reuse take slightly different adaptive
steps than the non-adaptive OOP solve (which uses fresh J every step).
This causes interpolation differences at ~1e-8, well below the solver
tolerance of 1e-14. Relax the test bound to 1e-7 for W-methods.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* Remove Julia 1.10 LTS fallback for calc_rosenbrock_differentiation
Import unconditionally; accept LTS failures.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* Remove do_newJW delegation; keep all Rosenbrock J/W logic in one place
_rosenbrock_jac_reuse_decision now always returns (new_jac, new_W)
directly instead of returning nothing to delegate to do_newJW.
For Rosenbrock methods (no nlsolver), do_newJW just returned
(true, true) anyway — the indirection split logic across two
functions for no benefit.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* Relax lazy-W utility test tolerance for Rosenbrock-W J reuse
The lazy-W vs explicit-W comparison for Rosenbrock23 diverges at ~3e-4
with Jacobian reuse active. Relax atol from 2e-5 to 5e-4.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* Disable J reuse for operator-based W (Krylov solvers)
WOperator and AbstractSciMLOperator wrap J internally for Krylov
solvers. A stale J degrades Krylov convergence, causing order loss
(e.g. Rodas5P+Enzyme+KrylovJL dropping from order 5 to 1.7).
Always recompute J for these W types.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* Fix linear-problem fast path and DelayDiffEq @test_broken
_rosenbrock_jac_reuse_decision was returning (true, true) for linear
operator ODEs with a WOperator-wrapped W because the WOperator check
fired before the linear-function check. That made Rosenbrock23 rebuild
J and W every step on ODEFunction(::MatrixOperator) problems (seen as
nw = 628 / 454 in lib/OrdinaryDiffEqNonlinearSolve linear_solver_tests
expecting nw == 1), instead of building W once and reusing it.
Reorder the decision so the linear-function branch fires immediately
after the iter<=1 check, matching the pre-reuse do_newJW behavior:
- iter <= 1 -> (true, true) [first step must build W]
- islin -> (false, false) [reuse afterwards, regardless of W type]
- non-adaptive / WOperator / mass-matrix / composite checks follow
Also flip DelayDiffEq jacobian.jl:57 from @test_broken to @test — the
nWfact_ts[] == njacs[] assertion now passes with the Rosenbrock J/W
accounting from this PR.
Verified locally:
Rosenbrock23 on ODEFunction(MatrixOperator, mass_matrix=...): nw=1
Rodas5P+KrylovJL convergence test: L2 order 5.004 (tight reltol)
Rosenbrock23 convergence on prob_ode_2Dlinear: order 1.996
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* DelayDiffEq jacobian test: only assert Wfact_t==jac count for Rodas5P
The `nWfact_ts[] == njacs[]` assertion holds for Rosenbrock methods
(one Wfact_t / jac call per step) but not for TRBDF2 — SDIRK methods
call Wfact_t per stage, so the SDIRK Wfact_t count is much larger
than the jac count from the jac-based solve (observed 282 vs 3).
The earlier 'Unexpected Pass' was for Rodas5P only, so flip @test_broken
to @test just for that alg and keep TRBDF2 broken.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Expose max_jac_age and jac_reuse_gamma_tol as Rosenbrock kwargs
Both thresholds used in _rosenbrock_jac_reuse_decision were hardcoded:
- Gamma ratio tolerance (|dtgamma/last_dtgamma - 1|) at 0.3
- Max accepted-step age between J recomputations at 20
Expose them as constructor kwargs on all Rosenbrock algorithm structs
(both macro-generated blocks, RosenbrockW6S4OS, and HybridExplicitImplicitRK):
Rosenbrock23(; max_jac_age = 20, jac_reuse_gamma_tol = 0.3)
Threading:
- Fields added to each alg struct; constructors take matching kwargs
- alg_cache passes alg.max_jac_age into JacReuseState at construction
- Decision function reads alg.jac_reuse_gamma_tol (hasproperty guarded for
robustness against any alg struct that skips the field)
- JacReuseState(dtgamma, max_jac_age = 20) gets an optional second arg
Set max_jac_age = 1 (or any small value) to effectively disable reuse
for debugging / comparison. Non-W-methods accept the kwargs harmlessly
since reuse is gated on isWmethod(alg) in the decision function.
Verified: Rosenbrock23 on prob_ode_2Dlinear with jac_reuse_gamma_tol=0.01
doubles njacs (6 → 12) as expected while preserving order 1.996.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Restore rejected-step J reuse for strict Rosenbrock methods
Regression in calc_rosenbrock_differentiation! for non-W-methods: the
IIP path was passing newJW = _rosenbrock_jac_reuse_decision(...) to
calc_W! for all algorithms, including strict Rosenbrock. For strict
methods _rosenbrock_jac_reuse_decision returns (true, true) via the
!isWmethod early return, which overrode calc_W!'s do_newJW errorfail
branch that reuses J across step rejections (since retries land at the
same (uprev, t) with only a smaller dt).
Effect on master: Rodas5P on Brusselator 1D reported njacs = naccept
(125), rejected steps did not add to njacs. Regression on the PR:
njacs = naccept + nreject (139), one extra J per rejection.
Fix: branch on isWmethod inside the IIP wrapper. W-methods use the
reuse decision + newJW; strict methods call calc_W! without newJW,
letting do_newJW handle the errorfail/reject case as on master.
Verified on Brusselator 1D at reltol=1e-6:
Rodas5P: njacs 139 → 125 (naccept=125, nreject=14), 23.7 → 23.1 ms
Rodas4: njacs 212 → 194 (naccept=194, nreject=18), 34.8 → 33.7 ms
Rosenbrock23 / ROS34PW2 / ROS34PW3 / Rodas23W unchanged.
Reported by @gstein3m.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert DelayDiffEq jacobian test back to @test_broken for nWfact_ts==njacs
My earlier 47ecc05 flipped @test_broken → @test based on a CI
"Unexpected Pass" report, but that pass was a symptom of the
strict-Rosenbrock jac-reuse regression I later fixed in 2db475d.
On genuine master, njacs counts accepted steps (do_newJW's errorfail
branch reuses J on retries) while nWfact_ts counts every step attempt
(calc_W! calls Wfact_t unconditionally). The test's invariant
nWfact_ts == njacs only holds when there are zero rejections, which
isn't true for Rodas5P on this DDE (57 Wfact_t calls vs 54 jac calls
= 3 rejected steps).
Keep the assertion as @test_broken and document why — this is the
correct master behavior, not a bug in the counting.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Lower jac_reuse_gamma_tol default from 0.3 to 0.03
A full benchmark sweep on ODEProblemLibrary stiff problems shows the
original 0.3 gamma tolerance was tuned only for PDE-like dynamics
(Brusselator) where dt barely changes. For chemistry and relaxation
oscillators (ROBER, Van der Pol stiff), 30% dt changes are normal,
so J goes stale fast and the solver pays for it with heavy step
rejections.
Benchmarking Rosenbrock23 across Brusselator1D, ROBER, VdP stiff,
HIRES, Pollution (geometric mean wall-time ratio vs master):
γ=0.30 +19.8% slower on average (old default)
γ=0.20 +18.6%
γ=0.15 +12.4%
γ=0.10 +8.5%
γ=0.05 +1.4%
γ=0.03 +1.4% <-- new default — dominates γ=0.30 on every class
γ=0.02 +1.6%
γ=0.01 +9.9%
γ=0.03 is strictly better than γ=0.30 on every problem class:
Problem class γ=0.30 γ=0.03
Bruss1D -31.6% -34.7% (bigger PDE win)
vdp_stiff +131.8% +43.4% (3x less cost)
rober +17.4% +5.7% (3x less cost)
hires +12.3% -0.5% (≈ master)
pollution +17.9% +8.7%
Intuition: at γ=0.3 we "reuse J until dt changes by 30%", which is
way too loose — chemistry phase transitions and relaxation oscillator
fast/slow switches move dt by 2–10× and the reuse keeps old J across
the transition. At γ=0.03 ("reuse until dt changes by 3%"), PDE
problems with near-constant dt still hit the reuse path (their dt
typically changes by <1% per step), but chemistry problems catch
their phase transitions early and recompute before rejections start
cascading.
Strict Rosenbrock methods are unaffected (verified via trace
fingerprint comparison across 9 strict algorithms × 3 problems ×
4 tolerances — byte-identical t-sequences and u endpoints, 0 diff
lines vs master).
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Rosenbrock23/32: default max_jac_age=1 (no reuse), opt-in via kwarg
Rosenbrock23 and Rosenbrock32 are low-order Rosenbrock-W methods
typically used on small problems, as a fallback in auto-switching
algorithms, or for stiff-detection — workloads where Jacobian
evaluation is cheap and per-step overhead dominates. On these
problems the general W-method reuse defaults (max_jac_age=20,
jac_reuse_gamma_tol=0.03) are a net loss: on Van der Pol stiff the
reuse causes step rejections that cost 2.3× master wall time even
at optimal γ.
Change: Rosenbrock23 and Rosenbrock32 default `max_jac_age = 1`.
Combined with a new cache-side optimization that skips JacReuseState
allocation when max_jac_age ≤ 1, the decision function short-circuits
through do_newJW (master-compatible path) and the entire reuse code
path is bypassed. Higher-order W-methods (Rodas23W, Rodas4P2,
Rodas5P/Pe/Pr, Rodas6P, RosenbrockW6S4OS, ROS34PW1a/b/2/3, ROS34PRw,
ROS3PRL/2, ROK4a) keep the default `max_jac_age = 20`, so the PDE
reuse wins are preserved.
Supporting changes:
- _make_jac_reuse_state(dtgamma, max_jac_age) helper returns `nothing`
when age ≤ 1, so Rosenbrock23/32 caches allocate no jac_reuse state.
- calc_rosenbrock_differentiation! now dispatches on
`isWmethod(alg) && jac_reuse !== nothing` so W-methods with reuse
disabled take the same master-compatible path as strict Rosenbrock.
- get_jac_reuse uses hasfield(typeof(cache), :jac_reuse) instead of
hasproperty — compile-time constant-foldable, removes runtime
reflection cost from the hot path on caches that opt out.
- gamma_tol fallback in the decision function also uses hasfield.
Benchmarks (wall time vs master, Rosenbrock23, best-of-3):
Problem/tol master PR Δ%
Bruss1D/1e-6 71.99 ms 55.61 ms -22.8%
Bruss1D/1e-8 258.79 ms 251.76 ms -2.7%
rober/1e-6 0.28 ms 0.29 ms +3.6%
rober/1e-8 1.02 ms 1.03 ms +1.0%
vdp_stiff/1e-6 3.16 ms 3.30 ms +4.4%
vdp_stiff/1e-8 19.73 ms 21.01 ms +6.5%
hires/1e-6 0.71 ms 0.73 ms +2.8%
hires/1e-8 3.26 ms 3.33 ms +2.1%
pollution/1e-6 0.53 ms 0.52 ms -1.9%
pollution/1e-8 2.08 ms 2.02 ms -2.9%
------------------------------------
Geomean -1.3%
All 10 configurations produce byte-identical (njacs, naccept,
nreject) sequences to master for Rosenbrock23. The remaining wall-
time deltas are measurement noise on sub-millisecond solves.
Previous Rosenbrock23 defaults on the same problem set:
γ=0.30 (original): +19.8% slower (Van der Pol +131.8%)
γ=0.03 (last tune): +1.4% slower (Van der Pol +43.4%)
max_jac_age=1 (this commit): -1.3% (Van der Pol +5.4%)
Strict Rosenbrock methods are unaffected — 144-config trace
fingerprint test still byte-identical to master.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: ChrisRackauckas-Claude <accounts@chrisrackauckas.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent 8d5ed6a commit 689d0b7
9 files changed
Lines changed: 384 additions & 43 deletions
File tree
- lib
- DelayDiffEq/test/interface
- OrdinaryDiffEqDifferentiation/src
- OrdinaryDiffEqRosenbrock/src
- test
- interface
- regression
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
54 | 54 | | |
55 | 55 | | |
56 | 56 | | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
57 | 62 | | |
58 | 63 | | |
59 | 64 | | |
| |||
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
| 47 | + | |
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| |||
Lines changed: 258 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
3 | 172 | | |
4 | 173 | | |
5 | 174 | | |
| |||
686 | 855 | | |
687 | 856 | | |
688 | 857 | | |
| 858 | + | |
689 | 859 | | |
690 | 860 | | |
691 | 861 | | |
692 | | - | |
693 | | - | |
694 | | - | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
| 866 | + | |
| 867 | + | |
| 868 | + | |
| 869 | + | |
| 870 | + | |
| 871 | + | |
| 872 | + | |
| 873 | + | |
| 874 | + | |
| 875 | + | |
| 876 | + | |
| 877 | + | |
| 878 | + | |
| 879 | + | |
| 880 | + | |
| 881 | + | |
695 | 882 | | |
696 | 883 | | |
697 | 884 | | |
698 | 885 | | |
699 | 886 | | |
700 | 887 | | |
| 888 | + | |
| 889 | + | |
| 890 | + | |
| 891 | + | |
| 892 | + | |
| 893 | + | |
| 894 | + | |
| 895 | + | |
| 896 | + | |
| 897 | + | |
| 898 | + | |
| 899 | + | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
701 | 956 | | |
702 | 957 | | |
703 | 958 | | |
| |||
Lines changed: 2 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
| 39 | + | |
38 | 40 | | |
39 | 41 | | |
40 | 42 | | |
| |||
0 commit comments