Add reusable homotopy solver caches

[ChrisRackauckas-Claude]

Please ignore this draft until it has been reviewed by @ChrisRackauckas.

Summary

• Add reusable init/reinit!/solve! caches for HomotopySweep and
  KantorovichHomotopy.
• Reuse continuation buffers and the inner nonlinear-solver cache across complete
  homotopy solves, including parameter, initial-value, and same-typed solver-option
  updates.
• Propagate option updates through every retained polyalgorithm subcache so later
  fallback uses the current tolerances.
• Keep dense ForwardDiff Jacobian writes and continuation correctors allocation-free
  on Julia 1.12.
• Add the documented NonlinearSolveBase.solve_cache! driver for nested solvers
  that need per-iteration observations without constructing a NonlinearSolution.
• Bump NonlinearSolveBase to 2.40.0 and NonlinearSolve to 4.24.0.

The first dense-Jacobian allocation fix is also available independently as #1116.
This PR retains the complete series needed by the reusable outer cache.

Why

The continuation drivers already reused an inner nonlinear cache within one homotopy
sweep, but the outer HomotopyProblem interface only implemented one-shot solve.
A caller solving a sequence of same-typed homotopy problems therefore rebuilt the
outer buffers and inner solver workspace every time. This was the path used by
OrdinaryDiffEq's HomotopyNonlinearSolveAlg for successive implicit stages.

The new outer cache owns the problem, fixed-lambda residual, inner cache, continuation
buffers, and controller state. reinit! updates u0, p, and existing same-typed
options; solve! resets the path state in place and drives the cached corrector.
Resizing remains the caller's responsibility because a changed state dimension
requires rebuilding the typed inner workspace.

Failure solutions now retain the most recent inner residual on stalled/max-step
paths. This keeps the cached return type consistent and provides more information
than the previous nothing residual.

Allocation behavior

On Julia 1.12.6, the warmed complete-cache reinit! plus solve! path allocates
exactly 0 bytes. The existing per-continuation-step regressions also remain exactly
zero after rebasing onto the current homotopy routing and
FastShortcutHomotopyPolyalg work.

The warmed solve_cache! path with a stateful step observer also allocates exactly
0 bytes. This is the cache driver used by the dependent ImplicitDiscreteSolve change
to collect Newton contraction rates without reaching into NonlinearSolve internals.

Local validation

• Exact head dca80d83c on upstream/master 22190d403.
• Official root NONLINEARSOLVE_TEST_GROUP=Core on Julia 1.12.6:
  Testing NonlinearSolve tests passed.
• Allocation-free observed cache solve: 8/8 passed, with exact warmed 0-byte
  allocation.
• Reusable outer cache: 24/24 passed.
• Concrete inner cache and exact zero-allocation stepping: 17/17 passed.
• Kantorovich controller: 51/51 passed.
• Current-master standard-algorithm routing and fast homotopy polyalgorithm:
  19/19 and 12/12 passed.
• Official NonlinearSolveBase Core on Julia 1.12.6:
  Testing NonlinearSolveBase tests passed, including Jacobian/restructure
  allocation fast paths 6/6 and dense LU allocation checks 14/14.
• Released SciMLTesting 2.6.0 prevents Base QA from starting due the
  already-fixed Aqua treat_as_own conversion bug. Against the merged SciMLTesting
  SciMLTesting.jl#37 source,
  public API documentation passed 2/2 and 19/20 QA checks passed; the remaining
  pre-existing public-owner check is addressed by Use public APIs for AutoDePSpecialize wrappers #1121.
• Repository-wide Runic --check: exit 0.
• git diff --check: exit 0.

No test was skipped, silenced, or loosened.

[ChrisRackauckas-Claude]

Implementation scratchpad:

1. Reproduced the unchanged allocation regression on clean current master b92a835e8, not just on Raise LinearSolve downgrade floor to 5.0.1 #1099: Julia 1.12.0/1.12.6/1.13.0-rc1 all measured exactly 154.4/80.0 B per sweep/arclength step, while Julia 1.11.9 measured 0.0/0.0.
2. Isolated the new-version bytes to dense ForwardDiff destination reshaping/same-buffer restructuring plus the 80-byte solution object built by each corrector.
3. Ported the three focused changes fresh onto current master, preserving the public API and allocation thresholds.
4. The first full Core run exposed a composability bug in the cache-only path for a custom solver cache without termination_cache. The final implementation now uses the allocation-free path only for caches supporting the standard InternalAPI.step! interface and standard finalization fields; custom caches retain their own CommonSolve.solve!. An explicit NonlinearSolution regression assertion covers this.
5. Re-ran full Core from clean detached worktrees on Julia 1.12.6 and 1.13.0-rc1 to actual exit 0, plus full NonlinearSolveBase Core on Julia 1.10.11 and 1.12.6 and whole-tree Runic.

[ChrisRackauckas-Claude]

Validation update for exact head 0feaad238964c48d6d8c1b4a2cfc05fb8f0761a0:

• GitHub reports 93 checks: 75 successful, 15 failed, and 3 skipped. The allocation-relevant lanes are green, including root Core on Julia current/pre Ubuntu, all functional sublibrary lanes, docs, Runic, downstreams, and downgrade sublibraries.
• Nine failures are the current-master SciMLTesting 2.4 QA docs/reexport set (with the additional known SingleCacheStorage internal-access report in NonlinearSolveBase). The three macOS Core Brown failures and the CUDA cuSOLVER QR DimensionMismatch failures also match clean-base failures.
• The remaining two failures are identical Brown-almost-linear algorithm Make iip a compile-time constant #4 residuals (5.5 > 0.001) in Julia 1.10 Ubuntu Core job 88580419743 and downgrade job 88580344763. They did not reproduce locally on Julia 1.10.11 with the PR root and all six local sublibraries: the focused Broyden file under coverage passed all 23 problems/five algorithms, and the exact CI-style NONLINEARSOLVE_TEST_GROUP=Core Pkg.test(; coverage=true, julia_args=["--check-bounds=auto", "--compiled-modules=yes", "--depwarn=yes"], force_latest_compatible_version=false, allow_reresolve=true) completed with Testing NonlinearSolve tests passed (exit 0).
• Exact job rerun requests for 88580419743 and 88580344763 were rejected with HTTP 403 (Must have admin rights to Repository). Whole-run retries for 29813781422 and 29813781139 were also refused (workflow file may be broken), so no retry run/job IDs were created.

No source change, synthetic commit, or rebase was made for these runner-only failures.

[ChrisRackauckas-Claude]

CI checkpoint (2026-07-22 08:31 EDT): Linux Core on LTS/release/prerelease, NonlinearSolveBase Core on all versions, homotopy-continuation Core and downgrade, CUDA, allocations, downstreams, docs, and Runic pass. The macOS Core failures are the same Brown almost-linear alg #4 failure present on untouched master run 29902380036 across all macOS Julia lanes; a separate clean-master bisect/fix investigation is in progress. The macOS LTS Adjoint job aborted inside the Julia compiler after a large inference dump, while the other five Adjoint lanes pass; a job-only rerun was attempted but the token lacks repository-admin permission. QA failures are the existing SciMLTesting 2.4 public-reexport audit described in the PR body.

[ChrisRackauckas-Claude]

Downstream integration is now in draft SciML/OrdinaryDiffEq.jl#3985.

That PR replaces the per-stage one-shot homotopy solve in HomotopyNonlinearSolveAlg with the public init / SciMLBase.reinit! / solve! cache path from this PR. The official OrdinaryDiffEqNonlinearSolve Core group resolved this local NonlinearSolve 4.23/Base 2.37 tree with LinearSolve 5.0.1 and exited 0; its homotopy file passed 38/38, including time-dependent per-stage parameter refresh, cache rebuild after integrator resize, and exact warmed-stage @allocated == 0 assertions for HomotopySweep and KantorovichHomotopy.

The ODE integration remains draft and explicitly depends on this PR.

[ChrisRackauckas-Claude]

Rebased the four-commit series onto current upstream/master (2b0dda5f6) and
force-updated the feature branch only after confirming that the remote branch still
contained exactly the prior four commits.

Local verification on exact head 5b44f3ea8 with Julia 1.12.6:

• NONLINEARSOLVE_TEST_GROUP=Core ... Pkg.test() exited 0 with
  Testing NonlinearSolve tests passed.
• The reusable-cache block passed 24/24, the concrete-cache/zero-allocation block
  passed 17/17, and Kantorovich passed 51/51.
• The current-master homotopy routing and FastShortcutHomotopyPolyalg blocks passed
  19/19 and 12/12, respectively.
• NonlinearSolveBase Core exited 0, including Jacobian/restructure allocation fast
  paths 6/6 and dense LU allocation checks 14/14.
• Whole-tree Runic --check and git diff --check exited 0.

The PR is now mergeable with master. No test was skipped, silenced, or loosened.

[ChrisRackauckas-Claude]

Clean-master allocation follow-up at 2b0dda5f6:

• The official root Core group on Julia 1.12.6 with LinearSolve 5.2.0 reproduces the unchanged failure exactly: sweep_per_step = 92.64 B and arclength_per_step = 48.0 B, against the existing strict < 48 assertions.
• The same master and LinearSolve 5.2.0 measure 0.0 / 0.0 B on Julia 1.11.9. Julia 1.12.0-beta1 (the earliest available 1.12 prerelease), 1.12.0, and 1.12.6 all reproduce 92.64 / 48.0 B.
• Revision 0343930c2 (Make AutoSpecialize infer through functor residuals; zero-alloc homotopy stepping #1046) likewise measures 0.0 / 0.0 B on Julia 1.11.9 but already fails on Julia 1.12. Therefore no later NonlinearSolve master commit introduced this remaining regression; Julia 1.12 exposed the latent allocation from materializing a full NonlinearSolution at every reused-cache corrector through CommonSolve.solve!(cache).
• Commit 78a6177d5 in this PR contains the substantive fix: the continuation hot loops run supported caches to completion without constructing an intermediate solution.

Fresh isolated-worktree validation on Julia 1.12.6 + LinearSolve 5.2.0:

• 78a6177d5: official homotopy allocation file 15/15; cache-only solve 0 B, sweep 0.0 B/step, arclength 0.0 B/step.
• Current head 5b44f3ea8: official homotopy allocation file 17/17 with the same exact zero measurements.
• Kantorovich controller 51/51, bounded-solve path 20/20, and arclength cache allocation checks 8/8.
• Whole-repository Runic --check and git diff --check exited 0.

No threshold was changed, loosened, skipped, or silenced; a duplicate PR is unnecessary because #1100 already fixes the clean-master failure.

[ChrisRackauckas-Claude]

Exact downstream integration validation against this PR's current head (5b44f3ea8) is now complete in SciML/OrdinaryDiffEq.jl#3985.

On OrdinaryDiffEq master plus the single cache-integration commit, Julia 1.12.6 with local NonlinearSolve 4.24 / NonlinearSolveBase 2.40:

• official OrdinaryDiffEqNonlinearSolve Core exited 0; homotopy tests 38/38 and default-nlsolve initialization tests 16/16;
• official QA exited 0 (JET 13 passed / 33 existing broken; Aqua 19/19);
• the added warmed implicit-stage assertions measured exactly 0 bytes for both reusable HomotopySweep and KantorovichHomotopy caches;
• a 20-sample, 1,000-stage benchmark measured 7.460 μs and 0 bytes/stage with reuse versus 9.244 μs and 5,520 bytes/stage when forcing a cache rebuild (1.239× faster), with identical endpoints.

This complements the Julia 1.12 continuation-corrector allocation evidence above: the new reusable outer caches work through the public init/reinit!/solve! path in an actual implicit ODE integrator.

[ChrisRackauckas-Claude]

Rebased the four-commit series onto current upstream/master (1d51c8dac) and
force-updated the feature branch only after confirming that the remote branch still
contained exactly the prior four commits.

Local verification on exact head dea394fd6 with Julia 1.12.6:

• NONLINEARSOLVE_TEST_GROUP=Core ... Pkg.test() exited 0 with
  Testing NonlinearSolve tests passed.
• The reusable-cache block passed 24/24, the concrete-cache/zero-allocation block
  passed 17/17, and Kantorovich passed 51/51.
• The current-master homotopy routing and FastShortcutHomotopyPolyalg blocks passed
  19/19 and 12/12, respectively.
• NonlinearSolveBase Core exited 0, including Jacobian/restructure allocation fast
  paths 6/6 and dense LU allocation checks 14/14.
• Whole-tree Runic --check and git diff --check exited 0.

The PR is now mergeable with master. No test was skipped, silenced, or loosened.

[ChrisRackauckas-Claude]

The DiffEqCallbacks downstream failure in job 90053534882 reproduces unchanged against clean NonlinearSolve master 1d51c8d (v4.23.2), so it is independent of this PR. The first resolver wall is DiffEqCallbacks test compat ForwardDiff 0.10 versus current NonlinearSolve ForwardDiff 1; clearing it exposed stale DataInterpolations 5–8 versus FindFirstFunctions 3 and the SciMLBase v2/v3 EnsembleProblem prob_func signature transition. A focused owner-side fix is now draft PR SciML/DiffEqCallbacks.jl#329. Exact downstream replay against clean NonlinearSolve master passes locally, as do DiffEqCallbacks Core, AD under ForwardDiff 1.4.2, QA, and Runic.

[ChrisRackauckas-Claude]

Rebased the four-commit series onto current upstream/master (22190d403) and
force-updated the feature branch only after confirming that the remote branch still
contained exactly the prior four commits.

git range-diff reported all four commits as patch-equivalent across the rebase.

Local verification on exact head 04e7c572e with Julia 1.12.6:

• NONLINEARSOLVE_TEST_GROUP=Core ... Pkg.test() exited 0 with
  Testing NonlinearSolve tests passed.
• The reusable-cache block passed 24/24, the concrete-cache/zero-allocation block
  passed 17/17, and Kantorovich passed 51/51.
• The current-master homotopy routing and FastShortcutHomotopyPolyalg blocks passed
  19/19 and 12/12, respectively.
• NonlinearSolveBase Core exited 0, including Jacobian/restructure allocation fast
  paths 6/6 and dense LU allocation checks 14/14.
• Whole-tree Runic --check and git diff --check exited 0.

The PR is now mergeable with master. No test was skipped, silenced, or loosened.

[ChrisRackauckas-Claude]

Added commits 8344032f4 and dca80d83c, which expose the documented NonlinearSolveBase.solve_cache! API for allocation-sensitive nested solvers and reject caches that do not support the iterator contract. The optional observer receives (u, residual, iteration) after each step, while the driver returns the final ReturnCode without constructing a NonlinearSolution.

Local Julia 1.12.6 verification on exact head dca80d83c:

• Focused API regression passed 8/8 and measured exactly 0 bytes for the warmed observed cache solve.
• Official root NONLINEARSOLVE_TEST_GROUP=Core exited 0 with Testing NonlinearSolve tests passed.
• Reusable homotopy cache 24/24, continuation allocation 17/17, Kantorovich 51/51, routing 19/19, and fast homotopy polyalgorithm 12/12 all passed.
• Official NonlinearSolveBase Core exited 0; allocation fast paths passed 6/6 and dense LU allocation checks passed 14/14.
• Base QA with released SciMLTesting 2.6.0 reproduced its existing pre-check crash. Against merged SciMLTesting.jl#37, public API documentation passed 2/2 and 19/20 QA checks passed; the sole remaining existing public-owner failure is addressed by Use public APIs for AutoDePSpecialize wrappers #1121.
• Whole-repository Runic and git diff --check exited 0.

No test was skipped, silenced, or loosened.

[ChrisRackauckas-Claude]

Final CI attribution for exact head dca80d83c: 71 checks passed, 3 were skipped, and 21 were red. I inspected the completed logs and reproduced the relevant cases locally; no red job is attributable to the homotopy/cache-driver changes.

• 6 Core/downgrade jobs: all fail only Brown almost linear function | alg #4 in the existing robustness matrix (for example residual 5.5 on x64 and 1.000005... on macOS arm64). Clean master 22190d403 and this PR produced the same local trajectory five-for-five (7.314149286230531e-13, 174 steps) in the exact downgrade environment. The earlier Watson failure was non-reproducible and passed in the final downgrade job. The separate platform-robustness change remains Stabilize unglobalized bad Broyden steps #1101.
• 9 QA jobs: released SciMLTesting 2.6.0's Aqua treat_as_own conversion issue plus existing unapproved-reexport/public-owner checks. The harness defect is fixed by merged SciMLTesting.jl#37; the repository follow-ups are Fix intentional sublibrary re-export QA failures #1120 and Use public APIs for AutoDePSpecialize wrappers #1121. Against CompatHelper: bump compat for "RecursiveFactorization" to "0.2" #37 source, the new public API docs passed 2/2 and QA reached 19/20, with the remaining owner check covered by Use public APIs for AutoDePSpecialize wrappers #1121.
• 5 downstream jobs: three ModelingToolkit jobs fail during dependency resolution on the current LinearSolve/OrdinaryDiffEqDefault compat combination; BoundaryValueDiffEq reproduces on clean master with a tagged ForwardDiff.Dual entering a persistent Float64 shooting cache; DiffEqCallbacks is the independent compatibility failure covered by Allow current SciML dependency stack in tests DiffEqCallbacks.jl#329. The MTK and BVD clean-master bisect/fix investigations are ongoing separately.
• 1 Runic-suggestions job: Runic itself checked all 385 files successfully; reviewdog installation then failed with curl: (35) Recv failure: Connection reset by peer. The real Runic CI job and the local whole-repository Runic check both pass.

Relevant behavioral lanes are green: root Core on Julia current, HomotopyContinuation Core, NonlinearSolveBase Core on LTS/current/pre, every downgrade sublibrary (including Base, FirstOrder, QuasiNewton, and HomotopyContinuation), CUDA, wrappers, polyalgorithms, trim, adjoint, and root downstream groups. No test was skipped, silenced, or loosened in response to these failures.