Add R2N and R2NLS solvers with HSL and QRMumps support by farhadrclass · Pull Request #347 · JuliaSmoothOptimizers/JSOSolvers.jl

farhadrclass · 2026-01-13T12:46:24Z

Introduces new second-order quadratic regularization solvers R2N and R2NLS for unconstrained and nonlinear least-squares optimization. Adds support for HSL (MA97, MA57) and QRMumps direct solvers, updates documentation and README, and extends test coverage for the new solvers. Updates dependencies and compat entries in Project.toml.

dpo

Lots more comments on R2NLS.

dpo

I propose a cleaner subsolver API.

farhadrclass · 2026-02-11T00:17:24Z

@dpo I have refactor the R2NLS,

dpo

That looks great. Here are a few minor comments.

farhadrclass · 2026-02-16T18:21:19Z

@dpo pushed the R2NLS now,
moving to the R2N

dpo

Just a few more comments. Starting to look great!

farhadrclass · 2026-02-21T02:28:25Z

@dpo I didn't move x_init to x0 since by doing so, we would remove the connection between LinearOp jac relying on x to be update to update the Jacobean, I can however add a new x_op to solver structure but that would allocate a memory and at the end do the same thing,

In the constructor, I am linking x which will be updated in each iteration to jacobean

Let me know what you think

Add Arpack, GenericLinearAlgebra and TSVD to Project.toml (deps and compat). Import Arpack, TSVD and GenericLinearAlgebra in JSOSolvers. Export MA97R2NSubsolver and MA57R2NSubsolver in R2N. Relax several parameter intervals (η1, γ3, σmin) to allow closed lower bound (zero). Qualify calls to estimate_opnorm and reset! with LinearOperators to avoid ambiguity. Broaden subsolver checks to accept either a Type or a Function in R2N and R2NLS. Remove an unused reset! call in initialize_subsolver!.

Move R2N and R2NLS subsolver implementations into dedicated files and unify the subsolver interface. Added src/R2N_subsolvers.jl and src/R2NLS_subsolvers.jl, renamed sub_solver_common.jl to src/r2n_subsolver_common.jl, and updated JSOSolvers.jl to include the new files. Standardized API: initialize_subsolver!/update_subsolver!/update_jacobian!/solve_subproblem! were replaced with initialize!/update! and subsolvers are now callable functors (sub(s, rhs, σ, atol, rtol...)). R2N/R2NLS constructors and solver code were adjusted to accept subsolver instances by default (and to instantiate types when provided), and operator/jacobian accessors and norm estimators were added/updated. Also includes minor formatting, logging, and calling-site fixes to use the new interface and helpers. This is a breaking change to the subsolver API and requires updating external code that implemented the old callbacks.

When computing the NPC direction, use the subsolver-provided vector `s` for HSL (HSLR2NSubsolver) instead of calling get_npc_direction. This ensures the line-search model is redirected to the correct direction for the HSL subsolver; other subsolvers still use get_npc_direction. A clarifying comment was added.

Replace the Goldstein-only NPC handler with an Armijo-Goldstein (:ag) strategy and make it the default. Add two new optional solver flags: `always_accept_npc_ag` (skip model-ratio computation and treat NPC AG steps as very successful to aggressively escape saddles) and `fast_local_convergence` (on very successful iterations scale σ using `γ3 * min(σ, ||g||)` to accelerate local convergence). Update npc_handler allowed values, docs, variable names, and the solver flow to mark NPC AG steps and branch to either skip the predicted-reduction computation or compute the usual model-predicted reduction and ρ. Adjust σ update logic accordingly.

Replace the generic update!(...) subsolver API with a more specific update_subsolver!(...) across the codebase. Updated call sites (src/R2N.jl, src/R2NLS.jl), subsolver implementations (src/R2NLS_subsolvers.jl, src/R2N_subsolvers.jl), and the exported symbol in src/r2n_subsolver_common.jl. Also adjusted the QRM update call to qrm_update_subsolver! where the subsolver updates the sparse values. This clarifies the API, avoids potential name collisions, and makes subsolver update intent explicit.

Call the subsolver with an explicit zero atol (pass T(0.0)) and set the subsolver operator's σ on sub.A.data.σ instead of sub.A. Adjust the adaptive subtol update to use sqrt(eps(T)) as the lower bound rather than rtol. Minor whitespace tidy and TODO comments left in place for future cleanup.

Allow passing subsolvers as Functions (constructors) or Types in R2N and R2NLS by extending the param unions and updating instantiation logic to call subsolver(nlp) when subsolver is a Type or Function. Adjusted default parameter types accordingly. Add Quadmath to Project.toml and add Float128/Quadmath and LBFGS usage examples to the test script (R2N_test.jl) to exercise the new constructor-style subsolver and quasi-Newton wrapping.

fix H not updating

Remove stored gradient-difference state and inline quasi-Newton updates from R2NSolver/solve!. Introduce a callback_quasi_newton argument (with default) and invoke it before the main callbacks and again each iteration, so quasi-Newton models can update their Hessian approximations externally. Removed the y field, its allocations and uses, and the direct push! of s,y into QuasiNewtonModel; added set_step_status! calls when steps are accepted/rejected. This decouples quasi-Newton bookkeeping from the solver core and provides a flexible callback hook for Hessian updates.

farhadrclass commented Jan 20, 2026

View reviewed changes