JSO Compliance: TR & TRDH by MaxenceGollier · Pull Request #199 · JuliaSmoothOptimizers/RegularizedOptimization.jl

MaxenceGollier · 2025-07-11T22:46:45Z

@dpo did the two solvers in one PR because i couldn't modify TR if TRDH was not udpated (due to when I use solve! in TR to call the subsolver)
Also, a few TODOs in the doc because i am not sure what some parameters mean.
Finally, as in #172, the solvers behaviours remain the same and there are no allocs (there are still for TR since opnorm allocates).

cc. @MohamedLaghdafHABIBOULLAH

I will showcase tomorrow that results here are identical to the master branch.

…dOptimization.jl into TR-JSO

codecov · 2025-07-11T22:50:18Z

Codecov Report

❌ Patch coverage is 83.04598% with 59 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.50%. Comparing base (e0f214d) to head (a7df86f).
⚠️ Report is 124 commits behind head on master.

Files with missing lines	Patch %	Lines
src/TRDH_alg.jl	72.82%	50 Missing ⚠️
src/TR_alg.jl	94.30%	9 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           master     #199       +/-   ##
===========================================
+ Coverage   61.53%   77.50%   +15.97%     
===========================================
  Files          11       12        +1     
  Lines        1292     1525      +233     
===========================================
+ Hits          795     1182      +387     
+ Misses        497      343      -154

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

MaxenceGollier · 2025-07-12T16:00:31Z

TRDH Tests

(NB: I tweaked the parameters, $$\Delta k, \alpha$$ etc. so that the algorithm would take a few iterations before converging)

For TRDH (without reduce_TR)

On this branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "TR-JSO")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 1, reduce_TR = false)

TRDH(bpdn, h, χ, options)
[ Info:   iter     f(x)     h(x)   √(ξ/ν)        ρ        Δ      ‖x‖      ‖s‖      ‖D‖ TRDH
[ Info:      0  1.8e+00  0.0e+00  5.1e+00  1.0e+00  1.0e-06  0.0e+00  1.0e-06  1.0e+00               ↗
[ Info:      1  1.8e+00  1.1e-05  8.2e-03  1.0e+00  3.0e-06  1.0e-06  3.0e-06  7.4e-01               ↗
[ Info:      2  1.8e+00  4.4e-05  1.4e-02  1.0e+00  9.0e-06  4.0e-06  9.0e-06  7.4e-01               ↗
[ Info:      3  1.8e+00  1.4e-04  2.5e-02  1.0e+00  2.7e-05  1.3e-05  2.7e-05  7.4e-01               ↗
[ Info:      4  1.8e+00  4.4e-04  4.3e-02  1.0e+00  8.1e-05  4.0e-05  8.1e-05  7.4e-01               ↗
[ Info:      5  1.8e+00  1.3e-03  7.4e-02  1.0e+00  2.4e-04  1.2e-04  2.4e-04  7.4e-01               ↗
[ Info:      6  1.8e+00  4.0e-03  1.3e-01  1.0e+00  7.3e-04  3.6e-04  7.3e-04  7.4e-01               ↗
[ Info:      7  1.8e+00  1.2e-02  2.2e-01  1.0e+00  2.2e-03  1.1e-03  2.2e-03  7.5e-01               ↗
[ Info:      8  1.7e+00  3.6e-02  3.7e-01  1.0e+00  6.6e-03  3.3e-03  6.6e-03  7.5e-01               ↗
[ Info:      9  1.6e+00  1.0e-01  5.8e-01  1.0e+00  2.0e-02  9.8e-03  2.0e-02  7.5e-01               ↗
[ Info:     10  1.2e+00  2.7e-01  7.6e-01  9.9e-01  5.9e-02  3.0e-02  5.9e-02  7.7e-01               ↗
[ Info:     11  5.9e-01  5.7e-01  6.0e-01  1.2e+00  1.8e-01  8.9e-02  1.8e-01  8.0e-01               ↗
[ Info:     12  2.2e-01  7.1e-01  3.7e-01  1.8e+00  5.3e-01  2.7e-01  2.0e-01  6.3e-01               ↗
[ Info:     13  2.0e-01  5.8e-01  5.5e-01  9.5e-01  6.1e-01  4.7e-01  6.1e-01  1.0e-01               ↗
[ Info:     14  3.6e-02  4.9e-01  5.1e-01 -1.2e+00  1.8e+00  1.1e+00  8.7e-01  1.1e-01               ↘
[ Info:     15  3.6e-02  4.9e-01  5.1e-01 -1.2e+00  9.2e-01  1.1e+00  8.7e-01  1.1e-01               ↘
[ Info:     16  3.6e-02  4.9e-01  4.9e-01 -3.0e-01  4.6e-01  1.1e+00  4.6e-01  1.1e-01               ↘
[ Info:     17  3.6e-02  4.9e-01  4.3e-01  3.1e-01  2.3e-01  1.1e+00  2.3e-01  1.1e-01               =
[ Info:     18  1.3e-01  3.4e-01  2.0e-01  1.2e+00  2.3e-01  8.9e-01  1.4e-01  5.0e-01               ↗
[ Info:     19  5.9e-02  3.8e-01  6.4e-02  1.2e+00  4.2e-01  8.7e-01  4.9e-02  3.9e-01               ↗
[ Info:     20  4.4e-02  3.9e-01  1.3e-02  9.2e-01  4.2e-01  8.9e-01  1.6e-02  3.0e-01               ↗
[ Info:     21  4.0e-02  3.9e-01  3.0e-03  7.2e-01  4.2e-01  8.9e-01  4.0e-03  3.2e-01               =
[ Info:     22  4.0e-02  3.9e-01  1.3e-03  6.9e-01  4.2e-01  8.9e-01  1.1e-03  4.1e-01               =
[ Info:     23  4.1e-02  3.9e-01  4.6e-04  1.0e+00  4.2e-01  8.9e-01  4.1e-04  5.4e-01               ↗
[ Info:     24  4.0e-02  3.9e-01  7.6e-05  1.5e+00  4.2e-01  8.9e-01  5.0e-05  5.3e-01               ↗
[ Info:     25  4.0e-02  3.9e-01  5.2e-05  1.1e+00  4.2e-01  8.9e-01  7.5e-05  2.6e-01               ↗
[ Info:     26  4.0e-02  3.9e-01  5.6e-06  1.1e+00  4.2e-01  8.9e-01  7.5e-06  2.4e-01
[ Info: TRDH: terminating with √(ξ/ν) = 5.619761355129398e-6

On the master branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl.git")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, β = 1.0, α = 1.0, ϵa = 1e-6, ϵr = 1e-6, verbose = 10, reduce_TR = false)

TRDH(bpdn, h, χ, options)
[ Info:  outer     f(x)     h(x)  √(ξ/ν)        ρ       Δ     ‖x‖     ‖s‖    ‖Dₖ‖ TRDH
[ Info:      1  1.8e+00  0.0e+00 5.1e+00  1.0e+00 1.0e-06 0.0e+00 1.0e-06 1.0e+00 ↗
[ Info:      2  1.8e+00  1.1e-05 8.2e-03  1.0e+00 3.0e-06 1.0e-06 3.0e-06 7.4e-01 ↗
[ Info:      3  1.8e+00  4.4e-05 1.4e-02  1.0e+00 9.0e-06 4.0e-06 9.0e-06 7.4e-01 ↗
[ Info:      4  1.8e+00  1.4e-04 2.5e-02  1.0e+00 2.7e-05 1.3e-05 2.7e-05 7.4e-01 ↗
[ Info:      5  1.8e+00  4.4e-04 4.3e-02  1.0e+00 8.1e-05 4.0e-05 8.1e-05 7.4e-01 ↗
[ Info:      6  1.8e+00  1.3e-03 7.4e-02  1.0e+00 2.4e-04 1.2e-04 2.4e-04 7.4e-01 ↗
[ Info:      7  1.8e+00  4.0e-03 1.3e-01  1.0e+00 7.3e-04 3.6e-04 7.3e-04 7.4e-01 ↗
[ Info:      8  1.8e+00  1.2e-02 2.2e-01  1.0e+00 2.2e-03 1.1e-03 2.2e-03 7.5e-01 ↗
[ Info:      9  1.7e+00  3.6e-02 3.7e-01  1.0e+00 6.6e-03 3.3e-03 6.6e-03 7.5e-01 ↗
[ Info:     10  1.6e+00  1.0e-01 5.8e-01  1.0e+00 2.0e-02 9.8e-03 2.0e-02 7.5e-01 ↗
[ Info:     11  1.2e+00  2.7e-01 7.6e-01  9.9e-01 5.9e-02 3.0e-02 5.9e-02 7.7e-01 ↗
[ Info:     12  5.9e-01  5.7e-01 6.0e-01  1.2e+00 1.8e-01 8.9e-02 1.8e-01 8.0e-01 ↗
[ Info:     13  2.2e-01  7.1e-01 3.7e-01  1.8e+00 5.3e-01 2.7e-01 2.0e-01 6.3e-01 ↗
[ Info:     14  2.0e-01  5.8e-01 5.5e-01  9.5e-01 6.1e-01 4.7e-01 6.1e-01 1.0e-01 ↗
[ Info:     15  3.6e-02  4.9e-01 5.1e-01 -1.2e+00 1.8e+00 1.1e+00 8.7e-01 1.1e-01 ↘
[ Info:     16  3.6e-02  4.9e-01 5.1e-01 -1.2e+00 9.2e-01 1.1e+00 8.7e-01 1.1e-01 ↘
[ Info:     17  3.6e-02  4.9e-01 4.9e-01 -3.0e-01 4.6e-01 1.1e+00 4.6e-01 1.1e-01 ↘
[ Info:     18  3.6e-02  4.9e-01 4.3e-01  3.1e-01 2.3e-01 1.1e+00 2.3e-01 1.1e-01 =
[ Info:     19  1.3e-01  3.4e-01 2.0e-01  1.2e+00 2.3e-01 8.9e-01 1.4e-01 5.0e-01 ↗
[ Info:     20  5.9e-02  3.8e-01 6.4e-02  1.2e+00 4.2e-01 8.7e-01 4.9e-02 3.9e-01 ↗
[ Info:     21  4.4e-02  3.9e-01 1.3e-02  9.2e-01 4.2e-01 8.9e-01 1.6e-02 3.0e-01 ↗
[ Info:     22  4.0e-02  3.9e-01 3.0e-03  7.2e-01 4.2e-01 8.9e-01 4.0e-03 3.2e-01 =
[ Info:     23  4.0e-02  3.9e-01 1.3e-03  6.9e-01 4.2e-01 8.9e-01 1.1e-03 4.1e-01 =
[ Info:     24  4.1e-02  3.9e-01 4.6e-04  1.0e+00 4.2e-01 8.9e-01 4.1e-04 5.4e-01 ↗
[ Info:     25  4.0e-02  3.9e-01 7.6e-05  1.5e+00 4.2e-01 8.9e-01 5.0e-05 5.3e-01 ↗
[ Info:     26  4.0e-02  3.9e-01 5.2e-05  1.1e+00 4.2e-01 8.9e-01 7.5e-05 2.6e-01 ↗
[ Info:     27  4.0e-02  3.9e-01 5.6e-06          4.2e-01 8.9e-01 7.5e-06 2.4e-01
[ Info: TRDH: terminating with √(ξ/ν) = 5.619761355129398e-6

For TRDH (with reduce_TR)

On this branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "TR-JSO")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, β = 1e12, α = 1e12, ϵa = 1e-6, ϵr = 1e-6, verbose = 1, reduce_TR = true)

TRDH(bpdn, h, χ, options)
[ Info:   iter     f(x)     h(x)   √(ξ/ν)        ρ        Δ      ‖x‖      ‖s‖      ‖D‖ TRDH
[ Info:      0  1.8e+00  0.0e+00  5.1e-03  1.0e+00  1.0e-06  0.0e+00  1.0e-06  1.0e+00               ↗
[ Info:      1  1.8e+00  1.1e-05  4.8e-06  3.0e+00  3.0e-06  1.0e-06  3.0e-06  7.4e-01               ↗
[ Info:      2  1.8e+00  4.4e-05  4.8e-06  9.0e+00  9.0e-06  4.0e-06  9.0e-06  7.4e-01               ↗
[ Info:      3  1.8e+00  1.4e-04  4.8e-06  2.7e+01  2.7e-05  1.3e-05  2.7e-05  7.4e-01               ↗
[ Info:      4  1.8e+00  4.4e-04  4.8e-06  8.1e+01  8.1e-05  4.0e-05  8.1e-05  7.4e-01               ↗
[ Info:      5  1.8e+00  1.3e-03  4.7e-06  2.4e+02  2.4e-04  1.2e-04  2.4e-04  7.4e-01               ↗
[ Info:      6  1.8e+00  4.0e-03  4.7e-06  7.2e+02  7.3e-04  3.6e-04  7.3e-04  7.4e-01               ↗
[ Info:      7  1.8e+00  1.2e-02  4.7e-06  2.1e+03  2.2e-03  1.1e-03  2.2e-03  7.5e-01               ↗
[ Info:      8  1.7e+00  3.6e-02  4.7e-06  6.0e+03  6.6e-03  3.3e-03  6.6e-03  7.5e-01               ↗
[ Info:      9  1.6e+00  1.0e-01  4.4e-06  1.5e+04  2.0e-02  9.8e-03  2.0e-02  7.5e-01               ↗
[ Info:     10  1.2e+00  2.7e-01  3.9e-06  2.5e+04  5.9e-02  3.0e-02  5.9e-02  7.7e-01               ↗
[ Info:     11  5.9e-01  5.7e-01  2.5e-06  1.8e+04  1.8e-01  8.9e-02  1.8e-01  8.0e-01               ↗
[ Info:     12  2.2e-01  7.1e-01  1.3e-06  1.1e+04  5.3e-01  2.7e-01  2.0e-01  6.3e-01               ↗
[ Info:     13  2.0e-01  5.8e-01  9.5e-07  1.1e+04  6.1e-01  4.7e-01  6.1e-01  1.0e-01
[ Info: TRDH: terminating with √(ξ/ν) = 9.498798951470827e-7

On the master branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl.git")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, β = 1e12, α = 1e12, ϵa = 1e-6, ϵr = 1e-6, verbose = 10, reduce_TR = true)

TRDH(bpdn, h, χ, options)
[ Info:  outer     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       Δ     ‖x‖     ‖s‖    ‖Dₖ‖ TRDH      
[ Info:      1  1.8e+00  0.0e+00 5.1e-06 3.6e-03  1.0e+00 1.0e-06 0.0e+00 1.0e-06 1.0e+00 ↗
[ Info:      2  1.8e+00  1.1e-05 4.8e-06 6.2e-03  1.0e+00 3.0e-06 1.0e-06 3.0e-06 7.4e-01 ↗
[ Info:      3  1.8e+00  4.4e-05 4.8e-06 1.1e-02  1.0e+00 9.0e-06 4.0e-06 9.0e-06 7.4e-01 ↗
[ Info:      4  1.8e+00  1.4e-04 4.8e-06 1.9e-02  1.0e+00 2.7e-05 1.3e-05 2.7e-05 7.4e-01 ↗
[ Info:      5  1.8e+00  4.4e-04 4.8e-06 3.2e-02  1.0e+00 8.1e-05 4.0e-05 8.1e-05 7.4e-01 ↗
[ Info:      6  1.8e+00  1.3e-03 4.7e-06 5.6e-02  1.0e+00 2.4e-04 1.2e-04 2.4e-04 7.4e-01 ↗
[ Info:      7  1.8e+00  4.0e-03 4.7e-06 9.7e-02  1.0e+00 7.3e-04 3.6e-04 7.3e-04 7.4e-01 ↗
[ Info:      8  1.8e+00  1.2e-02 4.7e-06 1.7e-01  1.0e+00 2.2e-03 1.1e-03 2.2e-03 7.5e-01 ↗
[ Info:      9  1.7e+00  3.6e-02 4.7e-06 2.8e-01  1.0e+00 6.6e-03 3.3e-03 6.6e-03 7.5e-01 ↗
[ Info:     10  1.6e+00  1.0e-01 4.4e-06 4.4e-01  1.0e+00 2.0e-02 9.8e-03 2.0e-02 7.5e-01 ↗
[ Info:     11  1.2e+00  2.7e-01 3.9e-06 5.7e-01  9.9e-01 5.9e-02 3.0e-02 5.9e-02 7.7e-01 ↗
[ Info:     12  5.9e-01  5.7e-01 2.5e-06 4.5e-01  1.2e+00 1.8e-01 8.9e-02 1.8e-01 8.0e-01 ↗
[ Info:     13  2.2e-01  7.1e-01 1.3e-06 2.9e-01  1.8e+00 5.3e-01 2.7e-01 2.0e-01 6.3e-01 ↗
[ Info:     14  2.0e-01  5.8e-01 9.5e-07 7.1e-01          6.1e-01 4.7e-01 6.1e-01 1.0e-01
[ Info: TRDH: terminating with √(ξ1/ν) = 9.498798951470827e-7

MaxenceGollier · 2025-07-12T17:47:31Z

TR Tests

For TR with R2

On this branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "TR-JSO")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, α = 1.0, ϵa = 1e-3, ϵr = 1e-3, verbose = 1)

TR(bpdn, h, χ, options, subsolver = R2Solver)
[ Info:  outer  inner     f(x)     h(x)  √(ξ1/ν)        ρ        Δ      ‖x‖      ‖s‖      ‖B‖ TR 
[ Info:      0     10  1.8e+00  0.0e+00  1.3e+00  1.0e+00  1.0e-06  0.0e+00  1.0e-06  1.0e+00               ↗
[ Info:      1      6  1.8e+00  1.1e-05  1.3e+00  1.0e+00  3.0e-06  1.0e-06  3.0e-06  7.4e-01               ↗
[ Info:      2      6  1.8e+00  4.3e-05  1.3e+00  1.0e+00  9.0e-06  4.0e-06  9.0e-06  7.5e-01               ↗
[ Info:      3      6  1.8e+00  1.4e-04  1.3e+00  1.0e+00  2.7e-05  1.3e-05  2.7e-05  7.5e-01               ↗
[ Info:      4      6  1.8e+00  4.2e-04  1.3e+00  1.0e+00  8.1e-05  4.0e-05  8.1e-05  7.5e-01               ↗
[ Info:      5      6  1.8e+00  1.3e-03  1.3e+00  1.0e+00  2.4e-04  1.2e-04  2.4e-04  7.5e-01               ↗
[ Info:      6      6  1.8e+00  3.9e-03  1.3e+00  1.0e+00  7.3e-04  3.6e-04  7.3e-04  7.5e-01               ↗
[ Info:      7      6  1.8e+00  1.2e-02  1.3e+00  1.0e+00  2.2e-03  1.1e-03  2.2e-03  7.5e-01               ↗
[ Info:      8      6  1.7e+00  3.4e-02  1.3e+00  1.0e+00  6.6e-03  3.3e-03  6.6e-03  7.5e-01               ↗
[ Info:      9      6  1.6e+00  1.0e-01  1.2e+00  1.0e+00  2.0e-02  9.8e-03  2.0e-02  7.6e-01               ↗
[ Info:     10      6  1.2e+00  2.7e-01  1.0e+00  9.9e-01  5.9e-02  3.0e-02  5.9e-02  7.7e-01               ↗
[ Info:     11     15  6.0e-01  5.6e-01  5.9e-01  1.2e+00  1.8e-01  8.9e-02  1.8e-01  8.0e-01               ↗
[ Info:     12      6  2.2e-01  7.0e-01  3.3e-01  1.8e+00  5.3e-01  2.7e-01  2.0e-01  6.3e-01               ↗
[ Info:     13      6  2.0e-01  5.9e-01  2.7e-01  1.0e+00  6.0e-01  4.7e-01  6.0e-01  1.1e-01               ↗
[ Info:     14      6  3.7e-02  4.8e-01  2.8e-01 -1.1e+00  1.8e+00  1.1e+00  8.1e-01  1.1e-01               ↘
[ Info:     15     11  3.7e-02  4.8e-01  3.0e-01 -1.0e+00  9.0e-01  1.1e+00  8.0e-01  1.1e-01               ↘
[ Info:     16      6  3.7e-02  4.8e-01  3.1e-01 -3.8e-01  4.5e-01  1.1e+00  4.5e-01  1.1e-01               ↘
[ Info:     17      5  3.7e-02  4.8e-01  3.1e-01  2.8e-01  2.3e-01  1.1e+00  2.3e-01  1.1e-01               =
[ Info:     18     13  1.4e-01  3.4e-01  1.7e-01  1.2e+00  2.3e-01  8.9e-01  1.4e-01  5.1e-01               ↗
[ Info:     19      5  6.2e-02  3.8e-01  6.3e-02  1.2e+00  4.2e-01  8.7e-01  4.1e-02  4.0e-01               ↗
[ Info:     20      3  4.8e-02  3.9e-01  1.8e-02  9.4e-01  4.2e-01  8.8e-01  1.4e-02  2.8e-01               ↗
[ Info:     21      0  4.3e-02  3.9e-01  6.4e-03  1.0e+00  4.2e-01  8.9e-01  8.6e-04  3.3e-01               ↗
[ Info:     22      0  4.3e-02  3.9e-01  5.8e-03  1.0e+00  4.2e-01  8.9e-01  7.9e-04  3.2e-01               ↗
[ Info:     23      0  4.3e-02  3.9e-01  5.3e-03  1.0e+00  4.2e-01  8.9e-01  7.2e-04  3.2e-01               ↗
[ Info:     24      0  4.2e-02  3.9e-01  4.8e-03  1.0e+00  4.2e-01  8.9e-01  6.6e-04  3.2e-01               ↗
[ Info:     25      0  4.2e-02  3.9e-01  4.3e-03  1.0e+00  4.2e-01  8.9e-01  6.0e-04  3.1e-01               ↗
[ Info:     26      0  4.2e-02  3.9e-01  4.0e-03  1.0e+00  4.2e-01  8.9e-01  5.5e-04  3.1e-01               ↗
[ Info:     27      0  4.2e-02  3.9e-01  3.6e-03  1.0e+00  4.2e-01  8.9e-01  5.0e-04  3.1e-01               ↗
[ Info:     28      0  4.2e-02  3.9e-01  3.3e-03  1.0e+00  4.2e-01  8.9e-01  4.5e-04  3.1e-01               ↗
[ Info:     29      0  4.2e-02  3.9e-01  3.0e-03  1.0e+00  4.2e-01  8.9e-01  4.1e-04  3.1e-01               ↗
[ Info:     30      0  4.2e-02  3.9e-01  2.7e-03  1.0e+00  4.2e-01  8.9e-01  3.7e-04  3.0e-01               ↗
[ Info:     31      0  4.1e-02  3.9e-01  2.5e-03  1.0e+00  4.2e-01  8.9e-01  3.4e-04  3.0e-01               ↗
[ Info:     32      0  4.1e-02  3.9e-01  2.3e-03  1.0e+00  4.2e-01  8.9e-01  3.1e-04  3.0e-01
[ Info: TR: terminating with √(ξ1/ν) = 0.002264931850548618

On the master branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl.git")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, α = 1.0, ϵa = 1e-3, ϵr = 1e-3, verbose = 10)

TR(bpdn, h, χ, options, subsolver = R2)
[ Info:  outer    inner     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       Δ     ‖x‖     ‖s‖    ‖Bₖ‖ TR
[ Info:      1       10  1.8e+00  0.0e+00 1.3e+00 3.6e-03  1.0e+00 1.0e-06 0.0e+00 1.0e-06 1.0e+00 ↗
[ Info:      2        6  1.8e+00  1.1e-05 1.3e+00 6.2e-03  1.0e+00 3.0e-06 1.0e-06 3.0e-06 7.4e-01 ↗
[ Info:      3        6  1.8e+00  4.3e-05 1.3e+00 1.1e-02  1.0e+00 9.0e-06 4.0e-06 9.0e-06 7.5e-01 ↗
[ Info:      4        6  1.8e+00  1.4e-04 1.3e+00 1.9e-02  1.0e+00 2.7e-05 1.3e-05 2.7e-05 7.5e-01 ↗
[ Info:      5        6  1.8e+00  4.2e-04 1.3e+00 3.2e-02  1.0e+00 8.1e-05 4.0e-05 8.1e-05 7.5e-01 ↗
[ Info:      6        6  1.8e+00  1.3e-03 1.3e+00 5.6e-02  1.0e+00 2.4e-04 1.2e-04 2.4e-04 7.5e-01 ↗
[ Info:      7        6  1.8e+00  3.9e-03 1.3e+00 9.7e-02  1.0e+00 7.3e-04 3.6e-04 7.3e-04 7.5e-01 ↗
[ Info:      8        6  1.8e+00  1.2e-02 1.3e+00 1.7e-01  1.0e+00 2.2e-03 1.1e-03 2.2e-03 7.5e-01 ↗
[ Info:      9        6  1.7e+00  3.4e-02 1.3e+00 2.8e-01  1.0e+00 6.6e-03 3.3e-03 6.6e-03 7.5e-01 ↗
[ Info:     10        6  1.6e+00  1.0e-01 1.2e+00 4.4e-01  1.0e+00 2.0e-02 9.8e-03 2.0e-02 7.6e-01 ↗
[ Info:     11        6  1.2e+00  2.7e-01 1.0e+00 5.7e-01  9.9e-01 5.9e-02 3.0e-02 5.9e-02 7.7e-01 ↗
[ Info:     12       15  6.0e-01  5.6e-01 5.9e-01 4.5e-01  1.2e+00 1.8e-01 8.9e-02 1.8e-01 8.0e-01 ↗
[ Info:     13        6  2.2e-01  7.0e-01 3.3e-01 2.9e-01  1.8e+00 5.3e-01 2.7e-01 2.0e-01 6.3e-01 ↗
[ Info:     14        6  2.0e-01  5.9e-01 2.7e-01 5.2e-01  1.0e+00 6.0e-01 4.7e-01 6.0e-01 1.1e-01 ↗
[ Info:     15        6  3.7e-02  4.8e-01 2.8e-01 4.5e-01 -1.1e+00 1.8e+00 1.1e+00 8.1e-01 1.1e-01 ↘
[ Info:     16       11  3.7e-02  4.8e-01 3.0e-01 4.5e-01 -1.0e+00 9.0e-01 1.1e+00 8.0e-01 1.1e-01 ↘
[ Info:     17        6  3.7e-02  4.8e-01 3.1e-01 4.3e-01 -3.8e-01 4.5e-01 1.1e+00 4.5e-01 1.1e-01 ↘
[ Info:     18        5  3.7e-02  4.8e-01 3.1e-01 3.9e-01  2.8e-01 2.3e-01 1.1e+00 2.3e-01 1.1e-01 =
[ Info:     19       13  1.4e-01  3.4e-01 1.7e-01 1.7e-01  1.2e+00 2.3e-01 8.9e-01 1.4e-01 5.1e-01 ↗
[ Info:     20        5  6.2e-02  3.8e-01 6.3e-02 6.0e-02  1.2e+00 4.2e-01 8.7e-01 4.1e-02 4.0e-01 ↗
[ Info:     21        3  4.8e-02  3.9e-01 1.8e-02 2.0e-02  9.4e-01 4.2e-01 8.8e-01 1.4e-02 2.8e-01 ↗
[ Info:     22        0  4.3e-02  3.9e-01 6.4e-03 3.3e-03  1.0e+00 4.2e-01 8.9e-01 8.6e-04 3.3e-01 ↗
[ Info:     23        0  4.3e-02  3.9e-01 5.8e-03 3.0e-03  1.0e+00 4.2e-01 8.9e-01 7.9e-04 3.2e-01 ↗
[ Info:     24        0  4.3e-02  3.9e-01 5.3e-03 2.8e-03  1.0e+00 4.2e-01 8.9e-01 7.2e-04 3.2e-01 ↗
[ Info:     25        0  4.2e-02  3.9e-01 4.8e-03 2.5e-03  1.0e+00 4.2e-01 8.9e-01 6.6e-04 3.2e-01 ↗
[ Info:     26        0  4.2e-02  3.9e-01 4.3e-03 2.3e-03  1.0e+00 4.2e-01 8.9e-01 6.0e-04 3.1e-01 ↗
[ Info:     27        0  4.2e-02  3.9e-01 4.0e-03 2.1e-03  1.0e+00 4.2e-01 8.9e-01 5.5e-04 3.1e-01 ↗
[ Info:     28        0  4.2e-02  3.9e-01 3.6e-03 1.9e-03  1.0e+00 4.2e-01 8.9e-01 5.0e-04 3.1e-01 ↗
[ Info:     29        0  4.2e-02  3.9e-01 3.3e-03 1.7e-03  1.0e+00 4.2e-01 8.9e-01 4.5e-04 3.1e-01 ↗
[ Info:     30        0  4.2e-02  3.9e-01 3.0e-03 1.6e-03  1.0e+00 4.2e-01 8.9e-01 4.1e-04 3.1e-01 ↗
[ Info:     31        0  4.2e-02  3.9e-01 2.7e-03 1.4e-03  1.0e+00 4.2e-01 8.9e-01 3.7e-04 3.0e-01 ↗
[ Info:     32        0  4.1e-02  3.9e-01 2.5e-03 1.3e-03  1.0e+00 4.2e-01 8.9e-01 3.4e-04 3.0e-01 ↗
[ Info:     33        1  4.1e-02  3.9e-01 2.3e-03 1.2e-03          4.2e-01 8.9e-01 3.1e-04 3.0e-01
[ Info: TR: terminating with √(ξ1/ν) = 0.0022649318505486168

For TR with TRDH

On this branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "TR-JSO")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, ϵa = 1e-3, ϵr = 1e-3, verbose = 1)

TR(bpdn, h, χ, options, subsolver = TRDHSolver)
using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

#import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "TR-JSO")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, ϵa = 1e-3, ϵr = 1e-3, verbose = 1)

TR(bpdn, h, χ, options, subsolver = TRDHSolver)
[ Info:  outer  inner     f(x)     h(x)  √(ξ1/ν)        ρ        Δ      ‖x‖      ‖s‖      ‖B‖ TR 
[ Info:      0      0  1.8e+00  0.0e+00  3.6e-03  1.0e+00  1.0e-06  0.0e+00  1.0e-06  1.0e+00               ↗
[ Info:      1      0  1.8e+00  1.1e-05  5.4e-03  1.0e+00  3.0e-06  1.0e-06  3.0e-06  7.4e-01               ↗
[ Info:      2      0  1.8e+00  4.4e-05  9.3e-03  1.0e+00  9.0e-06  4.0e-06  9.0e-06  7.4e-01               ↗
[ Info:      3      0  1.8e+00  1.4e-04  1.6e-02  1.0e+00  2.7e-05  1.3e-05  2.7e-05  7.4e-01               ↗
[ Info:      4      0  1.8e+00  4.4e-04  2.8e-02  1.0e+00  8.1e-05  4.0e-05  8.1e-05  7.4e-01               ↗
[ Info:      5      0  1.8e+00  1.3e-03  4.8e-02  1.0e+00  2.4e-04  1.2e-04  2.4e-04  7.4e-01               ↗
[ Info:      6      0  1.8e+00  4.0e-03  8.4e-02  1.0e+00  7.3e-04  3.6e-04  7.3e-04  7.4e-01               ↗
[ Info:      7      0  1.8e+00  1.2e-02  1.4e-01  1.0e+00  2.2e-03  1.1e-03  2.2e-03  7.5e-01               ↗
[ Info:      8      0  1.7e+00  3.6e-02  2.5e-01  1.0e+00  6.6e-03  3.3e-03  6.6e-03  7.5e-01               ↗
[ Info:      9      0  1.6e+00  1.0e-01  4.1e-01  1.0e+00  2.0e-02  9.8e-03  2.0e-02  7.5e-01               ↗
[ Info:     10      0  1.2e+00  2.7e-01  5.9e-01  9.9e-01  5.9e-02  3.0e-02  5.9e-02  7.7e-01               ↗
[ Info:     11      0  5.9e-01  5.7e-01  5.3e-01  1.2e+00  1.8e-01  8.9e-02  1.8e-01  8.0e-01               ↗
[ Info:     12      0  2.2e-01  7.1e-01  3.2e-01  1.8e+00  5.3e-01  2.7e-01  2.0e-01  6.3e-01               ↗
[ Info:     13      0  2.0e-01  5.8e-01  2.2e-01  9.5e-01  6.1e-01  4.7e-01  6.1e-01  1.0e-01               ↗
[ Info:     14      0  3.6e-02  4.9e-01  2.1e-01 -1.2e+00  1.8e+00  1.1e+00  8.7e-01  1.1e-01               ↘
[ Info:     15      0  3.6e-02  4.9e-01  2.1e-01 -1.2e+00  9.2e-01  1.1e+00  8.7e-01  1.1e-01               ↘
[ Info:     16      0  3.6e-02  4.9e-01  1.8e-01 -3.0e-01  4.6e-01  1.1e+00  4.6e-01  1.1e-01               ↘
[ Info:     17      0  3.6e-02  4.9e-01  1.5e-01  3.1e-01  2.3e-01  1.1e+00  2.3e-01  1.1e-01               =
[ Info:     18      0  1.3e-01  3.4e-01  1.6e-01  1.2e+00  2.3e-01  8.9e-01  1.4e-01  5.0e-01               ↗
[ Info:     19      0  5.9e-02  3.8e-01  4.5e-02  1.2e+00  4.2e-01  8.7e-01  4.9e-02  3.9e-01               ↗
[ Info:     20      0  4.4e-02  3.9e-01  8.9e-03  9.2e-01  4.2e-01  8.9e-01  1.6e-02  3.0e-01               ↗
[ Info:     21      0  4.0e-02  3.9e-01  2.1e-03  7.2e-01  4.2e-01  8.9e-01  4.0e-03  3.2e-01               =
[ Info:     22      0  4.0e-02  3.9e-01  1.0e-03  6.9e-01  4.2e-01  8.9e-01  1.1e-03  4.1e-01               =
[ Info:     23      0  4.1e-02  3.9e-01  3.8e-04  6.9e-01  4.2e-01  8.9e-01  4.1e-04  5.4e-01
[ Info:     17      0  3.6e-02  4.9e-01  1.5e-01  3.1e-01  2.3e-01  1.1e+00  2.3e-01  1.1e-01               =
[ Info:     18      0  1.3e-01  3.4e-01  1.6e-01  1.2e+00  2.3e-01  8.9e-01  1.4e-01  5.0e-01               ↗
[ Info:     19      0  5.9e-02  3.8e-01  4.5e-02  1.2e+00  4.2e-01  8.7e-01  4.9e-02  3.9e-01               ↗
[ Info:     20      0  4.4e-02  3.9e-01  8.9e-03  9.2e-01  4.2e-01  8.9e-01  1.6e-02  3.0e-01               ↗
[ Info:     17      0  3.6e-02  4.9e-01  1.5e-01  3.1e-01  2.3e-01  1.1e+00  2.3e-01  1.1e-01               =
[ Info:     18      0  1.3e-01  3.4e-01  1.6e-01  1.2e+00  2.3e-01  8.9e-01  1.4e-01  5.0e-01               ↗
[ Info:     17      0  3.6e-02  4.9e-01  1.5e-01  3.1e-01  2.3e-01  1.1e+00  2.3e-01  1.1e-01               =
[ Info:     17      0  3.6e-02  4.9e-01  1.5e-01  3.1e-01  2.3e-01  1.1e+00  2.3e-01  1.1e-01               =
[ Info:     18      0  1.3e-01  3.4e-01  1.6e-01  1.2e+00  2.3e-01  8.9e-01  1.4e-01  5.0e-01               ↗
[ Info:     19      0  5.9e-02  3.8e-01  4.5e-02  1.2e+00  4.2e-01  8.7e-01  4.9e-02  3.9e-01               ↗
[ Info:     20      0  4.4e-02  3.9e-01  8.9e-03  9.2e-01  4.2e-01  8.9e-01  1.6e-02  3.0e-01               ↗
[ Info:     21      0  4.0e-02  3.9e-01  2.1e-03  7.2e-01  4.2e-01  8.9e-01  4.0e-03  3.2e-01               =
[ Info:     22      0  4.0e-02  3.9e-01  1.0e-03  6.9e-01  4.2e-01  8.9e-01  1.1e-03  4.1e-01               =
[ Info:     23      0  4.1e-02  3.9e-01  3.8e-04  6.9e-01  4.2e-01  8.9e-01  4.1e-04  5.4e-01
[ Info: TR: terminating with √(ξ1/ν) = 0.00038148430978911664

On the master branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = SpectralGradientModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl.git")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, ϵa = 1e-3, ϵr = 1e-3, verbose = 10)

TR(bpdn, h, χ, options, subsolver = TRDH)
[ Info:  outer    inner     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       Δ     ‖x‖     ‖s‖    ‖Bₖ‖ TR
[ Info:      1        1  1.8e+00  0.0e+00 3.6e-03 3.6e-03  1.0e+00 1.0e-06 0.0e+00 1.0e-06 1.0e+00 ↗
[ Info:      2        1  1.8e+00  1.1e-05 5.4e-03 6.2e-03  1.0e+00 3.0e-06 1.0e-06 3.0e-06 7.4e-01 ↗
[ Info:      3        1  1.8e+00  4.4e-05 9.3e-03 1.1e-02  1.0e+00 9.0e-06 4.0e-06 9.0e-06 7.4e-01 ↗
[ Info:      4        1  1.8e+00  1.4e-04 1.6e-02 1.9e-02  1.0e+00 2.7e-05 1.3e-05 2.7e-05 7.4e-01 ↗
[ Info:      5        1  1.8e+00  4.4e-04 2.8e-02 3.2e-02  1.0e+00 8.1e-05 4.0e-05 8.1e-05 7.4e-01 ↗
[ Info:      6        1  1.8e+00  1.3e-03 4.8e-02 5.6e-02  1.0e+00 2.4e-04 1.2e-04 2.4e-04 7.4e-01 ↗
[ Info:      7        1  1.8e+00  4.0e-03 8.4e-02 9.7e-02  1.0e+00 7.3e-04 3.6e-04 7.3e-04 7.4e-01 ↗
[ Info:      8        1  1.8e+00  1.2e-02 1.4e-01 1.7e-01  1.0e+00 2.2e-03 1.1e-03 2.2e-03 7.5e-01 ↗
[ Info:      9        1  1.7e+00  3.6e-02 2.5e-01 2.8e-01  1.0e+00 6.6e-03 3.3e-03 6.6e-03 7.5e-01 ↗
[ Info:     10        1  1.6e+00  1.0e-01 4.1e-01 4.4e-01  1.0e+00 2.0e-02 9.8e-03 2.0e-02 7.5e-01 ↗
[ Info:     11        1  1.2e+00  2.7e-01 5.9e-01 5.7e-01  9.9e-01 5.9e-02 3.0e-02 5.9e-02 7.7e-01 ↗
[ Info:     12        1  5.9e-01  5.7e-01 5.3e-01 4.5e-01  1.2e+00 1.8e-01 8.9e-02 1.8e-01 8.0e-01 ↗
[ Info:     13        1  2.2e-01  7.1e-01 3.2e-01 2.9e-01  1.8e+00 5.3e-01 2.7e-01 2.0e-01 6.3e-01 ↗
[ Info:     14        1  2.0e-01  5.8e-01 2.2e-01 5.2e-01  9.5e-01 6.1e-01 4.7e-01 6.1e-01 1.0e-01 ↗
[ Info:     15        1  3.6e-02  4.9e-01 2.1e-01 4.8e-01 -1.2e+00 1.8e+00 1.1e+00 8.7e-01 1.1e-01 ↘
[ Info:     16        1  3.6e-02  4.9e-01 2.1e-01 4.8e-01 -1.2e+00 9.2e-01 1.1e+00 8.7e-01 1.1e-01 ↘
[ Info:     17        1  3.6e-02  4.9e-01 1.8e-01 4.6e-01 -3.0e-01 4.6e-01 1.1e+00 4.6e-01 1.1e-01 ↘
[ Info:     18        1  3.6e-02  4.9e-01 1.5e-01 4.1e-01  3.1e-01 2.3e-01 1.1e+00 2.3e-01 1.1e-01 =
[ Info:     19        1  1.3e-01  3.4e-01 1.6e-01 1.6e-01  1.2e+00 2.3e-01 8.9e-01 1.4e-01 5.0e-01 ↗
[ Info:     20        1  5.9e-02  3.8e-01 4.5e-02 5.4e-02  1.2e+00 4.2e-01 8.7e-01 4.9e-02 3.9e-01 ↗
[ Info:     21        1  4.4e-02  3.9e-01 8.9e-03 1.2e-02  9.2e-01 4.2e-01 8.9e-01 1.6e-02 3.0e-01 ↗
[ Info:     22        1  4.0e-02  3.9e-01 2.1e-03 2.6e-03  7.2e-01 4.2e-01 8.9e-01 4.0e-03 3.2e-01 =
[ Info:     23        1  4.0e-02  3.9e-01 1.0e-03 1.1e-03  6.9e-01 4.2e-01 8.9e-01 1.1e-03 4.1e-01 =
[ Info:     24        1  4.1e-02  3.9e-01 3.8e-04 5.2e-04          4.2e-01 8.9e-01 4.1e-04 5.4e-01
[ Info: TR: terminating with √(ξ1/ν) = 0.00038148430978911235

MaxenceGollier · 2025-08-08T14:18:26Z

-    # model for prox-gradient step to update Δk if ||s|| is too small and ξ1
-    φ1(d) = ∇fk' * d
-    mk1(d) = φ1(d) + ψ(d)
+  ν = (α * Δk)/(DNorm + one(T))


Suggested change

ν = (α * Δk)/(DNorm + one(T))

ν = 1/(DNorm + 1/(α * Δk))

Cf paper, this seems to be an error.

This can also be fixed in LMTR.

Co-authored-by: Dominique <dominique.orban@gmail.com>

dpo · 2025-09-07T16:20:38Z

-    u_bound = kwargs[:u_bound]
-    has_bnds = has_bnds || any(u_bound .!= R(Inf))
-  end
+  nlp = FirstOrderModel(f, ∇f!, x0)


FirstOrderModel no longer exists: https://github.com/JuliaSmoothOptimizers/RegularizedProblems.jl/blob/main/src/types.jl#L4

We show now use ManualNLPModels.jl.

MaxenceGollier · 2025-09-08T17:23:03Z

@MohamedLaghdafHABIBOULLAH

TR with LBFGS and R2
On this branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = LBFGSModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "TR-JSO")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, ϵa = 1e-3, ϵr = 1e-3, verbose = 1)

TR(bpdn, h, χ, options, subsolver = R2Solver)
[ Info:  outer  inner     f(x)     h(x)  √(ξ1/ν)        ρ        Δ      ‖x‖      ‖s‖      ‖B‖ TR 
[ Info:      0      0  1.8e+00  0.0e+00  3.6e-03  1.0e+00  1.0e-06  0.0e+00  1.0e-06  1.0e+00 ↗
[ Info:      1      0  1.8e+00  1.1e-05  7.7e-03  1.0e+00  3.0e-06  1.0e-06  3.0e-06  1.5e+00 ↗
[ Info:      2      0  1.8e+00  4.4e-05  1.3e-02  1.0e+00  9.0e-06  4.0e-06  9.0e-06  1.5e+00 ↗
[ Info:      3      0  1.8e+00  1.4e-04  2.3e-02  1.0e+00  2.7e-05  1.3e-05  2.7e-05  1.5e+00 ↗
[ Info:      4      0  1.8e+00  4.4e-04  4.0e-02  1.0e+00  8.1e-05  4.0e-05  8.1e-05  1.5e+00 ↗
[ Info:      5      0  1.8e+00  1.3e-03  6.9e-02  1.0e+00  2.4e-04  1.2e-04  2.4e-04  1.5e+00 ↗
[ Info:      6      0  1.8e+00  4.0e-03  1.2e-01  1.0e+00  7.3e-04  3.6e-04  7.3e-04  1.5e+00 ↗
[ Info:      7      0  1.8e+00  1.2e-02  2.1e-01  1.0e+00  2.2e-03  1.1e-03  2.2e-03  1.5e+00 ↗
[ Info:      8      1  1.7e+00  3.5e-02  3.5e-01  1.0e+00  6.6e-03  3.3e-03  6.6e-03  1.5e+00 ↗
[ Info:      9      3  1.6e+00  1.0e-01  5.7e-01  1.0e+00  2.0e-02  9.8e-03  2.0e-02  1.5e+00 ↗
[ Info:     10      6  1.2e+00  2.8e-01  7.5e-01  1.1e+00  5.9e-02  3.0e-02  5.9e-02  1.6e+00 ↗
[ Info:     11      4  5.6e-01  6.1e-01  5.7e-01  1.4e+00  1.8e-01  8.9e-02  1.8e-01  1.6e+00 ↗
[ Info:     12     10  2.9e-01  6.6e-01  3.4e-01  1.5e+00  5.3e-01  2.7e-01  2.7e-01  1.7e+00 ↗
[ Info:     13     10  1.7e-01  5.9e-01  2.7e-01  1.5e+00  8.0e-01  5.3e-01  2.5e-01  2.3e+00 ↗
[ Info:     14      8  1.2e-01  5.0e-01  2.6e-01  1.5e+00  8.0e-01  7.9e-01  1.8e-01  2.8e+00 ↗
[ Info:     15      8  7.9e-02  4.3e-01  2.1e-01  1.4e+00  8.0e-01  9.3e-01  1.5e-01  3.0e+00 ↗
[ Info:     16      5  4.9e-02  4.0e-01  1.2e-01  1.4e+00  8.0e-01  9.7e-01  8.0e-02  3.1e+00 ↗
[ Info:     17      2  4.4e-02  3.9e-01  5.7e-02  1.2e+00  8.0e-01  9.4e-01  2.8e-02  2.7e+00 ↗
[ Info:     18      2  4.3e-02  3.9e-01  2.1e-02  1.2e+00  8.0e-01  9.2e-01  1.7e-02  2.7e+00 ↗
[ Info:     19      1  4.2e-02  3.9e-01  1.2e-02  1.0e+00  8.0e-01  9.0e-01  5.1e-03  2.9e+00 ↗
[ Info:     20      0  4.2e-02  3.9e-01  1.0e-02  1.0e+00  8.0e-01  9.0e-01  2.1e-03  3.1e+00 ↗
[ Info:     21      0  4.2e-02  3.9e-01  9.2e-03  1.0e+00  8.0e-01  9.0e-01  2.0e-03  3.0e+00 ↗
[ Info:     22      0  4.2e-02  3.9e-01  8.4e-03  1.0e+00  8.0e-01  9.0e-01  2.0e-03  2.7e+00 ↗
[ Info:     23      0  4.2e-02  3.9e-01  7.7e-03  1.0e+00  8.0e-01  9.0e-01  2.2e-03  2.2e+00 ↗
[ Info:     24      0  4.2e-02  3.9e-01  6.8e-03  1.0e+00  8.0e-01  9.0e-01  2.2e-03  1.9e+00 ↗
[ Info:     25      0  4.1e-02  3.9e-01  6.0e-03  1.0e+00  8.0e-01  9.0e-01  1.9e-03  1.9e+00 ↗
[ Info:     26      0  4.1e-02  3.9e-01  5.2e-03  1.0e+00  8.0e-01  8.9e-01  1.7e-03  1.9e+00 ↗
[ Info:     27      0  4.1e-02  3.9e-01  4.6e-03  1.0e+00  8.0e-01  8.9e-01  1.5e-03  1.9e+00 ↗
[ Info:     28      0  4.1e-02  3.9e-01  4.0e-03  1.0e+00  8.0e-01  8.9e-01  1.3e-03  1.9e+00 ↗
[ Info:     29      0  4.1e-02  3.9e-01  3.5e-03  1.0e+00  8.0e-01  8.9e-01  1.1e-03  1.9e+00 ↗
[ Info:     30      0  4.1e-02  3.9e-01  3.1e-03  1.0e+00  8.0e-01  8.9e-01  9.7e-04  1.9e+00 ↗
[ Info:     31      0  4.1e-02  3.9e-01  2.7e-03  1.0e+00  8.0e-01  8.9e-01  8.5e-04  1.9e+00 ↗
[ Info:     32      0  4.1e-02  3.9e-01  2.3e-03  1.0e+00  8.0e-01  8.9e-01  7.4e-04  1.9e+00 ↗
[ Info:     33      0  4.1e-02  3.9e-01  2.1e-03  1.0e+00  8.0e-01  8.9e-01  6.5e-04  1.9e+00 ↗
[ Info:     34      0  4.1e-02  3.9e-01  1.8e-03  1.0e+00  8.0e-01  8.9e-01  5.7e-04  1.9e+00 ↗
[ Info:     35      0  4.1e-02  3.9e-01  1.6e-03  1.0e+00  8.0e-01  8.9e-01  4.9e-04  1.9e+00 ↗
[ Info:     36      0  4.1e-02  3.9e-01  1.4e-03  1.0e+00  8.0e-01  8.9e-01  4.3e-04  1.9e+00 ↗
[ Info:     37      0  4.1e-02  3.9e-01  1.2e-03  1.0e+00  8.0e-01  8.9e-01  3.8e-04  1.9e+00 ↗
[ Info:     38      0  4.1e-02  3.9e-01  1.1e-03  1.0e+00  8.0e-01  8.9e-01  3.3e-04  1.9e+00 ↗
[ Info:     39      0  4.1e-02  3.9e-01  9.2e-04  1.0e+00  8.0e-01  8.9e-01  2.9e-04  1.9e+00
[ Info: TR: terminating with √(ξ1/ν) = 0.0009188650449628023
"Execution stats: first-order stationary"

On the master branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = LBFGSModel(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl.git")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, ϵa = 1e-3, ϵr = 1e-3, verbose = 10)

TR(bpdn, h, χ, options, subsolver = R2)
[ Info:  outer    inner     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       Δ     ‖x‖     ‖s‖    ‖Bₖ‖ TR
[ Info:      1        0  1.8e+00  0.0e+00 3.6e-03 3.6e-03  1.0e+00 1.0e-06 0.0e+00 1.0e-06 1.0e+00 ↗
[ Info:      2        0  1.8e+00  1.1e-05 7.7e-03 6.2e-03  1.0e+00 3.0e-06 1.0e-06 3.0e-06 1.5e+00 ↗
[ Info:      3        0  1.8e+00  4.4e-05 1.3e-02 1.1e-02  1.0e+00 9.0e-06 4.0e-06 9.0e-06 1.5e+00 ↗
[ Info:      4        0  1.8e+00  1.4e-04 2.3e-02 1.9e-02  1.0e+00 2.7e-05 1.3e-05 2.7e-05 1.5e+00 ↗
[ Info:      5        0  1.8e+00  4.4e-04 4.0e-02 3.2e-02  1.0e+00 8.1e-05 4.0e-05 8.1e-05 1.5e+00 ↗
[ Info:      6        0  1.8e+00  1.3e-03 6.9e-02 5.6e-02  1.0e+00 2.4e-04 1.2e-04 2.4e-04 1.5e+00 ↗
[ Info:      7        0  1.8e+00  4.0e-03 1.2e-01 9.7e-02  1.0e+00 7.3e-04 3.6e-04 7.3e-04 1.5e+00 ↗
[ Info:      8        0  1.8e+00  1.2e-02 2.1e-01 1.7e-01  1.0e+00 2.2e-03 1.1e-03 2.2e-03 1.5e+00 ↗
[ Info:      9        1  1.7e+00  3.5e-02 3.5e-01 2.8e-01  1.0e+00 6.6e-03 3.3e-03 6.6e-03 1.5e+00 ↗
[ Info:     10        3  1.6e+00  1.0e-01 5.7e-01 4.4e-01  1.0e+00 2.0e-02 9.8e-03 2.0e-02 1.5e+00 ↗
[ Info:     11        6  1.2e+00  2.8e-01 7.5e-01 5.4e-01  1.1e+00 5.9e-02 3.0e-02 5.9e-02 1.6e+00 ↗
[ Info:     12        4  5.6e-01  6.1e-01 5.7e-01 3.9e-01  1.4e+00 1.8e-01 8.9e-02 1.8e-01 1.6e+00 ↗
[ Info:     13       10  2.9e-01  6.6e-01 3.4e-01 3.6e-01  1.5e+00 5.3e-01 2.7e-01 2.7e-01 1.7e+00 ↗
[ Info:     14       10  1.7e-01  5.9e-01 2.7e-01 3.1e-01  1.5e+00 8.0e-01 5.3e-01 2.5e-01 2.3e+00 ↗
[ Info:     15        8  1.2e-01  5.0e-01 2.6e-01 2.7e-01  1.5e+00 8.0e-01 7.9e-01 1.8e-01 2.8e+00 ↗
[ Info:     16        8  7.9e-02  4.3e-01 2.1e-01 2.1e-01  1.4e+00 8.0e-01 9.3e-01 1.5e-01 3.0e+00 ↗
[ Info:     17        5  4.9e-02  4.0e-01 1.2e-01 9.5e-02  1.4e+00 8.0e-01 9.7e-01 8.0e-02 3.1e+00 ↗
[ Info:     18        2  4.4e-02  3.9e-01 5.7e-02 4.7e-02  1.2e+00 8.0e-01 9.4e-01 2.8e-02 2.7e+00 ↗
[ Info:     19        2  4.3e-02  3.9e-01 2.1e-02 1.9e-02  1.2e+00 8.0e-01 9.2e-01 1.7e-02 2.7e+00 ↗
[ Info:     20        1  4.2e-02  3.9e-01 1.2e-02 9.3e-03  1.0e+00 8.0e-01 9.0e-01 5.1e-03 2.9e+00 ↗
[ Info:     21        0  4.2e-02  3.9e-01 1.0e-02 5.6e-03  1.0e+00 8.0e-01 9.0e-01 2.1e-03 3.1e+00 ↗
[ Info:     22        0  4.2e-02  3.9e-01 9.2e-03 5.2e-03  1.0e+00 8.0e-01 9.0e-01 2.0e-03 3.0e+00 ↗
[ Info:     23        0  4.2e-02  3.9e-01 8.4e-03 5.0e-03  1.0e+00 8.0e-01 9.0e-01 2.0e-03 2.7e+00 ↗
[ Info:     24        0  4.2e-02  3.9e-01 7.7e-03 5.0e-03  1.0e+00 8.0e-01 9.0e-01 2.2e-03 2.2e+00 ↗
[ Info:     25        0  4.2e-02  3.9e-01 6.8e-03 4.7e-03  1.0e+00 8.0e-01 9.0e-01 2.2e-03 1.9e+00 ↗
[ Info:     26        0  4.1e-02  3.9e-01 6.0e-03 4.2e-03  1.0e+00 8.0e-01 9.0e-01 1.9e-03 1.9e+00 ↗
[ Info:     27        0  4.1e-02  3.9e-01 5.2e-03 3.7e-03  1.0e+00 8.0e-01 8.9e-01 1.7e-03 1.9e+00 ↗
[ Info:     28        0  4.1e-02  3.9e-01 4.6e-03 3.2e-03  1.0e+00 8.0e-01 8.9e-01 1.5e-03 1.9e+00 ↗
[ Info:     29        0  4.1e-02  3.9e-01 4.0e-03 2.8e-03  1.0e+00 8.0e-01 8.9e-01 1.3e-03 1.9e+00 ↗
[ Info:     30        0  4.1e-02  3.9e-01 3.5e-03 2.4e-03  1.0e+00 8.0e-01 8.9e-01 1.1e-03 1.9e+00 ↗
[ Info:     31        0  4.1e-02  3.9e-01 3.1e-03 2.1e-03  1.0e+00 8.0e-01 8.9e-01 9.7e-04 1.9e+00 ↗
[ Info:     32        0  4.1e-02  3.9e-01 2.7e-03 1.9e-03  1.0e+00 8.0e-01 8.9e-01 8.5e-04 1.9e+00 ↗
[ Info:     33        0  4.1e-02  3.9e-01 2.3e-03 1.6e-03  1.0e+00 8.0e-01 8.9e-01 7.4e-04 1.9e+00 ↗
[ Info:     34        0  4.1e-02  3.9e-01 2.1e-03 1.4e-03  1.0e+00 8.0e-01 8.9e-01 6.5e-04 1.9e+00 ↗
[ Info:     35        0  4.1e-02  3.9e-01 1.8e-03 1.3e-03  1.0e+00 8.0e-01 8.9e-01 5.7e-04 1.9e+00 ↗
[ Info:     36        0  4.1e-02  3.9e-01 1.6e-03 1.1e-03  1.0e+00 8.0e-01 8.9e-01 4.9e-04 1.9e+00 ↗
[ Info:     37        0  4.1e-02  3.9e-01 1.4e-03 9.7e-04  1.0e+00 8.0e-01 8.9e-01 4.3e-04 1.9e+00 ↗
[ Info:     38        0  4.1e-02  3.9e-01 1.2e-03 8.5e-04  1.0e+00 8.0e-01 8.9e-01 3.8e-04 1.9e+00 ↗
[ Info:     39        0  4.1e-02  3.9e-01 1.1e-03 7.4e-04  1.0e+00 8.0e-01 8.9e-01 3.3e-04 1.9e+00 ↗
[ Info:     40        1  4.1e-02  3.9e-01 9.2e-04 6.7e-04          8.0e-01 8.9e-01 2.9e-04 1.9e+00
[ Info: TR: terminating with √(ξ1/ν) = 0.0009188650450764466
"Execution stats: first-order stationary"

TR with LSR1 and TRDH
On this branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = LSR1Model(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/MaxenceGollier/RegularizedOptimization.jl.git", rev = "TR-JSO")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, ϵa = 1e-3, ϵr = 1e-3, verbose = 1)

TR(bpdn, h, χ, options, subsolver = TRDHSolver)
julia> TR(bpdn, h, χ, options, subsolver = TRDHSolver)
[ Info:  outer  inner     f(x)     h(x)  √(ξ1/ν)        ρ        Δ      ‖x‖      ‖s‖      ‖B‖ TR 
[ Info:      0      0  1.8e+00  0.0e+00  3.6e-03  1.0e+00  1.0e-06  0.0e+00  1.0e-06  1.0e+00 ↗
[ Info:      1      0  1.8e+00  1.1e-05  6.2e-03  1.0e+00  3.0e-06  1.0e-06  3.0e-06  1.0e+00 ↗
[ Info:      2      0  1.8e+00  4.4e-05  1.1e-02  1.0e+00  9.0e-06  4.0e-06  9.0e-06  1.0e+00 ↗
[ Info:      3      0  1.8e+00  1.4e-04  1.9e-02  1.0e+00  2.7e-05  1.3e-05  2.7e-05  1.0e+00 ↗
[ Info:      4      0  1.8e+00  4.4e-04  3.2e-02  1.0e+00  8.1e-05  4.0e-05  8.1e-05  1.0e+00 ↗
[ Info:      5      0  1.8e+00  1.3e-03  5.6e-02  1.0e+00  2.4e-04  1.2e-04  2.4e-04  1.0e+00 ↗
[ Info:      6      0  1.8e+00  4.0e-03  9.7e-02  1.0e+00  7.3e-04  3.6e-04  7.3e-04  1.0e+00 ↗
[ Info:      7      0  1.8e+00  1.2e-02  1.7e-01  1.0e+00  2.2e-03  1.1e-03  2.2e-03  1.0e+00 ↗
[ Info:      8      0  1.7e+00  3.5e-02  2.8e-01  1.0e+00  6.6e-03  3.3e-03  6.6e-03  1.0e+00 ↗
[ Info:      9      0  1.6e+00  1.0e-01  4.7e-01  1.0e+00  2.0e-02  9.8e-03  2.0e-02  1.0e+00 ↗
[ Info:     10      0  1.2e+00  2.6e-01  6.5e-01  1.0e+00  5.9e-02  3.0e-02  5.9e-02  1.0e+00 ↗
[ Info:     11      0  6.4e-01  5.3e-01  5.8e-01  1.1e+00  1.8e-01  8.9e-02  1.8e-01  1.0e+00 ↗
[ Info:     12      0  2.6e-01  6.7e-01  3.3e-01  1.0e+00  5.3e-01  2.7e-01  1.4e-01  1.0e+00 ↗
[ Info:     13      0  2.1e-01  6.1e-01  2.8e-01  1.0e+00  5.3e-01  4.0e-01  9.6e-02  1.0e+00 ↗
[ Info:     14      0  1.9e-01  5.5e-01  2.5e-01  1.0e+00  5.3e-01  5.0e-01  7.8e-02  1.0e+00 ↗
[ Info:     15      0  1.8e-01  5.1e-01  2.3e-01  1.0e+00  5.3e-01  5.8e-01  6.5e-02  1.0e+00 ↗
[ Info:     16      0  1.6e-01  4.7e-01  2.2e-01  1.0e+00  5.3e-01  6.4e-01  6.1e-02  1.0e+00 ↗
[ Info:     17      0  1.5e-01  4.3e-01  2.0e-01  1.0e+00  5.3e-01  6.9e-01  5.6e-02  1.0e+00 ↗
[ Info:     18      0  1.4e-01  4.1e-01  1.8e-01  1.0e+00  5.3e-01  7.3e-01  5.4e-02  1.0e+00 ↗
[ Info:     19      0  1.2e-01  3.9e-01  1.6e-01  1.0e+00  5.3e-01  7.7e-01  4.9e-02  1.0e+00 ↗
[ Info:     20      0  1.1e-01  3.8e-01  1.4e-01  1.0e+00  5.3e-01  8.0e-01  4.3e-02  1.0e+00 ↗
[ Info:     21      0  9.8e-02  3.7e-01  1.2e-01  1.0e+00  5.3e-01  8.2e-01  3.8e-02  1.0e+00 ↗
[ Info:     22      0  8.6e-02  3.7e-01  1.0e-01  1.0e+00  5.3e-01  8.4e-01  3.2e-02  1.0e+00 ↗
[ Info:     23      0  7.4e-02  3.7e-01  8.0e-02  1.0e+00  5.3e-01  8.6e-01  2.6e-02  1.0e+00 ↗
[ Info:     24      0  6.5e-02  3.7e-01  5.8e-02  1.0e+00  5.3e-01  8.7e-01  2.1e-02  1.0e+00 ↗
[ Info:     25      0  5.8e-02  3.8e-01  3.9e-02  1.0e+00  5.3e-01  8.8e-01  1.6e-02  1.0e+00 ↗
[ Info:     26      0  5.2e-02  3.8e-01  2.7e-02  1.0e+00  5.3e-01  8.8e-01  1.1e-02  1.0e+00 ↗
[ Info:     27      0  4.8e-02  3.9e-01  1.9e-02  1.0e+00  5.3e-01  8.9e-01  8.5e-03  1.0e+00 ↗
[ Info:     28      0  4.6e-02  3.9e-01  1.3e-02  1.0e+00  5.3e-01  8.9e-01  6.4e-03  1.0e+00 ↗
[ Info:     29      0  4.4e-02  3.9e-01  8.7e-03  1.0e+00  5.3e-01  8.9e-01  4.6e-03  1.0e+00 ↗
[ Info:     30      0  4.3e-02  3.9e-01  6.1e-03  1.0e+00  5.3e-01  8.9e-01  3.4e-03  1.0e+00 ↗
[ Info:     31      0  4.2e-02  3.9e-01  4.4e-03  1.0e+00  5.3e-01  8.9e-01  2.5e-03  1.0e+00 ↗
[ Info:     32      0  4.2e-02  3.9e-01  3.2e-03  1.0e+00  5.3e-01  8.9e-01  1.8e-03  1.0e+00 ↗
[ Info:     33      0  4.1e-02  3.9e-01  2.3e-03  1.0e+00  5.3e-01  8.9e-01  1.4e-03  1.0e+00 ↗
[ Info:     34      0  4.1e-02  3.9e-01  1.7e-03  1.0e+00  5.3e-01  8.9e-01  1.0e-03  1.0e+00 ↗
[ Info:     35      0  4.1e-02  3.9e-01  1.3e-03  1.0e+00  5.3e-01  8.9e-01  7.7e-04  1.0e+00 ↗
[ Info:     36      0  4.1e-02  3.9e-01  9.6e-04  1.0e+00  5.3e-01  8.9e-01  5.8e-04  1.0e+00
[ Info: TR: terminating with √(ξ1/ν) = 0.0009554889363751277
"Execution stats: first-order stationary"

On the master branch,

using LinearAlgebra, NLPModels, NLPModelsModifiers, RegularizedProblems, ProximalOperators, Random, ShiftedProximalOperators
Random.seed!(0)
bpdn = LSR1Model(bpdn_model(1)[1])
λ = norm(grad(bpdn, zeros(bpdn.meta.nvar)), Inf) / 10
h = NormL1(λ)
χ = NormLinf(1.0)  

import Pkg; Pkg.add(url = "https://github.com/JuliaSmoothOptimizers/RegularizedOptimization.jl.git")
using RegularizedOptimization
options = ROSolverOptions(Δk = 1e-6, ϵa = 1e-3, ϵr = 1e-3, verbose = 10)

TR(bpdn, h, χ, options, subsolver = TRDH)
[ Info:  outer    inner     f(x)     h(x) √(ξ1/ν)      √ξ        ρ       Δ     ‖x‖     ‖s‖    ‖Bₖ‖ TR
[ Info:      1        1  1.8e+00  0.0e+00 3.6e-03 3.6e-03  1.0e+00 1.0e-06 0.0e+00 1.0e-06 1.0e+00 ↗
[ Info:      2        1  1.8e+00  1.1e-05 6.2e-03 6.2e-03  1.0e+00 3.0e-06 1.0e-06 3.0e-06 1.0e+00 ↗
[ Info:      3        1  1.8e+00  4.4e-05 1.1e-02 1.1e-02  1.0e+00 9.0e-06 4.0e-06 9.0e-06 1.0e+00 ↗
[ Info:      4        1  1.8e+00  1.4e-04 1.9e-02 1.9e-02  1.0e+00 2.7e-05 1.3e-05 2.7e-05 1.0e+00 ↗
[ Info:      5        1  1.8e+00  4.4e-04 3.2e-02 3.2e-02  1.0e+00 8.1e-05 4.0e-05 8.1e-05 1.0e+00 ↗
[ Info:      6        1  1.8e+00  1.3e-03 5.6e-02 5.6e-02  1.0e+00 2.4e-04 1.2e-04 2.4e-04 1.0e+00 ↗
[ Info:      7        1  1.8e+00  4.0e-03 9.7e-02 9.7e-02  1.0e+00 7.3e-04 3.6e-04 7.3e-04 1.0e+00 ↗
[ Info:      8        1  1.8e+00  1.2e-02 1.7e-01 1.7e-01  1.0e+00 2.2e-03 1.1e-03 2.2e-03 1.0e+00 ↗
[ Info:      9        1  1.7e+00  3.5e-02 2.8e-01 2.8e-01  1.0e+00 6.6e-03 3.3e-03 6.6e-03 1.0e+00 ↗
[ Info:     10        1  1.6e+00  1.0e-01 4.7e-01 4.4e-01  1.0e+00 2.0e-02 9.8e-03 2.0e-02 1.0e+00 ↗
[ Info:     11        1  1.2e+00  2.6e-01 6.5e-01 5.5e-01  1.0e+00 5.9e-02 3.0e-02 5.9e-02 1.0e+00 ↗
[ Info:     12        1  6.4e-01  5.3e-01 5.8e-01 4.7e-01  1.1e+00 1.8e-01 8.9e-02 1.8e-01 1.0e+00 ↗
[ Info:     13        1  2.6e-01  6.7e-01 3.3e-01 3.2e-01  1.0e+00 5.3e-01 2.7e-01 1.4e-01 1.0e+00 ↗
[ Info:     14        1  2.1e-01  6.1e-01 2.8e-01 2.7e-01  1.0e+00 5.3e-01 4.0e-01 9.6e-02 1.0e+00 ↗
[ Info:     15        1  1.9e-01  5.5e-01 2.5e-01 2.5e-01  1.0e+00 5.3e-01 5.0e-01 7.8e-02 1.0e+00 ↗
[ Info:     16        1  1.8e-01  5.1e-01 2.3e-01 2.3e-01  1.0e+00 5.3e-01 5.8e-01 6.5e-02 1.0e+00 ↗
[ Info:     17        1  1.6e-01  4.7e-01 2.2e-01 2.1e-01  1.0e+00 5.3e-01 6.4e-01 6.1e-02 1.0e+00 ↗
[ Info:     18        1  1.5e-01  4.3e-01 2.0e-01 2.0e-01  1.0e+00 5.3e-01 6.9e-01 5.6e-02 1.0e+00 ↗
[ Info:     19        1  1.4e-01  4.1e-01 1.8e-01 1.8e-01  1.0e+00 5.3e-01 7.3e-01 5.4e-02 1.0e+00 ↗
[ Info:     20        1  1.2e-01  3.9e-01 1.6e-01 1.6e-01  1.0e+00 5.3e-01 7.7e-01 4.9e-02 1.0e+00 ↗
[ Info:     21        1  1.1e-01  3.8e-01 1.4e-01 1.4e-01  1.0e+00 5.3e-01 8.0e-01 4.3e-02 1.0e+00 ↗
[ Info:     22        1  9.8e-02  3.7e-01 1.2e-01 1.2e-01  1.0e+00 5.3e-01 8.2e-01 3.8e-02 1.0e+00 ↗
[ Info:     23        1  8.6e-02  3.7e-01 1.0e-01 9.5e-02  1.0e+00 5.3e-01 8.4e-01 3.2e-02 1.0e+00 ↗
[ Info:     24        1  7.4e-02  3.7e-01 8.0e-02 7.5e-02  1.0e+00 5.3e-01 8.6e-01 2.6e-02 1.0e+00 ↗
[ Info:     25        1  6.5e-02  3.7e-01 5.8e-02 5.4e-02  1.0e+00 5.3e-01 8.7e-01 2.1e-02 1.0e+00 ↗
[ Info:     26        1  5.8e-02  3.8e-01 3.9e-02 3.5e-02  1.0e+00 5.3e-01 8.8e-01 1.6e-02 1.0e+00 ↗
[ Info:     27        1  5.2e-02  3.8e-01 2.7e-02 2.5e-02  1.0e+00 5.3e-01 8.8e-01 1.1e-02 1.0e+00 ↗
[ Info:     28        1  4.8e-02  3.9e-01 1.9e-02 1.7e-02  1.0e+00 5.3e-01 8.9e-01 8.5e-03 1.0e+00 ↗
[ Info:     29        1  4.6e-02  3.9e-01 1.3e-02 1.2e-02  1.0e+00 5.3e-01 8.9e-01 6.4e-03 1.0e+00 ↗
[ Info:     30        1  4.4e-02  3.9e-01 8.7e-03 8.0e-03  1.0e+00 5.3e-01 8.9e-01 4.6e-03 1.0e+00 ↗
[ Info:     31        1  4.3e-02  3.9e-01 6.1e-03 5.7e-03  1.0e+00 5.3e-01 8.9e-01 3.4e-03 1.0e+00 ↗
[ Info:     32        1  4.2e-02  3.9e-01 4.4e-03 4.1e-03  1.0e+00 5.3e-01 8.9e-01 2.5e-03 1.0e+00 ↗
[ Info:     33        1  4.2e-02  3.9e-01 3.2e-03 3.0e-03  1.0e+00 5.3e-01 8.9e-01 1.8e-03 1.0e+00 ↗
[ Info:     34        1  4.1e-02  3.9e-01 2.3e-03 2.2e-03  1.0e+00 5.3e-01 8.9e-01 1.4e-03 1.0e+00 ↗
[ Info:     35        1  4.1e-02  3.9e-01 1.7e-03 1.6e-03  1.0e+00 5.3e-01 8.9e-01 1.0e-03 1.0e+00 ↗
[ Info:     36        1  4.1e-02  3.9e-01 1.3e-03 1.2e-03  1.0e+00 5.3e-01 8.9e-01 7.7e-04 1.0e+00 ↗
[ Info:     37        1  4.1e-02  3.9e-01 9.6e-04 9.6e-04          5.3e-01 8.9e-01 5.8e-04 1.0e+00
[ Info: TR: terminating with √(ξ1/ν) = 0.0009554889363751225
"Execution stats: first-order stationary"

MohamedLaghdafHABIBOULLAH · 2025-09-08T18:31:32Z

Thank you @MaxenceGollier it seems that they are aligned, but in the second example with LSR1, why do we have 0 sub iterations whereas in the last one we have 1?

MaxenceGollier · 2025-09-08T18:35:51Z

I guess it is because TRDH in its original version (on the master branch) starts its iterations at 1 instead of 0 this is not the case for R2 because the implementation is the same regardless of the branch.

dpo

Great work! Thank you!

MaxenceGollier added 18 commits April 29, 2025 14:52

preliminary solve! for TRDH

14676d0

further solve! TRDH

298d0b5

Merge branch 'TR-JSO' of https://github.com/MaxenceGollier/Regularize…

4efd12b

…dOptimization.jl into TR-JSO

non-allocating, JSO-compliant TRDH

4b12fa9

make TRDH equal when reduce_TR = false

2866da0

TRDH function signatures

196edbc

TR JSO-struct

c294d57

fix minor bug in TRDH

6c43a51

add JSO compliant TR

a6e9b26

solve bugs for TR with TRDH subsolver

b737d55

add doc and unit tests for TRDH

7fbfc5c

solve unallocated memory issue

5f6164b

fix reproducibility in TR

52117ed

unit tests for TR

3358ff0

TR function signatures

836c02e

add kwargs for TRDH

2bd11ef

fix uninitalized memory in TRDH

c389274

update doc for TR

ad7990d

fix reproducibility in TRDH

ca4d66c

MaxenceGollier added 3 commits July 12, 2025 11:01

fix reproducibility with TR

1abfe57

fix bound test for new TR signature

ea0eaa2

fix reproducibility with TR-TRDH

5b2cc0a

fix bug in TRDH

b77a5a3

MaxenceGollier commented Jul 13, 2025

View reviewed changes

Comment thread src/TRDH_alg.jl Outdated

This was referenced Aug 7, 2025

JSO Compliance : LM #200

Merged

JSO Compliance : LMTR #201

Merged

apply julia formatter

8202046

MaxenceGollier commented Aug 8, 2025

View reviewed changes

Comment thread src/TR_alg.jl

dpo reviewed Aug 15, 2025

View reviewed changes

Comment thread src/TR_alg.jl Outdated

dpo reviewed Aug 15, 2025

View reviewed changes

Comment thread src/TR_alg.jl Outdated

MaxenceGollier and others added 7 commits August 17, 2025 11:32

implicit hessian update fix

08bc4d6

update doc

03bd516

Co-authored-by: Dominique <dominique.orban@gmail.com>

add callback docstring

120d073

add update_bounds function

fc63949

add subsolver logger

c4f9b68

remove alpha and beta params from keyword arguments

37d9bac

add reduce_TR doc

032a2b5

dpo mentioned this pull request Sep 7, 2025

Subproblem tolerance should be defined in terms of T #208

Open

dpo requested changes Sep 7, 2025

View reviewed changes

MaxenceGollier and others added 3 commits September 8, 2025 09:37

remove FirstOrderModel

d90ba89

remove commented code

709d455

add ManualNLPModels dep

a7df86f

dpo approved these changes Sep 9, 2025

View reviewed changes

dpo merged commit b7646ce into JuliaSmoothOptimizers:master Sep 9, 2025
13 of 14 checks passed

MaxenceGollier deleted the TR-JSO branch September 9, 2025 19:42

MaxenceGollier mentioned this pull request Feb 6, 2026

Reduce allocations in solvers #64

Open

This was referenced Feb 20, 2026

Make solvers more uniform #231

Open

fix computation of ν in TRDH and LMTR #295

Open

	ν = (α * Δk)/(DNorm + one(T))
	ν = 1/(DNorm + 1/(α * Δk))

Conversation

MaxenceGollier commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

MaxenceGollier commented Jul 12, 2025

TRDH Tests

For TRDH (without reduce_TR)

For TRDH (with reduce_TR)

Uh oh!

MaxenceGollier commented Jul 12, 2025

TR Tests

For TR with R2

For TR with TRDH

Uh oh!

Uh oh!

MaxenceGollier Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

dpo Aug 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dpo Sep 7, 2025

Choose a reason for hiding this comment

Uh oh!

MaxenceGollier commented Sep 8, 2025

Uh oh!

MohamedLaghdafHABIBOULLAH commented Sep 8, 2025

Uh oh!

MaxenceGollier commented Sep 8, 2025

Uh oh!

dpo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MaxenceGollier commented Jul 11, 2025 •

edited

Loading

codecov Bot commented Jul 11, 2025 •

edited

Loading