Avoid discarded objective evaluations in OptimizationOptimisers#1268
Conversation
|
Audit scratchpad for commit
No test was skipped, silenced, loosened, or changed to match buggy output. |
|
Scheduled CI checkpoint (2026-07-15 10:18 EDT):
No observed package-test failure is attributable to this PR. A separate NeuralLyapunov Benchmarking investigation should not be conflated with this result: its seeded stochastic policy tests remain context-sensitive, and a full run with this patch moves rather than robustly removes that package's accuracy failure. That will be tracked separately without changing assertions or tolerances. |
Evaluate the objective through the optimizer loop without a discarded pre-loop call, so stochastic objectives keep the expected RNG sequence and reported evaluation counts match actual work. Preserve positive fractional maxiters values that round to zero by evaluating the initial objective only when the loop performs no iterations, and count that fallback evaluation. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
1e488d7 to
33d9282
Compare
|
Post-rebase verification (2026-07-16): rebased commit Downstream audit note: a repeated full NeuralLyapunov Benchmarking run against this source passed 25/25 in 89m16.3s, but the same source failed the isolated ODESystem benchmark at 0.825 and an earlier full context failed the first benchmark at 0.62. I am therefore retaining this PR's deterministic evaluation-count/NeuralPDE evidence without claiming it reliably repairs NeuralLyapunov's stochastic benchmark. |
|
One-hour CI checkpoint (2026-07-16 15:09 EDT): 46 checks have succeeded, 3 were skipped, 1 failed, and 3 downstream checks remain in progress. The changed package's complete matrix is green: OptimizationOptimisers Core passes on LTS, current, and prerelease Julia, and its QA lane passes on current Julia. Both root AD lanes are also green. NeuralPDE NNPDE2 has completed successfully. The only failure is the standalone Documentation link check. I inspected job 87703529098 directly: Documenter received HTTP 429 responses for Still running at this checkpoint: DiffEqFlux All, ModelingToolkit Optimization, and NeuralPDE NNPDE1 downstream tests. I am recording these as in progress, not as passed or failed. |
2 participants
Summary
OptimizationOptimisers.OptimizationOptimisers 0.3.19 initialized the eventual objective result by evaluating
cache.fbefore the loop. Normal iterations overwrite that value, so the call was unreported work and advanced global RNG state for stochastic objectives before optimization began. The regression observes objective and combined objective/gradient call counts directly.Local verification
GROUP=OptimizationOptimisersharness: passed.0.16949334947633954 ≥ 0.2).The regression uses
OptimizationFunctionandOptimizationProblemfrom their declaring public module,SciMLBase, andsolvefromCommonSolve. No assertion, tolerance, or expected-broken marker was changed.A NeuralLyapunov benchmark that resamples quasi-random points from global RNG is being checked separately because this patch intentionally restores the pre-0.3.19 evaluation order; no speculative benchmark change is included here.
Please ignore this draft until it has been reviewed by @ChrisRackauckas.