bench: Ipopt + MadNLP allocation profile script#71
Closed
jack-champagne wants to merge 42 commits into
Closed
Conversation
Bumps [julia-actions/cache](https://github.com/julia-actions/cache) from 2 to 3. - [Release notes](https://github.com/julia-actions/cache/releases) - [Commits](julia-actions/cache@v2...v3) --- updated-dependencies: - dependency-name: julia-actions/cache dependency-version: '3' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5 to 6. - [Release notes](https://github.com/codecov/codecov-action/releases) - [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md) - [Commits](codecov/codecov-action@v5...v6) --- updated-dependencies: - dependency-name: codecov/codecov-action dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
Sets up benchmark/ with Project.toml, .gitignore, and three @testitem benchmarks: evaluator micro-benchmarks, Ipopt vs MadNLP comparison, and memory scaling sweep (N × state_dim grid). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Specs for: - Overall HarmoniqsBenchmarks.jl architecture - Altissimo GPU benchmarks (3-way: Ipopt CPU / MadNLP-GPU / Altissimo-GPU) - Implementation plan for DirectTrajOpt + HarmoniqsBenchmarks tasks
- Add .github/workflows/benchmark.yml that runs on PRs touching src/ or benchmark/ - Uses Pkg.add(url=...) to install HarmoniqsBenchmarks (unregistered) - Uploads JLD2 artifacts for 90 days - Add benchmark/README.md with run instructions - Remove empty BenchmarkUtils.jl leftover - Ignore Manifest.toml (regenerated on each CI run)
Uses Julia 1.11+ [sources] section to resolve: - DirectTrajOpt from local path (parent dir) - HarmoniqsBenchmarks from public GitHub URL CI workflow simplified to just Pkg.instantiate (no manual Pkg.add needed).
Remove docs/superpowers/ (specs and plans) from this PR to keep the diff focused on benchmarks and MadNLP integration. Fix stale Pkg.add instruction in benchmark README — deps resolve via [sources] now. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The benchmark @testItems require HarmoniqsBenchmarks which is only available in the benchmark/ project environment, not the test extras. Filter them out so `Pkg.test()` / julia-runtest CI doesn't pick them up. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
MadNLP Integration
Bumps [julia-actions/setup-julia](https://github.com/julia-actions/setup-julia) from 2 to 3. - [Release notes](https://github.com/julia-actions/setup-julia/releases) - [Commits](julia-actions/setup-julia@v2...v3) --- updated-dependencies: - dependency-name: julia-actions/setup-julia dependency-version: '3' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com>
…-actions/setup-julia-3 Bump julia-actions/setup-julia from 2 to 3
…ov/codecov-action-6 Bump codecov/codecov-action from 5 to 6
…-actions/cache-3 Bump julia-actions/cache from 2 to 3
Restores pass-throughs consumed by MadNLP's MOI layer so users can select CUDSSSolver, CuArray, KKT variants, and cuDSS ordering through the MadNLPOptions struct (required for MadNLPGPU/cuDSS-on-GPU flows). set_options! now skips fields left as `nothing` so MadNLP's own defaults stand.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This reverts commit 6b50bc7.
….9.1 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
benchmarks: Initial DirectTrajOpt benchmark suite (Ipopt vs MadNLP)
Companion to fix_trajectory_variable! for the global-variable case. Pins a global variable via GlobalEqualityConstraint with dedup logic that handles globals (the existing fix_trajectory_variable! filter only dedupes non-global constraints). This enables Intonato's QILC alternating calibration to bake learned parameters back into the control NLP integrator-agnostically — any globals-aware integrator (HermitianExponentialIntegrator, SplineIntegrator, NonHermitianExponentialIntegrator) reads the pinned value through the NLP variable instead of requiring an integrator rebuild on every iteration. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add MadNLP pass-through fields (linear_solver, array_type, etc.)
…4-26-01-02-53-176-00922538656 CompatHelper: bump compat for OrdinaryDiffEqTsit5 to 2, (keep existing compat)
Version bump v0.9.1
…dinaryDiffEq v6, which was never removed), to prevent upstream havoc with QuantumToolbox compat
Version bump to v0.9.2; add back in previously removed support for SciMLBase v2
Chore/coverage improvements: adding tests covering new Solvers interface, MadNLP support, etc.
Adds benchmark/alloc_profile.jl which runs Profile.Allocs-traced solves on the shared bilinear toy problem for both in-tree NLP backends, saving AllocProfileResult JLD2s via HarmoniqsBenchmarks' benchmark_memory! / save_alloc_profile API. benchmark/Project.toml pins HarmoniqsBenchmarks to the feat/alloc-profile branch until that PR merges.
analyze_allocs.jl ranks alloc hotspots by scaled bytes across all profile JLD2s in benchmark/results/allocs/ with noise-frame filtering. Companion to the existing alloc_profile.jl driver. alloc_profile.jl: coerce MadNLP.ERROR to Int for MadNLPOptions — symmetric to Piccolissimo commit 472fe4d.
Identified by Gennadi. abs2(tr(U_goal'·U))/n² is phase-insensitive, so +im·U_goal and -im·U_goal are equally valid optima. With free Δt, solvers fall into different basins depending on init + step sequence; with fixed Δt, both consistently land in the same basin.
jack-champagne
force-pushed
the
bench/ipopt-madnlp-alloc-profile
branch
from
9ba5557 to
a447b58
Compare
3 tasks
Member
Author
|
Closing — fully superseded. The valuable HBJ-shaped piece of this PR ( Other content here is no longer worth landing:
The base branch ( |
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
benchmark/alloc_profile.jl— standalone Julia script that runsDirectTrajOpt.solve!underProfile.Allocsfor both Ipopt and MadNLP on the shared bilinear toy problem, saving oneAllocProfileResultJLD2 per solver viaHarmoniqsBenchmarks.benchmark_memory!/save_alloc_profile.benchmark/Project.tomlwith DTO + the two NLP backends +HarmoniqsBenchmarks(pinned tofeat/alloc-profileuntil HarmoniqsBenchmarks.jl#1 merges)./benchmark/results/so local profile output stays out of history.The sibling
Piccolissimo.jlPR covers the Altissimo+MadNLP+Ipopt three-way on an X-gate; this PR is the DTO-only lane that tracks Ipopt/MadNLP performance over time on DTO's own test fixture.Rationale
sample_ratedefaults to0.01inmain()— Ipopt/MadNLP generate orders of magnitude more fine-grained allocations than asample_rate=1.0trace can complete in reasonable wall time (an N=10 bilinear toy hangs 15+ minutes at 1.0).0.01still gives statistically useful traces for hot-path triage.JIT warmup runs on a throwaway
deepcopyof the problem before the profiled pass so compile-time allocations stay out of the recorded trace.Dependencies
Blocked on HarmoniqsBenchmarks.jl#1 merging — once it does, drop the
rev =pin frombenchmark/Project.toml.Test plan
julia --project=benchmark benchmark/alloc_profile.jlproduces two*_allocs.jld2underbenchmark/results/allocs/AllocProfileResult.total_count > 0andtotal_bytes > 0for both solversload_alloc_profileroundtrips correctly