perf: JET tooling, 2 bug fixes, and hot-path regression guards#494
Merged
Conversation
…n CI
Starts the performance-tooling work: JET added as a test dependency and
JET.test_package enabled in test_code_quality.jl alongside Aqua.
The first report_package scan found two real issues:
- _strategy_type_name(::UnionAll) called nameof(::TypeVar), which has no
method; TypeVar exposes its name via the .name field instead. Fixes a
latent MethodError for any strategy parameter type that is an
uninstantiated generic. Added test_describe_registry.jl, which had zero
coverage of _strategy_type_name before this.
- OptionValue's constructor dispatched on Val(source::Symbol) across four
helper methods defined inside the struct body. JET's signature-based
analysis can't resolve that pattern (false positive), and it turned out
to also be genuinely type-unstable whenever source isn't a literal at
the call site (@code_warntype showed Body::ANY). Replaced with a single
inline validation in the named inner constructor: same behavior, no
JET blind spot, and now infers to OptionValue{T}.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Documents the hot-path-vs-setup-path methodology from the performance pass: a tools table, four live JET.@report_opt checks on hot-path entry points (VectorField/Hamiltonian/Interpolant calls, strategy-option read) that execute at doc-build time so a regression there breaks the build, a pointer to the JET.test_package check already enforced in CI, and the known/accepted setup-path dynamism (registry & strategy-options construction, VectorField's methods(f)-based mutability auto-detection). JET added to docs/Project.toml since the live checks need it at build time; registered under the "Developer Tools" nav group in make.jl. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… tests Adds Test.@inferred assertions on every hot-path entry point verified stable via JET.@report_opt during the performance pass: VectorField/ Hamiltonian/ControlLaw/PathConstraint calls, the Traits.time_dependence/ feedback accessors, Differentiation.ad_backend (which also exercises the Base.get(options, Val(:key)) strategy-option read path), and Core.matrix2vec. Added to the existing testset owning each fixture rather than a new file. Also re-enables Strategies' commented-out get_parameter_type stability test: verified by hand it actually infers cleanly today (Type{T} constant propagation resolves the try/catch + .parameters[1] access at compile time), so it was safe to turn back on. Interpolation already had @inferred coverage on both Interpolant call variants; deliberately left the known setup-path dispatch (registry/strategy-options construction, Descriptions.complete) untouched rather than add permanently-red assertions for an already-accepted trade-off. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds test/suite/meta/test_performance.jl: a dedicated "performance contract" file asserting CTBase's deterministic allocation invariants on the hot path, complementing the Phase-3 @inferred type-stability guards. Two classes, both machine-independent (allocation counts have no run-to-run noise, unlike wall-clock time, which is never asserted): - Zero-overhead wrappers: VectorField/Hamiltonian/ControlLaw/PathConstraint call allocations == the raw wrapped function's (compare-to-raw, so the guard is independent of Julia version / word size). - Zero-allocation reads: interpolant eval (Linear+Constant), trait accessors, and the strategy-option read (ad_backend) all == 0. Uses BenchmarkTools.@ballocated (added to the test target; already a CTSolvers test dep). Runs in ~7.5s in the normal suite — no new CI, no flakiness. Written to double as a template for the other control-toolbox packages. Deliberately scoped to 5A only; the PkgBenchmark/BenchmarkCI time-series option is deferred (single-digit-ns ops are too noisy for shared CI runners and foreign to the ecosystem) — see .reports/phase4-benchmarking-analysis.md. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Documents _SHOW_LIMIT, _show_axes, and _show_node — previously undocumented private Plotting helpers that the docs build was flagging with "No documentation found ... Skipping from API reference." No functional change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ance pass Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ocots
marked this pull request as ready for review
July 12, 2026 20:08
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Performance-tooling pass for CTBase: static analysis, two real bug fixes it found, a methodology guide, and regression guards (type-stability + allocation) locking in the hot-path guarantees the investigation verified. Ships as v0.27.7-beta.
1. JET static analysis (
77d474e)JETadded as a test-only dependency;JET.test_packageenabled for real intest_code_quality.jl(was commented out). The firstreport_packagecorrectness scan found 2 real bugs, both fixed:_strategy_type_name(::UnionAll)callednameof(::TypeVar), which has no method — a latentMethodErrorfor any strategy parameter type that's an uninstantiated generic. Fixed viaTypeVar's.namefield.test/suite/strategies/test_describe_registry.jladded — zero prior coverage of_strategy_type_nameexisted.OptionValue's constructor dispatched onVal(source::Symbol)across 4 helper methods defined inside the struct body — a pattern JET's signature-based analysis can't resolve (false positive), and which also turned out to be genuinely type-unstable wheneversourceisn't a literal (@code_warntypeshowedBody::ANYbefore). Replaced with a single inline validation check: same behavior, no JET blind spot, now infers toOptionValue{T}.With both fixed,
JET.test_packageruns clean in CI — no exclusions needed.2. Performance guide (
fce408f)New
docs/src/guide/performance.md("Developer Tools" nav group). Documents the core principle the investigation surfaced: CTBase code splits into a hot path (called every solve step — field/Hamiltonian/law/constraint evaluation, interpolation, option reads) which must stay type-stable, and a setup path (registry/strategy construction, called once per problem) where some dynamism is fine by design. Includes 4JET.@report_optchecks that execute live at doc-build time (Draft = falsemeta on this page) — a hot-path regression would make the docs build itself fail, not just go stale.3. Type-stability regression tests (
41e6934)Test.@inferredguards added on every hot-path entry point verified stable during the investigation:VectorField/Hamiltonian/ControlLaw/PathConstraintcalls,Traits.time_dependence/feedbackaccessors,Differentiation.ad_backend,Core.matrix2vec. Added inside each function's existing testset, not a new file. Also re-enables a previously-commented-outStrategies.get_parameter_typestability test — verified by hand it actually infers cleanly today.Deliberately not touched:
Strategies.create_registry,build_strategy_options,type_from_id,Descriptions.complete— these are setup-path functions, confirmed dispatch-heavy by design (registry/options need runtime-extensible storage). Confirmed with the user that strategies are always built once, before any solver call, never inside a solve loop — so this dynamism never reaches a hot path.4. Allocation-contract guards (
7881262)New
test/suite/meta/test_performance.jl: deterministic allocation assertions viaBenchmarkTools.@ballocated, complementing the type-stability guards above (a change can stay type-stable yet start allocating — e.g. a straycollect— which@inferredwon't catch). Two invariant classes, both machine-independent (allocation counts have no run-to-run noise, unlike wall-clock time — which is never asserted here):VectorField/Hamiltonian/ControlLaw/PathConstraintcall allocations==the raw wrapped function's (compared to raw, not a magic constant — version-independent).ad_backend) all== 0.BenchmarkToolsadded to the test target only (already a CTSolvers test dependency). Runs in ~7.5s, no new CI. Written to double as a template for the other control-toolbox packages, which don't yet have this kind of guard.5. Housekeeping (
b680092,66becab)Added docstrings for 3 previously-undocumented private
Plottingshow helpers (_SHOW_LIMIT,_show_axes,_show_node) — these were flagged by the docs build while working on item 2 above; no functional change. Version bumped to 0.27.7-beta with matchingCHANGELOG.mdentry and aBREAKING.mdnon-breaking note (this whole PR is additive: new tooling, tests, docs, and two internal bug fixes with no observable behavior change for valid usage).Scoped out (deliberately)
PkgBenchmark/BenchmarkCItime-series setup was evaluated and deferred: CTBase's hot ops are single-digit-nanosecond (trait dispatch ~0.9 ns, wrapper call ~8.7 ns), which would produce false regressions on shared CI runners, and no sibling package in the ecosystem uses it.Strategies/Optionsconstruction) is accepted as-is — confirmed out of any hot loop.Test plan
test/suite/strategies/test_describe_registry.jl— verified TDD-style (reverted the fix, confirmed the new test fails with the exactMethodErrorJET predicted, then re-applied and confirmed green)test/suite/meta/test_code_quality.jl— 12/12 green (Aqua + JET, JET now genuinely clean, not excluded)test/suite/meta/test_performance.jl— 10/10 green, ~7.5s@report_optblocks on the performance page executed and rendered clean (No errors detected), verified in the built HTML (no leaked ANSI escapes)Pkg.test("CTBase")) — 4731/4731 greenpkgversion(CTBase) == v"0.27.7-beta"after the bump, package still loads cleanly🤖 Generated with Claude Code