Skip to content

perf: JET tooling, 2 bug fixes, and hot-path regression guards#494

Merged
ocots merged 6 commits into
mainfrom
chore/performance-roadmap
Jul 12, 2026
Merged

perf: JET tooling, 2 bug fixes, and hot-path regression guards#494
ocots merged 6 commits into
mainfrom
chore/performance-roadmap

Conversation

@ocots

@ocots ocots commented Jul 12, 2026

Copy link
Copy Markdown
Member

Summary

Performance-tooling pass for CTBase: static analysis, two real bug fixes it found, a methodology guide, and regression guards (type-stability + allocation) locking in the hot-path guarantees the investigation verified. Ships as v0.27.7-beta.

1. JET static analysis (77d474e)

JET added as a test-only dependency; JET.test_package enabled for real in test_code_quality.jl (was commented out). The first report_package correctness scan found 2 real bugs, both fixed:

  • _strategy_type_name(::UnionAll) called nameof(::TypeVar), which has no method — a latent MethodError for any strategy parameter type that's an uninstantiated generic. Fixed via TypeVar's .name field. test/suite/strategies/test_describe_registry.jl added — zero prior coverage of _strategy_type_name existed.
  • OptionValue's constructor dispatched on Val(source::Symbol) across 4 helper methods defined inside the struct body — a pattern JET's signature-based analysis can't resolve (false positive), and which also turned out to be genuinely type-unstable whenever source isn't a literal (@code_warntype showed Body::ANY before). Replaced with a single inline validation check: same behavior, no JET blind spot, now infers to OptionValue{T}.

With both fixed, JET.test_package runs clean in CI — no exclusions needed.

2. Performance guide (fce408f)

New docs/src/guide/performance.md ("Developer Tools" nav group). Documents the core principle the investigation surfaced: CTBase code splits into a hot path (called every solve step — field/Hamiltonian/law/constraint evaluation, interpolation, option reads) which must stay type-stable, and a setup path (registry/strategy construction, called once per problem) where some dynamism is fine by design. Includes 4 JET.@report_opt checks that execute live at doc-build time (Draft = false meta on this page) — a hot-path regression would make the docs build itself fail, not just go stale.

3. Type-stability regression tests (41e6934)

Test.@inferred guards added on every hot-path entry point verified stable during the investigation: VectorField/Hamiltonian/ControlLaw/PathConstraint calls, Traits.time_dependence/feedback accessors, Differentiation.ad_backend, Core.matrix2vec. Added inside each function's existing testset, not a new file. Also re-enables a previously-commented-out Strategies.get_parameter_type stability test — verified by hand it actually infers cleanly today.

Deliberately not touched: Strategies.create_registry, build_strategy_options, type_from_id, Descriptions.complete — these are setup-path functions, confirmed dispatch-heavy by design (registry/options need runtime-extensible storage). Confirmed with the user that strategies are always built once, before any solver call, never inside a solve loop — so this dynamism never reaches a hot path.

4. Allocation-contract guards (7881262)

New test/suite/meta/test_performance.jl: deterministic allocation assertions via BenchmarkTools.@ballocated, complementing the type-stability guards above (a change can stay type-stable yet start allocating — e.g. a stray collect — which @inferred won't catch). Two invariant classes, both machine-independent (allocation counts have no run-to-run noise, unlike wall-clock time — which is never asserted here):

  • Zero-overhead wrappers: VectorField/Hamiltonian/ControlLaw/PathConstraint call allocations == the raw wrapped function's (compared to raw, not a magic constant — version-independent).
  • Zero-allocation reads: interpolant evaluation (Linear + Constant), trait accessors, and the strategy-option read (ad_backend) all == 0.

BenchmarkTools added to the test target only (already a CTSolvers test dependency). Runs in ~7.5s, no new CI. Written to double as a template for the other control-toolbox packages, which don't yet have this kind of guard.

5. Housekeeping (b680092, 66becab)

Added docstrings for 3 previously-undocumented private Plotting show helpers (_SHOW_LIMIT, _show_axes, _show_node) — these were flagged by the docs build while working on item 2 above; no functional change. Version bumped to 0.27.7-beta with matching CHANGELOG.md entry and a BREAKING.md non-breaking note (this whole PR is additive: new tooling, tests, docs, and two internal bug fixes with no observable behavior change for valid usage).

Scoped out (deliberately)

  • A full PkgBenchmark/BenchmarkCI time-series setup was evaluated and deferred: CTBase's hot ops are single-digit-nanosecond (trait dispatch ~0.9 ns, wrapper call ~8.7 ns), which would produce false regressions on shared CI runners, and no sibling package in the ecosystem uses it.
  • Tier-2 setup-path dynamism (Strategies/Options construction) is accepted as-is — confirmed out of any hot loop.

Test plan

  • test/suite/strategies/test_describe_registry.jl — verified TDD-style (reverted the fix, confirmed the new test fails with the exact MethodError JET predicted, then re-applied and confirmed green)
  • test/suite/meta/test_code_quality.jl — 12/12 green (Aqua + JET, JET now genuinely clean, not excluded)
  • test/suite/meta/test_performance.jl — 10/10 green, ~7.5s
  • Docs build — the 4 live @report_opt blocks on the performance page executed and rendered clean (No errors detected), verified in the built HTML (no leaked ANSI escapes)
  • Full suite (Pkg.test("CTBase")) — 4731/4731 green
  • pkgversion(CTBase) == v"0.27.7-beta" after the bump, package still loads cleanly

🤖 Generated with Claude Code

…n CI

Starts the performance-tooling work: JET added as a test dependency and
JET.test_package enabled in test_code_quality.jl alongside Aqua.

The first report_package scan found two real issues:
- _strategy_type_name(::UnionAll) called nameof(::TypeVar), which has no
  method; TypeVar exposes its name via the .name field instead. Fixes a
  latent MethodError for any strategy parameter type that is an
  uninstantiated generic. Added test_describe_registry.jl, which had zero
  coverage of _strategy_type_name before this.
- OptionValue's constructor dispatched on Val(source::Symbol) across four
  helper methods defined inside the struct body. JET's signature-based
  analysis can't resolve that pattern (false positive), and it turned out
  to also be genuinely type-unstable whenever source isn't a literal at
  the call site (@code_warntype showed Body::ANY). Replaced with a single
  inline validation in the named inner constructor: same behavior, no
  JET blind spot, and now infers to OptionValue{T}.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ocots and others added 3 commits July 12, 2026 18:22
Documents the hot-path-vs-setup-path methodology from the performance
pass: a tools table, four live JET.@report_opt checks on hot-path entry
points (VectorField/Hamiltonian/Interpolant calls, strategy-option read)
that execute at doc-build time so a regression there breaks the build,
a pointer to the JET.test_package check already enforced in CI, and the
known/accepted setup-path dynamism (registry & strategy-options
construction, VectorField's methods(f)-based mutability auto-detection).

JET added to docs/Project.toml since the live checks need it at build
time; registered under the "Developer Tools" nav group in make.jl.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… tests

Adds Test.@inferred assertions on every hot-path entry point verified
stable via JET.@report_opt during the performance pass: VectorField/
Hamiltonian/ControlLaw/PathConstraint calls, the Traits.time_dependence/
feedback accessors, Differentiation.ad_backend (which also exercises the
Base.get(options, Val(:key)) strategy-option read path), and
Core.matrix2vec. Added to the existing testset owning each fixture rather
than a new file.

Also re-enables Strategies' commented-out get_parameter_type stability
test: verified by hand it actually infers cleanly today (Type{T}
constant propagation resolves the try/catch + .parameters[1] access at
compile time), so it was safe to turn back on. Interpolation already had
@inferred coverage on both Interpolant call variants; deliberately left
the known setup-path dispatch (registry/strategy-options construction,
Descriptions.complete) untouched rather than add permanently-red assertions
for an already-accepted trade-off.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds test/suite/meta/test_performance.jl: a dedicated "performance
contract" file asserting CTBase's deterministic allocation invariants on
the hot path, complementing the Phase-3 @inferred type-stability guards.

Two classes, both machine-independent (allocation counts have no run-to-run
noise, unlike wall-clock time, which is never asserted):
- Zero-overhead wrappers: VectorField/Hamiltonian/ControlLaw/PathConstraint
  call allocations == the raw wrapped function's (compare-to-raw, so the
  guard is independent of Julia version / word size).
- Zero-allocation reads: interpolant eval (Linear+Constant), trait
  accessors, and the strategy-option read (ad_backend) all == 0.

Uses BenchmarkTools.@ballocated (added to the test target; already a
CTSolvers test dep). Runs in ~7.5s in the normal suite — no new CI, no
flakiness. Written to double as a template for the other control-toolbox
packages.

Deliberately scoped to 5A only; the PkgBenchmark/BenchmarkCI time-series
option is deferred (single-digit-ns ops are too noisy for shared CI runners
and foreign to the ecosystem) — see .reports/phase4-benchmarking-analysis.md.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ocots ocots changed the title perf: add JET static analysis, fix 2 bugs it found perf: JET tooling, 2 bug fixes, and hot-path regression guards Jul 12, 2026
ocots and others added 2 commits July 12, 2026 22:06
Documents _SHOW_LIMIT, _show_axes, and _show_node — previously undocumented
private Plotting helpers that the docs build was flagging with "No
documentation found ... Skipping from API reference." No functional change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ance pass

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@ocots
ocots marked this pull request as ready for review July 12, 2026 20:08
@ocots
ocots merged commit 5a75658 into main Jul 12, 2026
4 checks passed
@ocots
ocots deleted the chore/performance-roadmap branch July 12, 2026 20:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant