Skip to content

Target ModelingToolkit optimization downstream tests#1267

Merged
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/target-mtk-optimization-downstream
Jul 16, 2026
Merged

Target ModelingToolkit optimization downstream tests#1267
ChrisRackauckas merged 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:agent/target-mtk-optimization-downstream

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

Ignore this PR until reviewed by @ChrisRackauckas.

Summary

Run ModelingToolkit's dedicated Optimization test group from Optimization.jl's downstream workflow instead of ModelingToolkit's aggregate All group.

Why

The aggregate group runs ModelingToolkit's entire curated suite in one Julia process, including unrelated ControlSystems downstream coverage. In the repeated failing runs, an earlier group loaded LinearSolve 3.87, the downstream environment then resolved LinearSolve 4.3, and LinearSolveArnoldiMethodExt failed because the already-loaded 3.87 module did not define ArnoldiMethodJL.

Optimization.jl only needs ModelingToolkit's declared Optimization integration group. Selecting that group gives it its own declared test environment and avoids using Optimization.jl's downstream job as a general ModelingToolkit test job.

This is a test-setup-only change. It does not add an Enzyme compatibility restriction or alter package compatibility.

Evidence

  • The same unrelated UndefVarError: ArnoldiMethodJL not defined in LinearSolve occurred on master and in PR #1265's ModelingToolkit job.
  • Local workflow-shaped run with Julia 1.12 and GROUP=Optimization: 156 passed, 1 pre-existing broken, 157 total; Testing ModelingToolkit tests passed (32m10.3s, exit 0).
  • Local standalone GROUP=Downstream control: 31/31 passed (28m17.9s, exit 0).
  • actionlint .github/workflows/Downstream.yml
  • YAML parser load check
  • git diff --check
  • Runic 1.7.0 --check over all 165 tracked Julia files

Investigation notes

  • A history predicate bisect in LinearSolve found 571d0a868c490e420f3c48f469b1589adb72d746 as the first commit adding the ArnoldiMethod extension. The extension itself works in an isolated consistent environment; the failure requires mixed loaded/resolved LinearSolve versions.
  • Align downstream LinearSolve compatibility ModelingToolkit.jl#4730 proposed pinning LinearSolve in the unrelated downstream environment and was closed unmerged. Targeting the existing caller-specific group is smaller and avoids that pin.
  • Prior art: Optimization.jl#1259 corrected NeuralPDE downstream group selection at the caller.

Process

I inspected both failing job logs, reproduced the standalone groups locally with the reusable-workflow command shape, traced ModelingToolkit's run_tests group/environment declarations, checked relevant PR history, isolated the first extension-triggering LinearSolve commit, and validated this one-line workflow change. The requested scope was a cleaner test-setup fix that assumes upstream Enzyme support will be corrected before Julia 1.13 is released.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Local same-process reproduction completed on clean Optimization.jl master c3a60666a and ModelingToolkit.jl 7a1f575ea (the downstream SHA used by the failing CI jobs).

I loaded the root/test-compatible LinearSolve 3.87 first, then invoked ModelingToolkit's existing GROUP=Downstream runner in the same Julia process. The resolver reported:

PRELOADED_LINEAR_SOLVE=3.87.0
LinearSolve v4.3.0 [loaded: v3.87.0]

It then failed exactly as CI did:

UndefVarError: `ArnoldiMethodJL` not defined in `LinearSolve`
LinearSolveArnoldiMethodExt.jl:8

Final result: Downstream | 3 errors | 3 total, exit 1. The same group passes 31/31 when run standalone in a fresh process, and the proposed GROUP=Optimization run passes 156 tests with one pre-existing broken test.

This confirms that the problem is package-version state leaking across ModelingToolkit's aggregate in-process groups, not an Optimization integration failure or a bad isolated LinearSolve extension.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

The clean, exact GROUP=All reproduction has now completed on Optimization master (c3a60666a) with ModelingToolkit at the downstream workflow's tested revision (7a1f575ea) and Julia 1.12.6. It exited 1 after approximately 2h18m.

The groups before Downstream all completed:

InterfaceI                    1396 passed, 3 broken   (62m33.6s)
Initialization                 793 passed, 12 broken  (18m05.0s)
InterfaceII                    945 passed, 7 broken   (50m06.4s)
SymbolicIndexingInterface     1964 passed              (5m29.7s)

All then activated test/downstream in the same Julia process and failed exactly as CI does:

Test Summary:                          | Error  Total     Time
Downstream                             |     3      3  1m39.1s
  Linearization Dummy Derivative Tests |     1      1    33.0s
  Inverse Models Test                  |     1      1    27.5s
  Disturbance model Test               |     1      1    38.5s

All three errors are:

UndefVarError: ArnoldiMethodJL not defined in LinearSolve
.../LinearSolveArnoldiMethodExt.jl:8

This confirms that the failure is caused by All reaching unrelated ControlSystems downstream coverage after an older LinearSolve has already been loaded. It is not an Optimization integration failure. The caller-specific GROUP=Optimization run passes locally (156 passed, 1 broken, 157 total), and the corresponding PR CI job also passes with the same counts.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Additional base-branch verification (2026-07-15): a clean current ModelingToolkit master checkout with current Optimization master developed and GROUP=Downstream completed successfully (31/31, exit 0; 40m38.69s locally).

The earlier GROUP=All failure is specifically a same-process environment-version collision, not a standalone Downstream regression: job 86695582054 first loaded LinearSolve v3.87.0, then the downstream sub-environment resolved LinearSolve v4.3.0 [loaded: v3.87.0]; v4.3's LinearSolveArnoldiMethodExt then referenced ArnoldiMethodJL on the already-loaded v3.87 module.

This supports restricting Optimization's MTK downstream lane to the dedicated Optimization group as this PR does. The exact current GROUP=All local run is still running for an independent aggregate reproduction.

Copy link
Copy Markdown
Member Author

Exact local verification for the downstream group selected by this PR has now terminated successfully:

GROUP=Optimization julia +1.12 --project=. -e 'using Pkg; Pkg.test()'

Result: Optimization | 156 passed | 1 pre-existing broken | 157 total, followed by Testing ModelingToolkit tests passed and process exit 0.

This confirms the ModelingToolkit.jl / Optimization matrix entry reaches and completes the intended substantive group locally; it is not relying only on the green hosted check.

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review July 16, 2026 09:40
@ChrisRackauckas
ChrisRackauckas merged commit 75d2054 into SciML:master Jul 16, 2026
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants