Resolve benchmark MTKBase before runner startup

[ChrisRackauckas-Claude]

Please ignore this PR until it has been reviewed by @ChrisRackauckas.

Summary

• resolve ModelingToolkitBase from the pull request head repository and SHA before AirspeedVelocity launches the benchmark runner
• remove the late Pkg.develop mutation from benchmark/benchmarks.jl
• route fork-PR benchmark tables to the job summary, where the read-only fork token does not need comment-write access
• keep ModelingToolkitBase source changes in the benchmark environment through Pkg's supported URL#revision:subdirectory monorepo specification

Dependency and current base

This fix depends on the typo correction in #4745. That PR merged while this PR was queued, so this branch is now rebased onto current master at db56a6a7d174f44efc96511a9679fbca23c323d8; #4745 is already in its ancestry. The PR therefore contains two focused benchmark commits on top of master, without duplicating the merged prerequisite.

Root cause

AirspeedVelocity imports the benchmarked ModelingToolkit package before it includes the benchmark script. The script then called Pkg.develop for the local ModelingToolkitBase checkout after the registered ModelingToolkitBase module was already loaded. Julia retained the registered module while the manifest expected the local module, producing a loaded-path mismatch, missing-cache warnings, and eventually:

UndefVarError: analytically_integrated not defined in ModelingToolkit

A formal bisect from 6b5d6c6dcd to 9e7dc990cb identifies 9e7dc990cb04dae4f66e16d32bacd23e594ba32a (ci: use local MTKBase in benchmarks) as the first bad commit. Its own historical benchmark run (Actions run 26644089638) already failed with an incompatible ModelingToolkit/ModelingToolkitBase precompile, so the late environment mutation never provided valid source resolution.

No existing open issue or PR covers this benchmark failure.

Investigation notes

A local-path extra-pkgs candidate was rejected after the exact Airspeed runner showed that Pkg.add requires the supplied path to be a Git repository; lib/ModelingToolkitBase is a monorepo subdirectory. The final workflow uses the pull request head clone URL and SHA with Pkg's supported monorepo syntax, so both fork and same-repository pull requests resolve the intended source before any package is loaded.

Validation

Using Julia 1.12.4, AirspeedVelocity 0.6.5, and an isolated workspace depot and temporary directory:

• clean master reproduction: exit 1, with registered/local ModelingToolkitBase loaded-path mismatch followed by the exact analytically_integrated error
• corrected Airspeed run with https://github.com/SciML/ModelingToolkit.jl.git#master:lib/ModelingToolkitBase: exit 0
• benchmark output:

  [runner] Running benchmarks for ModelingToolkit@master.
  (1/1) benchmarking "mtkcompile"...
  done (took 6.492992116 seconds)
  [runner] Benchmark runner exited.
  [Info] Finished.
  

• fork-head resolution check for ChrisRackauckas-Claude/ModelingToolkit.jl at 94efb41cdb1423c7d35f42140a88e503bd554183: exit 0; Pkg status reported ModelingToolkitBase from that repository, subdirectory, and SHA
• Typos CLI 1.48.0 repository check: exit 0
• Runic 1.7.0 repository-wide --check .: exit 0
• actionlint workflow validation: exit 0
• exact extra-pkgs and fork job-summary assertions: exit 0
• git diff --check upstream/master...HEAD: exit 0
• stable patch ID before and after the base-only rebase: 93b2db3b6ee5a0f707b244264605225984eaf398

The Airspeed filter is applied after the benchmark script is included, so the full script setup and first-call workloads were exercised; only the measured benchmark group was narrowed to mtkcompile for the local before/after run. ASV was not rerun after rebasing because the benchmark patch is unchanged; the exact earlier ASV result is reported above.

[ChrisRackauckas-Claude]

CI follow-up after rebasing onto merged #4745:

• The initial code-check failure was Spell Check run 29280222240, job 86919309599 on old head 624ddd7ef3bbb75a007852989d504867709113e5.
• That job explicitly ran crate-ci/typos@v1.48.0 against the synthetic merge into base 34ee9182900688f8545d61fb92d143ed29a8ad7f and reported:

  error: `reprsenting` should be `representing`
     --> ./src/systems/alias_elimination.jl:494:20
  

• The benchmark PR does not modify that file. Correct alias elimination docstring typo #4745 fixed that exact clean-master typo in 94efb41cdb1423c7d35f42140a88e503bd554183 and has now merged.
• I rebased this PR onto current master (db56a6a7d174f44efc96511a9679fbca23c323d8), which contains Correct alias elimination docstring typo #4745. New head: 7ba423c11678724427b77fa3ef8a3a065adddb06.
• Local checks on the new head: Typos CLI 1.48.0 exit 0; Runic 1.7.0 repo-wide --check . exit 0; YAML parse plus exact workflow input assertion exit 0; git diff --check upstream/master...HEAD exit 0.
• The stable patch ID is unchanged across the rebase (93b2db3b6ee5a0f707b244264605225984eaf398). I did not rerun ASV after the base-only rebase; the earlier exact passing ASV run applies to the identical benchmark patch and remains documented in the PR body.

[ChrisRackauckas-Claude]

Current red-check classification after exact log inspection:

• Both Catalyst Modeling jobs fail at parent_sys in symbolic_stoichiometry.jl; Catalyst Feature Request: Option to define limits for parameters when using ModelingToolkit.jl #1509 fixes that clean-base regression, and its focused test plus full symbolic_stoichiometry.jl passed locally on Julia 1.12.6.
• Downgrade InterfaceI fails because CodegenFunctionOptions is missing; that clean-base failure is dependency-gated by Symbolics Help Solving Coupled Differential Equations Involving Complex Numbers #1928 and ModelingToolkit Require Symbolics 7.32 for code generation #4744.
• ModelingToolkitStandardLibrary Core fails only the TranslationalPosition AccelerationSensor case, with 15 passes and 1 error; MTSL Restructure tutorial section of the documentation #489 fixes it, and the full translational file passed locally 16/16 on Julia 1.10.11 and 1.12.6.

These failures are independent of the benchmark-workflow change. The benchmark jobs themselves are still running.

[ChrisRackauckas-Claude]

Benchmark workflow follow-up on head cf3e05d:

• The Julia LTS benchmark completed both revisions and produced the full time/memory tables. The benchmarked mtkcompile entry completed; the job failed afterward because the fork PR token received HTTP 403 while posting the table comment.
• The Julia 1 matrix job was canceled by fail-fast after the LTS permission failure while still resolving the master benchmark environment.
• The AirspeedVelocity action supports job-summary mode. This head enables that mode only when github.event.pull_request.head.repo.fork is true, so fork PRs write their table to the job summary while same-repository PRs retain the existing comment behavior.
• Local verification on cf3e05d: actionlint exit 0; Typos 1.48.0 exit 0; whole-repository Runic 1.7.0 exit 0; exact workflow-input assertion and git diff --check exit 0.

Fresh CI is now running.

[ChrisRackauckas-Claude]

Replacement benchmark CI is complete on head cf3e05d:

• Julia LTS: passed in 30m30s.
• Julia current: passed in 44m47s.
• Both jobs ran the benchmark action end to end and published results through the fork-safe job-summary path.

This is direct CI validation of both changes in this PR. The remaining red checks are the independently reproduced clean-base failures already classified above.

[ChrisRackauckas-Claude]

Follow-up CI classification:

• Both benchmark jobs passed: LTS in 30m30s and current Julia in 44m47s.
• The QA red check has one GitHub annotation: the self-hosted runner lost communication with the server. This is infrastructure failure, not a reported test assertion.
• The ModelingToolkitBase downgrade job and InterfaceI downgrade job fail on the clean-base CodegenFunctionOptions dependency floor, which is addressed separately by Require Symbolics 7.32 for code generation #4744 and verified locally against registered Symbolics 7.32.0.
• The remaining Catalyst and ModelingToolkitStandardLibrary reds are clean-base failures already routed to focused upstream fixes.

No additional change to this benchmark PR is indicated by these results.