Skip to content

Use SciMLTesting v1.0.0 (run_tests harness)#81

Closed
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:sciml-testing-rollout
Closed

Use SciMLTesting v1.0.0 (run_tests harness)#81
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:sciml-testing-rollout

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

Replaces the hand-written GROUP-dispatch in test/runtests.jl with a single SciMLTesting.run_tests(...) call (SciMLTesting v1.0.0).

Shape

Single-package, GROUP-dispatched, QA in an isolated test/qa sub-env, default GROUP="Core".

Mapping (behavior-equivalent)

  • core = thunk reproducing the original Core||All body verbatim: include("interface_tests.jl"), include("jet_tests.jl"), and the inline @ode_def_bare/solve MATLAB-runtime checks. A thunk (not a file include) is required because that body runs in the runtests.jl scope where using DiffEqBase, MATLABDiffEq, ParameterizedFunctions is in effect.
  • QA = groups["QA"] with a declared env = test/qa. A declared-env group is not run under "All" and is selectable only via GROUP="QA", exactly matching the original (if GROUP == "QA"). activate_group_env reproduces the old activate_qa_env (activate test/qa + Pkg.develop the repo root + instantiate). QA body kept verbatim (custom JET.report_package + report-count assertion), not collapsed into run_qa.
  • default = "Core" preserves the original get(ENV, "GROUP", "Core").

Routing verified for every CI-relevant GROUP: unset→core, All→core, Core→core, QA→qa.

Project.toml

  • [extras]/[targets].test/[compat]: add SciMLTesting (UUID 09d9d899-...), drop Pkg (only the old harness used it; Pkg. appears in no other test file; SciMLTesting bundles Pkg).

Ignore until reviewed by @ChrisRackauckas.

🤖 Generated with Claude Code

Replace the hand-written GROUP-dispatch in test/runtests.jl with a single
SciMLTesting.run_tests(...) call. Behavior-equivalent refactor: same Core/QA
group bodies, same group selection. Add SciMLTesting to the test deps and
drop Pkg (now bundled by SciMLTesting; only used by the old harness).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Closing: this run_tests v1.0.0 conversion is NOT behavior-equivalent. This repo runs QA only for GROUP=="QA" and excludes it from "All" (the default local ]test). In released SciMLTesting v1.0.0 the only encodings are (a) the qa= kwarg, which also runs QA under "All" (changes behavior), or (b) a groups["QA"] entry, which is unreachable for GROUP=="QA": run_tests' elseif group=="QA" branch short-circuits and throws no qa body was provided before reaching the groups table (src lines 499-501, before 504). Empirically reproduced (GROUP="QA" throws). Blocked on SciML/SciMLTesting.jl#5; re-attempt after that lands + a new release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants