Skip to content

Use SciMLTesting v1.2 (folder-based run_tests)#158

Merged
ChrisRackauckas merged 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:sciml-testing-rollout
Jun 14, 2026
Merged

Use SciMLTesting v1.2 (folder-based run_tests)#158
ChrisRackauckas merged 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:sciml-testing-rollout

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jun 13, 2026

Copy link
Copy Markdown
Member

Converts test/runtests.jl to the SciMLTesting v1.2 folder-discovery model: the whole file becomes

using SciMLTesting
run_tests()

run_tests() reads test/test_groups.toml (unchanged) and discovers each group's files from the folder layout, running every file in its own isolated @safetestset. This is behavior-equivalent to the old hand-written GROUP dispatch.

Folder mapping (behavior-preserving)

  • Core (GROUP == "Core", and the Core portion of "All"): the top-level test/*.jl files — DataReduction.jl, deim.jl, utils.jl — exactly the three files the old GROUP == "All" || "Core" block ran (as POD/utils/DEIM). They stay at the top level (no move) and run in the main test env.
  • QA (GROUP == "QA"): the existing test/qa/ folder — qa_tests.jl, explicit_imports_tests.jl, jet_tests.jl — the same three files the old GROUP == "All" || "QA" block ran. The folder carries its own test/qa/Project.toml, so the harness activate_group_env(test/qa) does Pkg.activate + Pkg.develop(repo root) + Pkg.instantiate(), equivalent to the old manual Pkg.activate/Pkg.develop/Pkg.instantiate.

test/test_groups.toml is unchanged ([Core], [QA]); the CI matrix runs Core and QA as separate lanes, matching the harness mapping.

Project.toml

  • test/Project.toml: add SciMLTesting = "09d9d899-5365-40a9-917a-5f67fddea283" (+ [compat] SciMLTesting = "1"); drop Pkg (deps + compat) — only the old harness used it (no test/*.jl references Pkg.; the v1.2 harness owns all Pkg operations). SafeTestsets and Test retained.
  • test/qa/Project.toml: add SciMLTesting (+compat "1"), SafeTestsets (+compat "0.0.1, 0.1"), Test (+compat "1") so the folder-discovered files run under @safetestset in the QA sub-env. Added a relative [sources] ModelOrderReduction = {path = "../.."} (belt-and-suspenders for Julia 1.11+; activate_group_env also develops the package).

Verification

  • TOML.parsefile on every Project.toml + test_groups.toml; asserted SciMLTesting UUID/compat present, Pkg removed from test deps, test_groups.toml keys unchanged.
  • Meta.parseall on runtests.jl and every test file: no parse errors.
  • Drove SciMLTesting v1.2.0's folder-discovery resolution: Core{DataReduction, deim, utils}, QA{qa_tests, explicit_imports_tests, jet_tests} with sub-env activation, All → Core (QA excluded by design, run as its own CI lane). Matches the old dispatcher.

Heavy suite not run locally; per-PR CI validates.

Ignore until reviewed by @ChrisRackauckas.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude ChrisRackauckas-Claude changed the title Use SciMLTesting v1.0.0 (run_tests harness) Use SciMLTesting v1.2 (folder-based run_tests) Jun 14, 2026
@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review June 14, 2026 23:54
@ChrisRackauckas
ChrisRackauckas merged commit 0497c0b into SciML:main Jun 14, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants