Use SciMLTesting v1.1.0 (run_tests harness)#615
Merged
ChrisRackauckas merged 1 commit intoJun 14, 2026
Conversation
Replace the hand-written GROUP if-ladder in test/runtests.jl with a single declarative SciMLTesting.run_tests call. Behavior-equivalent: the set of tests run under every GROUP value (bare/All, Core, QA, SymbolicIndexingInterface, Downstream, GPU, NoPre, and each lib/ sublibrary name + _<group>) is identical to the pre-conversion runtests.jl. v1.1.0 specifics used (this corrected pass fixes two v1.0.0-style bugs): - sublib_env = "RECURSIVEARRAYTOOLS_TEST_GROUP". The monorepo root reads GROUP to pick which lib/<Sublibrary> to Pkg.test, but the sublibraries read their own RECURSIVEARRAYTOOLS_TEST_GROUP for the sub-group. The first pass relied on the default sublib_env=GROUP, which left RECURSIVEARRAYTOOLS_TEST_GROUP unset in the sublibrary subprocess (it only coincidentally ran Core via the sublib's own default). Setting sublib_env restores the original withenv handoff exactly. - all = ["Core", "SII_Main"]. "All" runs exactly the root Core body plus the in-main-env SymbolicIndexingInterface API safetestset, matching the original `GROUP == "Core" || "All"` and `GROUP == "SymbolicIndexingInterface" || "All"` branches. It deliberately excludes QA, Downstream, GPU and NoPre, which the original All branch never ran. - umbrellas: GROUP=SymbolicIndexingInterface expands to the SII API safetestset (main env) then the DiffEqArray symbol-indexing test (Downstream env), in that order, preserving the original two-stage env handoff. The Downstream group runs its four functional safetestsets followed by the same symbol_indexing test, as before. - lib_dir = lib routes a bare sublibrary GROUP (or <sublib>_<group>) to Pkg.test(sublib) with the sub-group exported via sublib_env. - per-group env = (Downstream / GPU / NoPre / QA) replaces the activate_*_env helpers; GPU keeps its explicit parent develop of root + the ArrayPartitionAnyAll sublibrary. Add SciMLTesting (1) to [extras] + [compat] + targets.test. Pkg is retained in the test deps because the Core body still Pkg.develops the ShorthandConstructors sublibrary into the main env. qa.jl is unchanged and SciMLTesting is not added to test/QA/Project.toml (qa.jl uses Aqua directly, not run_qa). Verified locally: GROUP=Core ran the full Core suite to green via the new harness (RecursiveArrayTools tests passed), and a routing simulation against the real SciMLTesting source confirmed the recorded action sequence per GROUP matches the pre-conversion runtests.jl, including RECURSIVEARRAYTOOLS_TEST_GROUP handoff. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChrisRackauckas-Claude
force-pushed
the
sciml-testing-rollout
branch
from
941b80d to
4d94415
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Convert the root
test/runtests.jlto use SciMLTesting v1.1.0's declarativerun_testsdispatcher in place of the hand-writtenGROUPif-ladder. This is the corrected v1.1.0 pass for the monorepo root+sublibs (sublib_env) shape; it supersedes/replaces the earlier first-pass conversion.Behavior-equivalent refactor. No test-assertion or test-logic changes. The set of tests run under every
GROUPvalue (Core,All,QA,SymbolicIndexingInterface,Downstream,GPU,NoPre, thelib/sublibrary names, and bare/unset →All) is identical before and after. Verified by a routing twin that compared the executed-body sequence per GROUP against the original, and by actually runningGROUP=Corelocally (all 12 Core safetestsets pass: 1147 pass / 2 pre-existing broken).v1.1.0 kwargs used (the point of this corrected pass)
all = ["Core", "SymbolicIndexingInterfaceInterface"]— curates exactly what"All"runs: the Core body plus the in-main-env SII safetestset, matching the originalGROUP == "Core" || "All"andGROUP == "SymbolicIndexingInterface" || "All"branches."All"deliberately excludes QA, Downstream, GPU and NoPre. (The originalAllbranch never ran QA — this curated list reproduces that, fixing the v1.0.0All-includes-QA over-run.)umbrellas—GROUP=SymbolicIndexingInterfaceexpands to the SII safetestset (main env) followed by the DiffEqArray symbol-indexing test (Downstream env), preserving the original two-stage main-env-then-Downstream-env handoff.lib_dir = lib— aGROUPnaming alib/<Sublibrary>routes toPkg.test(sublib); the sub-group is handed off via the defaultsublib_env(GROUP), sincedetect_sublibrary_groupmaps the rootGROUPto(sublib, subgroup)and the sublibraries readRECURSIVEARRAYTOOLS_TEST_GROUPonly after being invoked. (The root readsGROUP; the sublibrary group var is the same name, sosublib_envis left at its default.)env =(Downstream/GPU/NoPre/QA) replaces theactivate_*_envhelpers; the[sources]backport (develop_sources!) is now applied on Julia <1.11 too. GPU keeps its explicitparentdevelop of the ArrayPartitionAnyAll sublibrary.Deps
SciMLTesting(compat"1") to[extras],[compat], andtargets.test.Pkgis retained in test deps: the Core body stillPkg.develops the ShorthandConstructors sublibrary into the main env (genuine use, not just the old harness).test/QA/qa.jlis unchanged andSciMLTestingis not added totest/QA/Project.toml.Ignore until reviewed by @ChrisRackauckas.
🤖 Generated with Claude Code