Skip to content

Commit f5a9eb1

Browse files
ChrisRackauckas-ClaudeChrisRackauckasclaude
authored
Uniformize monorepo structure to OrdinaryDiffEq canonical style (#611)
* Uniformize monorepo structure to OrdinaryDiffEq canonical style Align RecursiveArrayTools.jl with the OrdinaryDiffEq.jl canonical monorepo layout: - Add [sources] to each sublibrary pointing at the umbrella root via ../.. (true leaf->root dependency direction; the 3 sublibs depend on the registered root today). No cyclic root->leaf [sources] are added because the root does not depend on its sublibs. - Remove the redundant double-dispatch of the sublibraries: the root Tests.yml no longer GROUP-dispatches RaggedArrays/ArrayPartitionAnyAll/ ShorthandConstructors, and the root test/runtests.jl no longer activates and Pkg.tests them. Sublibraries are now covered exclusively by SublibraryCI's per-project model. - Rewrite root test/runtests.jl as a _detect_sublibrary_group dispatcher mirroring OrdinaryDiffEq: a GROUP naming lib/<X> activates that sublib, develops its [sources] path deps on Julia < 1.11, sets the sub group env, and Pkg.tests it; otherwise the root runs its own @safetestset groups. - SublibraryCI.yml: thread group-env-name RECURSIVEARRAYTOOLS_TEST_GROUP and check-bounds auto. - DowngradeSublibraries.yml: add group-env-name/group-env-value Core, expand the skip list to the siblings (root + 3 sublibs) plus stdlibs; auto-discovers lib/* (sublibrary-downgrade.yml@v1 has no allow-reresolve input). - Each sublibrary test/runtests.jl now reads get(ENV, "RECURSIVEARRAYTOOLS_TEST_GROUP", "Core") and gates its tests on the Core group. - Add test/test_groups.toml to each sublibrary declaring only [Core] on [lts, 1.11, 1, pre], since none of them have QA-group tests (the default expansion would otherwise emit an unsupported QA group on [1]). Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * Use canonical Title-case All test-group name in sublib gates Match the SciML/OrdinaryDiffEq.jl convention: the catch-all test group is canonical Title-case All, not all-caps ALL. The sublib runtests.jl gates compared TEST_GROUP against "ALL" while the default and the test_groups.toml key are already canonical ("Core"). Align the gate string casing so all group-name strings are consistent. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: canonicalize TagBot, .gitignore, and NoPre group casing - TagBot.yml: adopt canonical OrdinaryDiffEq structure — top-level permissions block, workflow_dispatch.inputs.lookback default "3", named root TagBot job, and a TagBot-Subpackages matrix (fail-fast: false) over the three registered sublibraries, each invoking TagBot@v1 with subdir: lib/<package>. - .gitignore: complete to canonical set (docs/build, LocalPreferences.toml, *.jl.*.mem, *.DS_Store, profile.pb.gz, .*.swp, .claude/, etc.). - Casing: rename the special test group "nopre" -> "NoPre" on both sides of the workflow<->runtests contract (Tests.yml matrix.group + exclude, and runtests.jl GROUP comparison) so it is canonical Title-case and case-consistent. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs/ci: canonical sublib README headers + declare ShorthandConstructors test dep - Sublibrary READMEs: prepend the canonical OrdinaryDiffEq-style badge block (Zulip, Global Docs -> RecursiveArrayTools, ColPrac, SciML Code Style) and the "<Name>.jl is a component of the RecursiveArrayTools.jl monorepo. <role>. While completely independent and usable on its own, users wanting the full functionality should use RecursiveArrayTools.jl." wording, substituting names. Existing content preserved below. - Project.toml: declare RecursiveArrayToolsShorthandConstructors as a proper test dependency — added to [extras], [targets].test, [compat] ("1"), and a [sources] path entry so the Core test group resolves it from the in-repo lib/ on Julia >= 1.11 (runtests.jl Pkg.develop fallback still covers Julia < 1.11 where [sources] is ignored). Verified: project resolves and `VA[...]`/`AP[...]` from the sublibrary load in a Core-test-like environment. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: isolate root QA (Aqua) into canonical test/qa/ sub-environment Move the inline Aqua quality-assurance test out of the main test environment and into a dedicated test/qa/ project, matching the canonical OrdinaryDiffEq sublib layout (lib/*/test/qa/). - Add test/qa/Project.toml: Aqua + RecursiveArrayTools (via [sources]) + Pkg + Test, with [compat] for each dep and julia. - Move test/qa.jl -> test/qa/qa.jl (Aqua testset + the existing downstream-package manifest sentinel; behavior unchanged). - runtests.jl: add activate_qa_env() and, in the Core/All branch, activate the qa env, run the QA safetestset, then restore the main test project so the remaining Core functional tests keep their full dependency set. - Remove Aqua from the main Project.toml [extras], [targets].test, and [compat]; it now lives only in the qa env. JET (NoPre group) is already isolated in test/nopre/ and is left as-is. The three lib/* sublibraries run no QA today and are left untouched. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix: correct root test env — drop ShorthandConstructors [sources] and downstream-only NLsolve The branch had added RecursiveArrayToolsShorthandConstructors to the umbrella root [sources]. It is a test-only dependency (the VA[]/AP[] shorthand tests), not a dependency of the RecursiveArrayTools package itself, so sourcing it at the package level makes Pkg.test add it to the Core test env as a declared package dependency. Aqua.test_stale_deps then flags it as a stale dep of RecursiveArrayTools (the module never imports it), failing the QA group. master never had this [sources] entry: its runtests.jl Pkg.develops the sublibrary into the test env directly, which the current runtests.jl still does. Removing the [sources] entry restores master's green QA behavior (verified: Core+QA Pkg.test now passes — Quality Assurance 9 pass / 1 broken (expected ambiguities @test_broken) / 0 fail; previously 1 fail). Also drop NLsolve from the main [extras]/[targets].test/[compat]: it is used only by test/downstream/odesolve.jl and is already declared in test/downstream/Project.toml, so it does not belong in the base test env. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: fold bespoke GPU.yml into CI.yml [GPU] group; canonical group Project.tomls Replace the root Tests.yml + bespoke GPU.yml with a single canonical CI.yml (matching the OrdinaryDiffEq umbrella layout): - CI.yml carries the version matrix for the root base groups (Core/SymbolicIndexingInterface/Downstream/NoPre on [lts, 1, pre], NoPre excluded on pre) via SciML/.github tests.yml@v1. - The GPU tests become a dep-adding group on the self-hosted GPU runner: a dedicated CI.yml job passes group=GPU and runner=[self-hosted, Linux, X64, gpu] to tests.yml@v1 (which sets GROUP=GPU; runtests.jl then activates test/gpu and runs the GPU safetestsets). This reproduces the former GPU.yml (same runner labels, 60-min timeout, src,ext coverage) so GPU.yml is deleted. - Give the dep-adding root group Project.tomls the canonical [sources] = {path=../..} (+ sibling sublib paths) plus the package and Test deps so they resolve on Julia >= 1.11 against the PR-branch code: - test/gpu: add Adapt/ArrayInterface/RecursiveArrayTools/Test/Zygote + [sources] for the root and RecursiveArrayToolsArrayPartitionAnyAll. - test/nopre: add RecursiveArrayTools + [sources]. - test/downstream: add Test + [sources] for root and RecursiveArrayToolsShorthandConstructors. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: canonicalize test-group structure (standalone QA group, group folders) Make the root test layout match the OrdinaryDiffEq canonical pattern: every test lives in exactly one group, each group in its own capitalized test/<Group>/ folder, dep-adding groups have their own Project.toml (excluded from the light main env), and the version matrix is consistent between CI and test_groups.toml. - Split QA (Aqua) out of the Core branch into a standalone QA group keyed on GROUP == "QA" || GROUP == "All", running in its dep-adding test/QA env. Add a dedicated QA matrix row to CI.yml restricted to [lts, 1] (excluded on pre), so QA no longer runs inside the functional Core job. - Normalize sublib test_groups.toml [Core] versions ["lts","1.11","1","pre"] -> ["lts","1","pre"] for all three sublibraries. - Move the loose root Core test files into test/Core/, including the previously orphaned partitions_and_static_arrays.jl which was never wired into runtests and is now part of the Core group. Capitalize the remaining group folders (downstream->Downstream, gpu->GPU, nopre->NoPre, qa->QA) and route the SymbolicIndexingInterface and NoPre tests into their own group folders so all group folder names match their CI GROUP names. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: exclude standalone QA group from All QA is its own dep-adding test group (test/QA/Project.toml) and is run as a separate CI matrix entry, so it should not also fire under the "All" group. CI never runs "All", and a local "All" run should not pull the QA tooling into the functional test pass. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * ci: canonical TagBot thin-caller + downgrade-caller cleanup Replace TagBot.yml with the canonical SciML/.github tagbot.yml@v1 thin caller, plus a tagbot-subpackages matrix over the registered monorepo sublibraries (RecursiveArrayToolsArrayPartitionAnyAll, RecursiveArrayToolsRaggedArrays, RecursiveArrayToolsShorthandConstructors). Drop hand-maintained skip lists and pinned julia-version floors from the downgrade callers; the centralized workflow now auto-populates skip (stdlibs union in-repo sublibs union caller package) and defaults to the LTS floor. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Chris Rackauckas <accounts@chrisrackauckas.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 94cefea commit f5a9eb1

52 files changed

Lines changed: 1247 additions & 1107 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/Tests.yml renamed to .github/workflows/CI.yml

Lines changed: 24 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: "Tests"
1+
name: CI
22

33
on:
44
pull_request:
@@ -23,23 +23,40 @@ jobs:
2323
fail-fast: false
2424
matrix:
2525
version:
26-
- "1"
2726
- "lts"
27+
- "1"
2828
- "pre"
2929
group:
3030
- "Core"
31-
- "RaggedArrays"
32-
- "ArrayPartitionAnyAll"
33-
- "ShorthandConstructors"
31+
- "QA"
32+
- "SymbolicIndexingInterface"
3433
- "Downstream"
35-
- "nopre"
34+
- "NoPre"
3635
exclude:
36+
# JET (NoPre) is not run on prereleases.
3737
- version: "pre"
38-
group: "nopre"
38+
group: "NoPre"
39+
# QA (Aqua) runs only on lts and 1, never on prereleases.
40+
- version: "pre"
41+
group: "QA"
3942
uses: "SciML/.github/.github/workflows/tests.yml@v1"
4043
with:
4144
julia-version: "${{ matrix.version }}"
4245
group: "${{ matrix.group }}"
4346
num-threads: "2"
47+
check-bounds: "auto"
48+
coverage-directories: "src,ext"
49+
secrets: "inherit"
50+
51+
# GPU tests are a dep-adding group (test/GPU/Project.toml) that runs on the
52+
# self-hosted GPU runner. Folded in from the former bespoke GPU.yml.
53+
gpu:
54+
name: "GPU Tests"
55+
uses: "SciML/.github/.github/workflows/tests.yml@v1"
56+
with:
57+
julia-version: "1"
58+
group: "GPU"
59+
runner: '["self-hosted", "Linux", "X64", "gpu"]'
60+
timeout-minutes: 60
4461
coverage-directories: "src,ext"
4562
secrets: "inherit"

‎.github/workflows/Downgrade.yml‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,5 @@ jobs:
2020
- Core
2121
uses: "SciML/.github/.github/workflows/downgrade.yml@v1"
2222
with:
23-
julia-version: "1.10"
2423
group: "${{ matrix.group }}"
25-
skip: "Pkg,TOML"
2624
secrets: "inherit"

‎.github/workflows/DowngradeSublibraries.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,6 @@ jobs:
1919
secrets: "inherit"
2020
with:
2121
julia-version: "lts"
22-
skip: "RecursiveArrayTools, Pkg, TOML, Statistics, LinearAlgebra, SparseArrays, InteractiveUtils, Random, Test"
22+
group-env-name: "RECURSIVEARRAYTOOLS_TEST_GROUP"
23+
group-env-value: "Core"
24+
# Every lib/* sublibrary is downgrade-tested (projects auto-discovered, no exclusions).

‎.github/workflows/GPU.yml‎

Lines changed: 0 additions & 38 deletions
This file was deleted.

‎.github/workflows/SublibraryCI.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,7 @@ concurrency:
1919
jobs:
2020
sublibraries:
2121
uses: "SciML/.github/.github/workflows/sublibrary-project-tests.yml@v1"
22+
with:
23+
group-env-name: RECURSIVEARRAYTOOLS_TEST_GROUP
24+
check-bounds: auto
2225
secrets: "inherit"

‎.github/workflows/TagBot.yml‎

Lines changed: 17 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,21 @@
1-
name: TagBot
1+
name: "TagBot"
22
on:
33
issue_comment:
4-
types:
5-
- created
4+
types: [created]
65
workflow_dispatch:
76
jobs:
8-
TagBot:
9-
if: github.event_name == 'workflow_dispatch' || github.actor == 'JuliaTagBot'
10-
runs-on: ubuntu-latest
11-
steps:
12-
- uses: JuliaRegistries/TagBot@v1
13-
with:
14-
token: ${{ secrets.GITHUB_TOKEN }}
15-
ssh: ${{ secrets.DOCUMENTER_KEY }}
7+
tagbot:
8+
uses: "SciML/.github/.github/workflows/tagbot.yml@v1"
9+
secrets: "inherit"
10+
tagbot-subpackages:
11+
strategy:
12+
fail-fast: false
13+
matrix:
14+
subdir:
15+
- "lib/RecursiveArrayToolsArrayPartitionAnyAll"
16+
- "lib/RecursiveArrayToolsRaggedArrays"
17+
- "lib/RecursiveArrayToolsShorthandConstructors"
18+
uses: "SciML/.github/.github/workflows/tagbot.yml@v1"
19+
with:
20+
subdir: "${{ matrix.subdir }}"
21+
secrets: "inherit"

‎.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
11
*.jl.cov
22
*.jl.*.cov
33
*.jl.mem
4+
*.jl.*.mem
45
Manifest.toml
6+
*.DS_Store
57
.vscode
8+
profile.pb.gz
9+
.*.swp
10+
LocalPreferences.toml
11+
12+
docs/build
13+
.claude/

‎Project.toml‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,6 @@ RecursiveArrayToolsZygoteExt = "Zygote"
5050

5151
[compat]
5252
Adapt = "4"
53-
Aqua = "0.8"
5453
ArrayInterface = "7.17.0"
5554
CUDA = "5, 6.0"
5655
DocStringExtensions = "0.9.3"
@@ -62,12 +61,12 @@ LinearAlgebra = "1.10"
6261
Measurements = "2.11"
6362
MonteCarloMeasurements = "1.2"
6463
Mooncake = "0.5"
65-
NLsolve = "4.5"
6664
Pkg = "1"
6765
Polyester = "0.7.16"
6866
PrecompileTools = "1.2.1"
6967
Random = "1"
7068
RecipesBase = "1.3.4"
69+
RecursiveArrayToolsShorthandConstructors = "1"
7170
ReverseDiff = "1.15"
7271
SafeTestsets = "0.1"
7372
SparseArrays = "1.10"
@@ -84,17 +83,16 @@ Zygote = "0.7"
8483
julia = "1.10"
8584

8685
[extras]
87-
Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
8886
FastBroadcast = "7034ab61-46d4-4ed7-9d0f-46aef9175898"
8987
ForwardDiff = "f6369f11-7733-5829-9624-2563aa707210"
9088
KernelAbstractions = "63c18a36-062a-441e-b654-da1e3ab1ce7c"
9189
Measurements = "eff96d63-e80a-5855-80a2-b1b0885c5ab7"
9290
Mooncake = "da2b9cff-9c12-43a0-ae48-6db2b0edb7d6"
9391
MonteCarloMeasurements = "0987c9cc-fe09-11e8-30f0-b96dd679fdca"
94-
NLsolve = "2774e3e8-f4cf-5e23-947b-6d7e65073b56"
9592
Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f"
9693
Polyester = "f517fe37-dbe3-4b94-8317-1923a5111588"
9794
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
95+
RecursiveArrayToolsShorthandConstructors = "39fb7555-b4ad-4efd-8abe-30331df017d3"
9896
SafeTestsets = "1bc83da4-3b8d-516f-aca4-4fe02f6d838f"
9997
SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
10098
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
@@ -106,4 +104,4 @@ Unitful = "1986cc42-f94f-5a68-af5c-568840ba703d"
106104
Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
107105

108106
[targets]
109-
test = ["Aqua", "FastBroadcast", "ForwardDiff", "KernelAbstractions", "Measurements", "Mooncake", "NLsolve", "Pkg", "Polyester", "Random", "ReverseDiff", "SafeTestsets", "SparseArrays", "StaticArrays", "Statistics", "StructArrays", "Tables", "Test", "Unitful", "Zygote"]
107+
test = ["FastBroadcast", "ForwardDiff", "KernelAbstractions", "Measurements", "Mooncake", "Pkg", "Polyester", "Random", "RecursiveArrayToolsShorthandConstructors", "ReverseDiff", "SafeTestsets", "SparseArrays", "StaticArrays", "Statistics", "StructArrays", "Tables", "Test", "Unitful", "Zygote"]

‎lib/RecursiveArrayToolsArrayPartitionAnyAll/Project.toml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@ version = "1.0.0"
55
[deps]
66
RecursiveArrayTools = "731186ca-8d62-57ce-b412-fbd966d074cd"
77

8+
[sources]
9+
RecursiveArrayTools = {path = "../.."}
10+
811
[compat]
912
RecursiveArrayTools = "4"
1013
julia = "1.10"

‎lib/RecursiveArrayToolsArrayPartitionAnyAll/README.md‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,13 @@
11
# RecursiveArrayToolsArrayPartitionAnyAll.jl
22

3-
Optimized `any` and `all` for
4-
[RecursiveArrayTools.jl](https://github.com/SciML/RecursiveArrayTools.jl)'s
5-
`ArrayPartition` type.
3+
[![Join the chat at https://julialang.zulipchat.com #sciml-bridged](https://img.shields.io/static/v1?label=Zulip&message=chat&color=9558b2&labelColor=389826)](https://julialang.zulipchat.com/#narrow/stream/279055-sciml-bridged)
4+
[![Global Docs](https://img.shields.io/badge/docs-SciML-blue.svg)](https://docs.sciml.ai/RecursiveArrayTools/stable/)
5+
6+
[![ColPrac: Contributor's Guide on Collaborative Practices for Community Packages](https://img.shields.io/badge/ColPrac-Contributor%27s%20Guide-blueviolet)](https://github.com/SciML/ColPrac)
7+
[![SciML Code Style](https://img.shields.io/static/v1?label=code%20style&message=SciML&color=9558b2&labelColor=389826)](https://github.com/SciML/SciMLStyle)
8+
9+
RecursiveArrayToolsArrayPartitionAnyAll.jl is a component of the [RecursiveArrayTools.jl](https://github.com/SciML/RecursiveArrayTools.jl) monorepo. Optimized `any` and `all` for the `ArrayPartition` type.
10+
While completely independent and usable on its own, users wanting the full functionality should use [RecursiveArrayTools.jl](https://github.com/SciML/RecursiveArrayTools.jl).
611

712
`ArrayPartition` stores data as a tuple of arrays. The default `AbstractArray`
813
`any`/`all` iterates element-by-element through the partition, which incurs

0 commit comments

Comments
 (0)