|
| 1 | +# MatrixAlgebraKit.jl test suite |
| 2 | + |
| 3 | +Tests are driven by [ParallelTestRunner.jl](https://github.com/JuliaTesting/ParallelTestRunner.jl). |
| 4 | +Every `.jl` file under `test/` is auto-discovered and executed in its own worker process, so test |
| 5 | +files share no state and must be self-contained. |
| 6 | + |
| 7 | +The test environment is a [Pkg workspace](https://pkgdocs.julialang.org/dev/workspaces/) member: |
| 8 | +`test/Project.toml` declares the test dependencies and resolves the parent package through |
| 9 | +`[sources]`, while the whole workspace shares a single `Manifest.toml` at the repository root. |
| 10 | + |
| 11 | +## Running tests |
| 12 | + |
| 13 | +```julia |
| 14 | +# Standard — works on all supported Julia versions |
| 15 | +using Pkg; Pkg.test() |
| 16 | + |
| 17 | +# With arguments (see "Selecting tests" below) |
| 18 | +using Pkg; Pkg.test("MatrixAlgebraKit"; test_args = ["decompositions", "--fast"]) |
| 19 | +``` |
| 20 | + |
| 21 | +```bash |
| 22 | +# Direct invocation — requires Julia 1.12+ (workspace support) |
| 23 | +julia --project=test test/runtests.jl |
| 24 | +``` |
| 25 | + |
| 26 | +The direct form is the convenient one for local iteration: it reuses the already-instantiated |
| 27 | +workspace environment instead of building a fresh temporary one on every run. |
| 28 | + |
| 29 | +## Selecting tests |
| 30 | + |
| 31 | +Each discovered test is keyed by its path relative to `test/`, with the `.jl` extension stripped — |
| 32 | +for example `decompositions/svd`, `common/truncate`, `mooncake/qr`. Positional arguments are matched |
| 33 | +against those keys with `startswith`, so a directory name selects a whole group and a longer path |
| 34 | +selects a single file: |
| 35 | + |
| 36 | +```bash |
| 37 | +julia --project=test test/runtests.jl --list # print all discovered test keys |
| 38 | + |
| 39 | +julia --project=test test/runtests.jl decompositions # one group |
| 40 | +julia --project=test test/runtests.jl decompositions/svd # one file |
| 41 | +julia --project=test test/runtests.jl mooncake enzyme # union of several selectors |
| 42 | +``` |
| 43 | + |
| 44 | +Available flags: |
| 45 | + |
| 46 | +| Flag | Effect | |
| 47 | +|------|--------| |
| 48 | +| `--list` | List all discovered tests and exit | |
| 49 | +| `--fast` | Reduced test matrix (see below) | |
| 50 | +| `--jobs=N` | Use `N` worker processes (default: derived from CPU count and available memory) | |
| 51 | +| `--verbose` | More detailed progress output | |
| 52 | +| `--quickfail` | Abort the whole run as soon as one test errors | |
| 53 | +| `--help` | Usage information | |
| 54 | + |
| 55 | +Note that passing an explicit selector **disables** the automatic OS- and CI-based filtering in |
| 56 | +`runtests.jl` — `filter_tests!` returns `false` once positional arguments are present, so you get |
| 57 | +exactly the tests you asked for. |
| 58 | + |
| 59 | +### Fast mode (`--fast`) |
| 60 | + |
| 61 | +Sets a `fast_tests = true` constant in every test sandbox. Files branch on |
| 62 | +`@isdefined(fast_tests) && fast_tests` to shrink their element-type matrix, typically from |
| 63 | +`(Float32, Float64, ComplexF32, ComplexF64)` plus generic floats down to `(Float64, ComplexF64)` and |
| 64 | +one generic float. CI uses this for draft pull requests. |
| 65 | + |
| 66 | +## Test groups |
| 67 | + |
| 68 | +The top-level directories under `test/` are the test groups, and they are exactly the matrix jobs of |
| 69 | +the CI workflow — `.github/workflows/Tests.yml` delegates to |
| 70 | +`QuantumKitHub/QuantumKitHubActions/.github/workflows/TestGroups.yml`, which discovers groups by |
| 71 | +listing those directories. Adding a new directory therefore adds a new CI job automatically; a test |
| 72 | +file placed directly in `test/` would belong to no group and never run in CI. |
| 73 | + |
| 74 | +| Group | Contents | |
| 75 | +|-------|----------| |
| 76 | +| `common` | Algorithm selection and defaults, truncation strategies, projections, matrix exponential, Aqua code-quality checks | |
| 77 | +| `decompositions` | `qr`, `lq`, `svd`, `eig`, `eigh`, `gen_eig`, `schur`, `polar`, `orthnull` on CPU and GPU array types | |
| 78 | +| `chainrules` | ChainRulesCore rules, exercised through ChainRulesTestUtils and Zygote | |
| 79 | +| `mooncake` | Mooncake AD rules | |
| 80 | +| `enzyme` | Enzyme AD rules, exercised through EnzymeTestUtils | |
| 81 | +| `genericlinearalgebra` | `MatrixAlgebraKitGenericLinearAlgebraExt` | |
| 82 | +| `genericschur` | `MatrixAlgebraKitGenericSchurExt` | |
| 83 | + |
| 84 | +Two directories are *not* groups: `testsuite/` holds the shared implementation described below and |
| 85 | +is excluded both from discovery (in `runtests.jl`) and from CI (`exclude: '["testsuite"]'`). |
| 86 | + |
| 87 | +## `testsuite/` — the shared test suite |
| 88 | + |
| 89 | +`test/testsuite/TestSuite.jl` defines the `TestSuite` module, a backend-parametric suite modelled on |
| 90 | +GPUArrays.jl and intended to be reusable by packages that build on MatrixAlgebraKit. It contains the |
| 91 | +actual assertions; the files under the group directories are thin drivers that pick element types |
| 92 | +and array types and call into it. |
| 93 | + |
| 94 | +Because it is excluded from discovery, it is loaded explicitly by each test file that needs it: |
| 95 | + |
| 96 | +```julia |
| 97 | +@isdefined(TestSuite) || include("../testsuite/TestSuite.jl") |
| 98 | +using .TestSuite |
| 99 | +``` |
| 100 | + |
| 101 | +The suite exposes `test_*` entry points, one per operation, taking a *matrix type* and a size: |
| 102 | + |
| 103 | +```julia |
| 104 | +TestSuite.test_qr(Matrix{Float64}, (54, 37)) |
| 105 | +TestSuite.test_qr(CuMatrix{ComplexF64}, (54, 37); test_pivoted = false, test_blocksize = false) |
| 106 | +TestSuite.test_qr(Diagonal{Float64, CuVector{Float64}}, 54) |
| 107 | +TestSuite.test_qr_algs(Matrix{Float64}, (54, 37), (Householder(; positive = true),)) |
| 108 | +``` |
| 109 | + |
| 110 | +The AD entry points (`test_chainrules`, `test_mooncake_*`, `test_enzyme_*`) instead take a scalar |
| 111 | +element type and a size, plus `atol`/`rtol` keywords: |
| 112 | + |
| 113 | +```julia |
| 114 | +TestSuite.test_chainrules(ComplexF64, (19, 17); atol = tol, rtol = tol) |
| 115 | +``` |
| 116 | + |
| 117 | +Supporting infrastructure in the module: |
| 118 | + |
| 119 | +- `instantiate_matrix(AT, size)` — builds a random matrix of the requested type, with methods for |
| 120 | + `Array`, `CuArray`, `ROCArray`, `Diagonal`, and `Diagonal` wrapping GPU vectors. This is the single |
| 121 | + hook a new backend needs to implement. |
| 122 | +- `rng` / `seed_rng!(seed)` — a shared `StableRNG(123)`. Drivers call `TestSuite.seed_rng!(123)` |
| 123 | + before each parameter combination so results are reproducible independent of test ordering. |
| 124 | +- `precision(T)` — the default tolerance, `sqrt(eps(real(T)))`. |
| 125 | +- Predicates used throughout the assertions: `isleftnull`, `isrightnull`, `isleftcomplete`, |
| 126 | + `isrightcomplete`, `has_positive_diagonal`. |
| 127 | +- `instantiate_unitary`, `instantiate_rank_deficient_matrix` — inputs with prescribed structure. |
| 128 | + |
| 129 | +`test/linearmap.jl` is a further helper, defining a `LinearMap` wrapper that is deliberately *not* an |
| 130 | +`AbstractMatrix`, used to check the generic code paths. It is excluded from discovery and included |
| 131 | +directly where needed. |
| 132 | + |
| 133 | +## Hardware and platform gating |
| 134 | + |
| 135 | +GPU tests are guarded at runtime rather than by dependency availability: CUDA and AMDGPU are |
| 136 | +unconditional dependencies of the test environment, and the tests themselves are wrapped in |
| 137 | +`CUDA.functional()` / `AMDGPU.functional()` blocks. Running the suite on a machine without a GPU |
| 138 | +therefore skips them silently. |
| 139 | + |
| 140 | +`runtests.jl` applies two further filters, both only when no explicit selector was given: |
| 141 | + |
| 142 | +- **`BUILDKITE=true`** (the GPU CI runners) drops the files that carry no GPU coverage — |
| 143 | + `common/algorithms`, `common/codequality`, `common/truncate`, `decompositions/gen_eig`, and the |
| 144 | + `chainrules` group — so GPU runners spend their time on GPU code paths. |
| 145 | +- **macOS CI** drops the `mooncake` and `chainrules` groups; **macOS and Windows CI** drop the |
| 146 | + `enzyme` group. These reflect AD backends that are unsupported or unreliable on those platforms. |
| 147 | + |
| 148 | +## Adding a test |
| 149 | + |
| 150 | +Create a `.jl` file inside one of the group directories. It is picked up automatically, runs in a |
| 151 | +fresh worker process, and must therefore load everything it needs itself: |
| 152 | + |
| 153 | +```julia |
| 154 | +using MatrixAlgebraKit |
| 155 | +using Test |
| 156 | +using CUDA, AMDGPU |
| 157 | + |
| 158 | +@isdefined(TestSuite) || include("../testsuite/TestSuite.jl") |
| 159 | +using .TestSuite |
| 160 | + |
| 161 | +if @isdefined(fast_tests) && fast_tests |
| 162 | + eltypes = (Float64, ComplexF64) |
| 163 | +else |
| 164 | + eltypes = (Float32, Float64, ComplexF32, ComplexF64) |
| 165 | +end |
| 166 | + |
| 167 | +for T in eltypes |
| 168 | + TestSuite.seed_rng!(123) |
| 169 | + TestSuite.test_myop(Matrix{T}, (54, 37)) |
| 170 | + CUDA.functional() && TestSuite.test_myop(CuMatrix{T}, (54, 37)) |
| 171 | +end |
| 172 | +``` |
| 173 | + |
| 174 | +Add assertions that should be shared with downstream packages to `test/testsuite/` and call them |
| 175 | +from the driver, rather than writing them inline. |
0 commit comments