Skip to content

Commit 0af9ec8

Browse files
giordanoclaude
andcommitted
Add documentation for serial execution
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 46c1afc commit 0af9ec8

3 files changed

Lines changed: 59 additions & 2 deletions

File tree

docs/src/advanced.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,53 @@ end # hide
127127
```
128128
The `init_worker_code` is evaluated once per worker, so all definitions can be imported for use by the test module.
129129

130+
## Serial Tests
131+
132+
Some tests cannot safely run in parallel with other tests — for example, tests that allocate very large arrays and would exhaust memory if multiple ran simultaneously.
133+
The `serial` keyword argument to [`runtests`](@ref) lets you designate specific tests to run one at a time, while the remaining tests still run in parallel.
134+
135+
```@example mypackage
136+
using ParallelTestRunner
137+
using MyPackage
138+
139+
testsuite = Dict(
140+
"big_alloc" => quote
141+
# This test allocates ~4 GB and should not overlap with other tests
142+
@test true
143+
end,
144+
"huge_matrix" => quote
145+
@test true
146+
end,
147+
"fast_unit" => quote
148+
@test 1 + 1 == 2
149+
end,
150+
"fast_integration" => quote
151+
@test true
152+
end,
153+
)
154+
155+
# "big_alloc" and "huge_matrix" run one at a time; the rest run in parallel
156+
runtests(MyPackage, ARGS; testsuite, serial=["big_alloc", "huge_matrix"])
157+
```
158+
159+
By default serial tests run **before** the parallel batch.
160+
Use `serial_position=:after` to run them after instead:
161+
162+
```@example mypackage
163+
runtests(MyPackage, ARGS; testsuite, serial=["big_alloc", "huge_matrix"], serial_position=:after)
164+
```
165+
166+
Serial tests participate in the same ordering logic as parallel tests (sorted by historical
167+
duration, longest first) and their results appear in the same overall summary.
168+
169+
!!! tip
170+
With automatic test discovery via [`find_tests`](@ref), the `serial` names are the same
171+
keys that appear in the testsuite dictionary (e.g. `"subdir/memory_test"`).
172+
173+
!!! note
174+
If the user filters tests via positional arguments (e.g. `julia test/runtests.jl unit`),
175+
any serial test names that were filtered out are silently removed from the serial list.
176+
130177
## Custom Workers
131178

132179
For tests that require specific environment variables or Julia flags, you can use the `test_worker` keyword argument to [`runtests`](@ref) to assign tests to custom workers:
@@ -254,3 +301,5 @@ function jltest {
254301
Having few long-running test files and other short-running ones hinders scalability.
255302

256303
1. **Use custom workers sparingly**: Custom workers add overhead. Only use them when tests genuinely require different configurations.
304+
305+
1. **Use `serial` for resource-intensive tests**: If a test allocates significant memory or uses exclusive hardware resources, mark it as serial rather than reducing `--jobs` globally. This keeps the rest of your suite running in parallel.

docs/src/api.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,12 +39,13 @@ addworkers
3939
default_njobs
4040
```
4141

42-
## Internal Types
42+
## Internal Functionalities
4343

44-
These are internal types, not subject to semantic versioning contract (could be changed or removed at any point without notice), not intended for consumption by end-users.
44+
These are internal types or functions, not subject to semantic versioning contract (could be changed or removed at any point without notice), not intended for consumption by end-users.
4545
They are documented here exclusively for `ParallelTestRunner` developers and contributors.
4646

4747
```@docs
4848
ParsedArgs
4949
WorkerTestSet
50+
partition_tests
5051
```

docs/src/index.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,13 @@ Pkg.test("MyPackage"; test_args=`--verbose --jobs=4 integration`)
107107
Tests run concurrently in isolated worker processes, each inside own module.
108108
`ParallelTestRunner` records historical tests duration for each package, so that in subsequent runs long-running tests are executed first, to improve load balancing.
109109

110+
### Serial Test Support
111+
112+
Certain tests (e.g. memory-hungry tests) may need to run one at a time.
113+
The `serial` keyword argument to [`runtests`](@ref) lets you designate specific tests
114+
for sequential execution, either before or after the parallel batch.
115+
See [Serial Tests](@ref) in the advanced usage guide for details.
116+
110117
### Real-time Progress
111118

112119
The test runner provides real-time output showing:

0 commit comments

Comments
 (0)