Skip to content

Commit 0aeb1c7

Browse files
committed
performance-test.yml: parallelise Connectivity Tests via pytest-xdist
The 5 connectivity test files in step #8 are pytest-style, integration-marked, share no in-process fixtures, and hit the live upstream which advertises max_concurrent=40 on /status. SOLR cache writes are idempotent, so concurrent cold workers on the same term_id produce identical writes — no corruption. Adds `pip install pytest-xdist` to the install step and `-n auto` to both initial and retry pytest invocations. GitHub-hosted ubuntu runners typically expose 2-4 cores, so wall time on this step drops to ~30-50% of the sequential run.
1 parent c9d29dc commit 0aeb1c7

1 file changed

Lines changed: 13 additions & 3 deletions

File tree

.github/workflows/performance-test.yml

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,10 @@ jobs:
3333
python -m pip install --upgrade pip
3434
python -m pip install --upgrade -r requirements.txt
3535
python -m pip install -e . # Editable install ensures we test the actual source code
36+
# pytest-xdist powers the `-n auto` parallel run in the
37+
# Connectivity Tests step below. Installed here rather than in
38+
# requirements.txt because it's only needed at test time.
39+
python -m pip install pytest-xdist
3640
3741
- name: Test Owlery Connectivity
3842
run: |
@@ -96,10 +100,16 @@ jobs:
96100
run: |
97101
# These files are pytest-style (plain classes + @pytest.mark.integration).
98102
# Run with pytest so the markers are honoured and collection works.
103+
# `-n auto` parallelises across all available CPU cores via
104+
# pytest-xdist (typically 2-4 on GitHub-hosted ubuntu runners). The
105+
# connectivity tests hit the live upstream and don't share fixtures
106+
# or in-process state, so they parallelise cleanly. SOLR cache writes
107+
# are idempotent so a race between two cold workers on the same
108+
# term_id just produces two identical writes.
99109
# Auto-retry once on failure — same rationale as Run Performance Test.
100110
set +e
101-
echo "=== Connectivity test attempt 1/2 ==="
102-
pytest -v \
111+
echo "=== Connectivity test attempt 1/2 (parallel) ==="
112+
pytest -v -n auto \
103113
src/test/test_neuron_neuron_connectivity.py \
104114
src/test/test_neuron_region_connectivity.py \
105115
src/test/test_upstream_class_connectivity.py \
@@ -113,7 +123,7 @@ jobs:
113123
echo ""
114124
echo "=== Connectivity attempt 1 failed (exit $FIRST_EXIT). Retrying once with warm cache. ==="
115125
echo "=== CONNECTIVITY RETRY ATTEMPT ===" >> performance_test_output.log
116-
pytest -v \
126+
pytest -v -n auto \
117127
src/test/test_neuron_neuron_connectivity.py \
118128
src/test/test_neuron_region_connectivity.py \
119129
src/test/test_upstream_class_connectivity.py \

0 commit comments

Comments
 (0)