Skip to content

Commit 453a7b5

Browse files
committed
perf-test: parallelise step #6 + register pytest integration marker
1. .github/workflows/performance-test.yml - Run Performance Test (step #6): `python -m unittest …` (single-threaded) → `pytest -v -n auto …`. pytest discovers unittest.TestCase subclasses automatically and pytest-xdist parallelises them across the runner's vCPUs. No test refactoring needed. Expected ~3 min → ~1.5 min. - Run Connectivity Tests (step #8): xdist `-n auto` (~4 workers on ubuntu-latest) → `-n 8`. These tests are I/O-bound on Neo4j round-trips; oversubscribing CPU cores fills network-wait time. Upstream advertises max_concurrent=40 on /status — 8 workers stays well within budget. Expected ~7 min → ~4 min. 2. pyproject.toml - Add [tool.pytest.ini_options] markers = [...] declaring the `integration` marker that the connectivity test files already decorate with. Stops the `PytestUnknownMarkWarning` per-occurrence log spam that was bloating performance.md (each unregistered mark was emitting four-plus warning blocks per xdist worker per file).
1 parent dae1ae3 commit 453a7b5

2 files changed

Lines changed: 24 additions & 15 deletions

File tree

.github/workflows/performance-test.yml

Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -47,27 +47,27 @@ jobs:
4747
curl -v --max-time 10 "http://owl.virtualflybrain.org/kbs/vfb" || echo "Server unreachable"
4848
4949
- name: Run Performance Test
50-
# Auto-retry once on failure: cold-start runs hit Neo4j + SOLR with
51-
# empty result caches and frequently breach THRESHOLD_VERY_SLOW
52-
# purely on first-call latency. The retry runs with the cache the
53-
# first attempt warmed; flakes-from-cold-start no longer mask real
54-
# regressions. If the second attempt also fails, surface the failure.
55-
# IMPORTANT: the retry OVERWRITES performance_test_output.log so that
56-
# the downstream "Fail job on test failures" step grades on the
57-
# second attempt's output only — otherwise attempt 1's `FAIL:` lines
58-
# leak through and turn a green retry red.
50+
# Switched from `python -m unittest` (single-threaded) to `pytest -n auto`
51+
# so the suite parallelises across the runner's vCPUs. pytest discovers
52+
# unittest.TestCase subclasses automatically; no test refactoring needed.
53+
# Auto-retry once on failure: cold-start runs hit Neo4j + SOLR with empty
54+
# result caches and can breach thresholds on first-call latency. The
55+
# retry runs warm. If both fail, surface the failure.
56+
# IMPORTANT: the retry OVERWRITES performance_test_output.log so the
57+
# downstream "Fail job on test failures" step grades on the second
58+
# attempt's output only.
5959
run: |
6060
set +e
61-
echo "=== Performance test attempt 1/2 ==="
62-
python -m unittest src.test.test_query_performance -v 2>&1 | tee performance_test_output.log
61+
echo "=== Performance test attempt 1/2 (parallel) ==="
62+
pytest -v -n auto src/test/test_query_performance.py 2>&1 | tee performance_test_output.log
6363
FIRST_EXIT=${PIPESTATUS[0]}
6464
if [[ $FIRST_EXIT -eq 0 ]]; then
6565
exit 0
6666
fi
6767
echo ""
6868
echo "=== Attempt 1 failed (exit $FIRST_EXIT). Retrying once with warm cache. ==="
6969
echo "=== RETRY ATTEMPT (auto-retry on first failure) ===" > performance_test_output.log
70-
python -m unittest src.test.test_query_performance -v 2>&1 | tee -a performance_test_output.log
70+
pytest -v -n auto src/test/test_query_performance.py 2>&1 | tee -a performance_test_output.log
7171
exit ${PIPESTATUS[0]}
7272
7373
- name: Run Legacy Performance Test
@@ -123,7 +123,7 @@ jobs:
123123
# grades on the last attempt only.
124124
set +e
125125
echo "=== Connectivity test attempt 1/2 (parallel) ==="
126-
pytest -v -n auto \
126+
pytest -v -n 8 \
127127
src/test/test_neuron_neuron_connectivity.py \
128128
src/test/test_neuron_region_connectivity.py \
129129
src/test/test_upstream_class_connectivity.py \
@@ -137,7 +137,7 @@ jobs:
137137
fi
138138
echo ""
139139
echo "=== Connectivity attempt 1 failed (exit $FIRST_EXIT). Retrying once with warm cache. ==="
140-
pytest -v -n auto \
140+
pytest -v -n 8 \
141141
src/test/test_neuron_neuron_connectivity.py \
142142
src/test/test_neuron_region_connectivity.py \
143143
src/test/test_upstream_class_connectivity.py \

pyproject.toml

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,13 @@ requires = [
33
"setuptools>=45,<69",
44
"wheel"
55
]
6-
build-backend = "setuptools.build_meta"
6+
build-backend = "setuptools.build_meta"
7+
8+
[tool.pytest.ini_options]
9+
# Register custom marks so the connectivity test files' `@pytest.mark.integration`
10+
# decorations stop logging `PytestUnknownMarkWarning` per occurrence per
11+
# xdist worker. Each unregistered mark was emitting four-plus warning
12+
# blocks in performance.md — pure noise.
13+
markers = [
14+
"integration: integration tests that exercise the live VFB upstream (Neo4j, SOLR, owlery). Skip via -m 'not integration'.",
15+
]

0 commit comments

Comments
 (0)