Skip to content

Commit e668601

Browse files
jucorclaude
andcommitted
Remove dead xdist_group markers from make_dataset_params
pyproject.toml forces -n0 (sequential) because the session-scoped conversation cache makes sequential ~1.6-1.8x faster than parallel. The xdist_group markers were never active. Restore the plain pytest.param() calls that match the rest of the stack. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 447ab38 commit e668601

1 file changed

Lines changed: 5 additions & 17 deletions

File tree

delphi/tests/conftest.py

Lines changed: 5 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@
55
- Command line options --include-local and --datasets for dataset selection
66
- Fixtures for accessing dataset information
77
- @pytest.mark.use_discovered_datasets for dynamic dataset parametrization
8-
- Helper functions for parallel test execution with xdist_group markers
98
- require_dynamodb() and require_s3() helpers for failing fast when services are unavailable
109
- Session-scoped conversation cache for efficient test execution
1110
"""
@@ -148,36 +147,25 @@ def _get(dataset_name: str) -> dict:
148147

149148

150149
# =============================================================================
151-
# Parallel Execution Helpers
150+
# Dataset Parametrization Helpers
152151
# =============================================================================
153152

154153
def make_dataset_params(datasets: list[str]) -> list:
155154
"""
156-
Create pytest.param objects with xdist_group markers for parallel execution.
157-
158-
When using pytest-xdist with --dist=loadgroup, tests with the same
159-
xdist_group marker will run on the same worker. This ensures fixtures
160-
are computed only once per dataset per worker.
155+
Create pytest.param objects for dataset parametrization.
161156
162157
Args:
163158
datasets: List of dataset names (or "dataset-blob_type" composite IDs)
164159
165160
Returns:
166-
List of pytest.param objects with xdist_group markers
161+
List of pytest.param objects
167162
168163
Example:
169164
@pytest.mark.parametrize("dataset_name", make_dataset_params(["biodiversity", "vw"]))
170165
def test_something(dataset_name):
171166
...
172167
"""
173-
# Uses the full composite ID (e.g., 'biodiversity-incremental') as the group
174-
# key, so blob variants of the same dataset may land on different workers.
175-
# This is intentional: once incremental blob processing is implemented, each
176-
# variant will run a different computation, so cross-variant caching won't help.
177-
return [
178-
pytest.param(ds, marks=pytest.mark.xdist_group(ds))
179-
for ds in datasets
180-
]
168+
return [pytest.param(ds) for ds in datasets]
181169

182170

183171
def parse_dataset_blob_id(composite_id: str) -> tuple[str, str]:
@@ -272,7 +260,7 @@ def pytest_generate_tests(metafunc):
272260
With use_blobs=True, parametrize with 'dataset-blob_type' composite IDs
273261
(e.g., 'biodiversity-incremental', 'engage-cold_start') for each filled blob variant.
274262
275-
Uses xdist_group markers for efficient parallel execution with pytest-xdist.
263+
Uses the session-scoped conversation cache for efficient test execution.
276264
"""
277265
markers = list(metafunc.definition.iter_markers("use_discovered_datasets"))
278266
if not markers:

0 commit comments

Comments
 (0)