Skip to content

Commit ff05f08

Browse files
committed
docs: update testing guide with parallel execution best practices
Update the testing guide to document the new parallel test execution strategy and best practices for maintaining test isolation. Major additions: - New "Parallel Test Execution" section explaining default behavior - Worker configuration and distribution strategies - Performance benchmarks (5 min parallel vs 15+ min sequential) - Coverage achievement documentation (86% achieved, exceeds 85% target) - When to use parallel vs sequential execution New "Test Isolation & Parallel Execution" section: - File-level isolation with loadfile strategy explained - Test independence requirements documented - Global state management patterns described - Debugging guidance for parallel execution issues Updated sections: - Quick Commands: Add parallel execution examples and timing info - Advanced Testing: Add parallel worker customization options - Coverage Goals: Update to show 86% achievement status - Command examples: Clarify parallel execution is default Updated command examples to reflect parallel-first approach: ```bash # Default parallel execution (recommended) uv run pytest # Sequential execution (for debugging) uv run pytest -n 0 # Customize workers uv run pytest -n 4 ``` The guide now reflects the production-ready test infrastructure with 100% test pass rate in parallel execution and provides clear guidance for developers on when and how to use different test execution modes. Developers can now: - Understand why tests run in parallel by default - Know when to use sequential execution for debugging - Customize worker count for their development environment - Understand test isolation requirements and patterns
1 parent af74243 commit ff05f08

1 file changed

Lines changed: 95 additions & 14 deletions

File tree

docs/testing-guide.md

Lines changed: 95 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -35,11 +35,12 @@ graph BT
3535

3636
### Coverage Goals
3737

38-
| Test Type | Target Coverage | Current Focus |
38+
| Test Type | Target Coverage | Current Status |
3939
| ----------- | --------------- | ------------------- |
40-
| Unit Tests | 80%+ | Core business logic |
41-
| Integration | 70%+ | Service boundaries |
40+
| Unit Tests | 80%+ | **86%** |
41+
| Integration | 70%+ | Core business logic |
4242
| E2E Tests | Critical paths | User workflows |
43+
| **Overall** | **85%+** | **86%**|
4344

4445
## 🛠️ Test Structure
4546

@@ -213,6 +214,31 @@ def test_client(test_server):
213214

214215
## 🚀 Running Tests
215216

217+
### Parallel Test Execution
218+
219+
**All tests run in parallel by default** using pytest-xdist with `loadfile` distribution strategy:
220+
221+
- **Default behavior**: `pytest` automatically runs with `-n auto --dist loadfile`
222+
- **Workers**: Auto-detected based on CPU cores (typically 8-10 workers)
223+
- **Distribution**: `loadfile` keeps all tests from the same file in the same worker for better isolation
224+
- **Performance**: Tests complete in ~5 minutes vs ~15+ minutes sequential
225+
- **Pass rate**: 100% (1,543 passed, 2 skipped)
226+
227+
```bash
228+
# Default parallel execution (recommended)
229+
uv run pytest
230+
231+
# Customize number of workers
232+
uv run pytest -n 4
233+
234+
# Sequential execution (slower, for debugging)
235+
uv run pytest -n 0
236+
237+
# Different distribution strategies
238+
uv run pytest --dist loadscope # Group by test scope
239+
uv run pytest --dist load # Round-robin (not recommended)
240+
```
241+
216242
### Test Execution Flow
217243

218244
```mermaid
@@ -255,24 +281,29 @@ flowchart TD
255281
### Quick Commands
256282

257283
```bash
258-
# Run all tests (excluding E2E)
284+
# Run all tests in parallel (default, ~5 minutes)
259285
just test
286+
# or
287+
uv run pytest
260288

261-
# Run with coverage
289+
# Run with coverage (parallel)
262290
just test-cov
263291

264-
# Run specific service tests
265-
uv run pytest tests/dashboard/ -v
266-
uv run pytest tests/extractor/ -v
292+
# Run specific service tests (parallel)
293+
uv run pytest tests/dashboard/
294+
uv run pytest tests/extractor/
267295

268-
# Run only unit tests
296+
# Run only unit tests (parallel)
269297
uv run pytest -m "not integration and not e2e"
270298

271-
# Run only integration tests
299+
# Run only integration tests (parallel)
272300
uv run pytest -m integration
273301

274302
# Run E2E tests
275303
just test-e2e
304+
305+
# Sequential execution (for debugging, ~15+ minutes)
306+
uv run pytest -n 0
276307
```
277308

278309
### Advanced Testing
@@ -281,15 +312,25 @@ just test-e2e
281312
# Run specific test
282313
uv run pytest tests/test_config.py::test_config_validation -v
283314

284-
# Run with debugging
285-
uv run pytest -xvs --tb=short
315+
# Run with debugging (sequential for better output)
316+
uv run pytest -n 0 -xvs --tb=short
286317

287-
# Run parallel
288-
uv run pytest -n auto
318+
# Customize parallel workers
319+
uv run pytest -n 4 # Use 4 workers
320+
uv run pytest -n 0 # Sequential execution
321+
322+
# Different distribution strategies
323+
uv run pytest --dist loadfile # Default: file-level isolation
324+
uv run pytest --dist loadscope # Scope-level isolation
325+
uv run pytest --dist load # Round-robin (faster but less isolation)
289326

290327
# Generate HTML coverage report
291328
uv run pytest --cov --cov-report=html
292329
open htmlcov/index.html
330+
331+
# Performance analysis
332+
uv run pytest --durations=10 # Show 10 slowest tests
333+
uv run pytest --durations=0 # Show all test durations
293334
```
294335

295336
## 🎭 E2E Testing with Playwright
@@ -432,6 +473,46 @@ class TestArtistProcessor:
432473
pass
433474
```
434475

476+
## 🔄 Test Isolation & Parallel Execution
477+
478+
### Best Practices
479+
480+
**File-Level Isolation (`loadfile` strategy)**:
481+
- All tests in the same file run in the same worker
482+
- Provides better isolation than round-robin (`load`)
483+
- Prevents cross-file global state pollution
484+
- Recommended for most use cases
485+
486+
**Test Independence**:
487+
- Each test should be fully independent
488+
- Global state is reset between tests via `conftest.py` fixtures
489+
- Module-level variables are reset automatically
490+
- Database connections and event loops are properly isolated
491+
492+
**When to Use Sequential Execution**:
493+
- Debugging specific test failures
494+
- Profiling test performance
495+
- Investigating race conditions
496+
- When you need predictable execution order
497+
498+
```bash
499+
# Debug a specific failing test
500+
uv run pytest -n 0 tests/test_specific.py::test_function -xvs
501+
502+
# Profile test performance
503+
uv run pytest -n 0 --durations=0
504+
```
505+
506+
### Global State Management
507+
508+
The test suite handles global state through `conftest.py` fixtures:
509+
510+
- **graphinator**: `shutdown_requested`, `message_counts`, `progress_interval`
511+
- **tableinator**: `connection_pool`, `message_counts`, `consumer_tags`
512+
- **extractor**: `shutdown_requested`
513+
514+
These are automatically reset before and after each test to ensure isolation.
515+
435516
## ⚠️ Common Pitfalls
436517

437518
### 1. Missing Async Markers

0 commit comments

Comments
 (0)