Skip to content

Commit 0370c36

Browse files
gmoonclaude
andcommitted
docs: Add comprehensive test types and failure handling strategies
- Expanded test types table: unit, integration, E2E, concurrency, antagonistic, fuzz/property, load/performance - Added fail-open vs fail-closed guidance with examples - Decision table for when to use each failure strategy - Mentioned circuit breaker pattern for hybrid approach Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 5560d2a commit 0370c36

1 file changed

Lines changed: 51 additions & 12 deletions

File tree

CONTRIBUTING.md

Lines changed: 51 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -107,18 +107,17 @@ npm run lint -- --fix # Auto-fix
107107
| Backend | 70% | 85% |
108108
| Frontend | 20% | 60% |
109109

110-
### Test Categories
111-
112-
**Unit Tests** (fast, isolated):
113-
```bash
114-
make test-backend # Backend unit tests
115-
make test-frontend # Frontend unit tests
116-
```
117-
118-
**E2E Tests** (slow, integrated):
119-
```bash
120-
make test-e2e # Full browser tests with server lifecycle
121-
```
110+
### Test Types
111+
112+
| Type | Purpose | When to Write | Command |
113+
|------|---------|---------------|---------|
114+
| **Unit** | Test functions in isolation | Core business logic, utilities | `make test-backend` |
115+
| **Integration** | Test components together (API + DB) | API endpoints, service layers | `make test-backend` |
116+
| **E2E** | Browser tests simulating users | Critical user flows | `make test-e2e` |
117+
| **Concurrency** | Race conditions, thread safety | Shared resources, parallel ops | `pytest tests/test_concurrency.py` |
118+
| **Antagonistic** | Behavior when dependencies fail | External APIs, DB, network | See guidance below |
119+
| **Fuzz/Property** | Random inputs to find edge cases | Input parsing, validation | Use `hypothesis` library |
120+
| **Load/Performance** | Response times under stress | Before production release | Use `locust` or `k6` |
122121

123122
### What to Test
124123

@@ -133,6 +132,46 @@ make test-e2e # Full browser tests with server lifecycle
133132
- Simple getters/setters
134133
- Framework internals
135134

135+
### Antagonistic Tests (Failure Handling)
136+
137+
Test how your system behaves when dependencies fail. Choose the right failure strategy:
138+
139+
**Fail-Closed** (stop and return error) - Use for security-critical operations:
140+
```python
141+
def test_auth_service_down_rejects_requests():
142+
"""When auth fails, reject all requests (fail-closed)."""
143+
with mock.patch('app.core.auth.verify_token', side_effect=ConnectionError):
144+
response = client.get("/api/protected")
145+
assert response.status_code == 503 # Service unavailable, not 200
146+
```
147+
148+
**Fail-Open** (continue with degraded functionality) - Use for non-critical features:
149+
```python
150+
def test_analytics_down_continues_request():
151+
"""When analytics fails, request still succeeds (fail-open)."""
152+
with mock.patch('app.services.analytics.track', side_effect=ConnectionError):
153+
response = client.post("/api/action")
154+
assert response.status_code == 200 # Action succeeds despite analytics failure
155+
```
156+
157+
**When to use each:**
158+
159+
| Scenario | Strategy | Rationale |
160+
|----------|----------|-----------|
161+
| Auth/permission check fails | **Fail-closed** | Never grant access on failure |
162+
| Payment processor fails | **Fail-closed** | Don't complete transaction |
163+
| Rate limiter fails | **Fail-closed** | Prevent abuse |
164+
| Analytics/tracking fails | **Fail-open** | Non-critical, don't block user |
165+
| Recommendation engine fails | **Fail-open** | Show defaults instead |
166+
| Cache fails | **Fail-open** | Fall back to database |
167+
| Core feature dependency fails | **Fail-closed** | Can't provide degraded version |
168+
169+
**Hybrid: Circuit Breaker Pattern**
170+
```python
171+
# After N failures, stop trying and return cached/default response
172+
# Periodically retry to see if service recovered
173+
```
174+
136175
### Test Naming
137176

138177
```python

0 commit comments

Comments
 (0)