Skip to content

Commit f5d60f9

Browse files
authored
Improve e2e test suite (#12)
1 parent 9be3dd3 commit f5d60f9

63 files changed

Lines changed: 2452 additions & 1209 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.dockerignore

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
# Exclude files from Docker build context
2+
# Reduces image size and build time for e2e tests
3+
4+
# Version control
5+
.git/
6+
.gitignore
7+
.github/
8+
9+
# Documentation (not needed for runtime)
10+
*.md
11+
docs/
12+
LICENSE
13+
14+
# Python bytecode and caches
15+
__pycache__/
16+
*.py[codz]
17+
*$py.class
18+
*.so
19+
20+
# Build artifacts
21+
build/
22+
dist/
23+
*.egg-info/
24+
.eggs/
25+
eggs/
26+
develop-eggs/
27+
wheels/
28+
*.egg
29+
MANIFEST
30+
31+
# Virtual environments
32+
.venv/
33+
venv/
34+
env/
35+
ENV/
36+
.pixi/
37+
38+
# Testing artifacts
39+
.tox/
40+
.nox/
41+
.pytest_cache/
42+
.hypothesis/
43+
.coverage
44+
.coverage.*
45+
htmlcov/
46+
coverage.xml
47+
*.cover
48+
nosetests.xml
49+
50+
# Type checking and linting caches
51+
.mypy_cache/
52+
.dmypy.json
53+
.pytype/
54+
.pyre/
55+
.ruff_cache/
56+
57+
# IDE and editor settings
58+
.vscode/
59+
.idea/
60+
*.swp
61+
*.swo
62+
*~
63+
64+
# Tusk test artifacts (traces/logs from previous runs)
65+
**/.tusk/traces/
66+
**/.tusk/logs/
67+
68+
# macOS
69+
.DS_Store
70+
71+
# Misc
72+
*.log
73+
*.pid
74+
*.rdb
75+
*.aof
76+
.env
77+
.direnv

.gitignore

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -215,7 +215,10 @@ __marimo__/
215215
.streamlit/secrets.toml
216216

217217
.direnv
218-
.tusk/traces
218+
219+
# Tusk traces and logs (but keep config.yaml)
220+
**/.tusk/traces/
221+
**/.tusk/logs/
219222

220223
# macOS
221224
.DS_Store

CONTRIBUTING.md

Lines changed: 94 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -60,9 +60,100 @@ uv run python -m unittest tests.integration.test_database -v
6060
docker compose -f docker-compose.test.yml down
6161
```
6262

63-
### Demo Scripts
63+
### E2E Tests
64+
65+
E2E tests validate full instrumentation workflows using Docker containers. They record real API interactions and verify replay behavior using the Tusk CLI.
66+
67+
#### Prerequisites
68+
69+
1. Build the base Docker image (required before running any e2e test):
70+
71+
```bash
72+
docker build -t python-e2e-base:latest -f drift/instrumentation/e2e_common/Dockerfile.base .
73+
```
74+
75+
2. Docker and Docker Compose must be installed.
76+
77+
#### Running E2E Tests
78+
79+
Run all e2e tests:
80+
81+
```bash
82+
./run-all-e2e-tests.sh # Sequential (default)
83+
./run-all-e2e-tests.sh 2 # 2 tests in parallel
84+
./run-all-e2e-tests.sh 0 # All tests in parallel
85+
```
86+
87+
Run a single instrumentation's e2e test:
6488
6589
```bash
66-
timeout 10 uv run python tests/test_flask_demo.py
67-
timeout 10 uv run python tests/test_fastapi_demo.py
90+
cd drift/instrumentation/flask/e2e-tests
91+
./run.sh
92+
93+
# Or with a custom port:
94+
./run.sh 8001
6895
```
96+
97+
#### Available E2E Tests
98+
99+
| Instrumentation | Location | Services |
100+
|-----------------|----------|----------|
101+
| Flask | `drift/instrumentation/flask/e2e-tests/` | None (external APIs) |
102+
| FastAPI | `drift/instrumentation/fastapi/e2e-tests/` | None (external APIs) |
103+
| Django | `drift/instrumentation/django/e2e-tests/` | None (external APIs) |
104+
| Redis | `drift/instrumentation/redis/e2e-tests/` | Redis 7 |
105+
| Psycopg | `drift/instrumentation/psycopg/e2e-tests/` | PostgreSQL 13 |
106+
| Psycopg2 | `drift/instrumentation/psycopg2/e2e-tests/` | PostgreSQL 13 |
107+
108+
#### E2E Test Structure
109+
110+
Each e2e test directory contains:
111+
112+
```text
113+
e2e-tests/
114+
├── Dockerfile # Builds on python-e2e-base
115+
├── docker-compose.yml # Service orchestration
116+
├── run.sh # External runner script
117+
├── entrypoint.py # Test orchestrator (setup → record → test)
118+
├── requirements.txt # Python dependencies
119+
├── .tusk/config.yaml # Tusk CLI configuration
120+
└── src/
121+
├── app.py # Test application
122+
└── test_requests.py # HTTP request script
123+
```
124+
125+
#### Debugging E2E Tests
126+
127+
View traces after a test:
128+
129+
```bash
130+
cd drift/instrumentation/flask/e2e-tests
131+
# JSONL files contain one JSON object per line, use jq to format them
132+
cat .tusk/traces/*.jsonl | jq .
133+
```
134+
135+
View service logs:
136+
137+
```bash
138+
cat .tusk/logs/*
139+
```
140+
141+
Run test app locally (outside Docker):
142+
143+
```bash
144+
cd drift/instrumentation/flask/e2e-tests
145+
pip install -r requirements.txt
146+
TUSK_DRIFT_MODE=RECORD python src/app.py
147+
148+
# In another terminal:
149+
python src/test_requests.py
150+
```
151+
152+
For more details, see `drift/instrumentation/README-e2e-tests.md`.
153+
154+
## Documentation
155+
156+
| Document | Description |
157+
|----------|-------------|
158+
| `docs/context-propagation.md` | Context propagation behavior, edge cases, and patterns |
159+
| `drift/instrumentation/README-e2e-tests.md` | E2E test architecture and debugging |

docs/context-propagation.md

Lines changed: 133 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,133 @@
1+
# Context Propagation in Python
2+
3+
This document covers how the Drift Python SDK handles tracing context propagation across different execution contexts, including edge cases and recommended patterns.
4+
5+
## Overview
6+
7+
The SDK uses OpenTelemetry for distributed tracing, which relies on Python's `contextvars` module for context propagation. Understanding when context propagates automatically vs. when it requires explicit handling is crucial for correct trace hierarchies.
8+
9+
## Context Propagation Behavior
10+
11+
| Scenario | Auto-propagates? | Notes |
12+
|----------|------------------|-------|
13+
| `async/await` chains | ✅ Yes | Native `contextvars` support |
14+
| `ThreadPoolExecutor` | ❌ No | Requires explicit propagation |
15+
| `ProcessPoolExecutor` | ❌ No | Context cannot cross process boundaries |
16+
| `asyncio.run_in_executor()` | ❌ No | Same as ThreadPoolExecutor |
17+
| `asyncio.to_thread()` (Python 3.9+) | ✅ Yes | Recommended for blocking calls |
18+
| Callback-based libraries | ❌ No | Context lost when callback executes |
19+
20+
## Stack Trace Capture
21+
22+
The SDK captures stack traces for debugging and mock matching. Different components use different truncation levels:
23+
24+
| Component | Max Frames | Use Case |
25+
|-----------|------------|----------|
26+
| Socket instrumentation (unpatched alerts) | Unlimited | Full debugging info |
27+
| `SpanUtils.capture_stack_trace()` | 10 (default) | Span metadata |
28+
| Communicator debug traces | 20 | Internal debugging |
29+
30+
## ThreadPoolExecutor Pattern
31+
32+
Context does **not** automatically propagate to thread pool workers. Use the explicit propagation pattern:
33+
34+
```python
35+
from opentelemetry import context as otel_context
36+
37+
def _run_with_context(ctx, fn, *args, **kwargs):
38+
"""Run function with OpenTelemetry context in a thread pool."""
39+
token = otel_context.attach(ctx)
40+
try:
41+
return fn(*args, **kwargs)
42+
finally:
43+
otel_context.detach(token)
44+
45+
# Usage
46+
ctx = otel_context.get_current()
47+
with ThreadPoolExecutor(max_workers=4) as executor:
48+
future = executor.submit(_run_with_context, ctx, my_function, arg1)
49+
```
50+
51+
### Alternative: asyncio.to_thread() (Python 3.9+)
52+
53+
For async code needing to run blocking operations, `asyncio.to_thread()` automatically propagates context:
54+
55+
```python
56+
# Context propagates automatically - no wrapper needed
57+
result = await asyncio.to_thread(blocking_function, arg1)
58+
```
59+
60+
## Possible SDK-Level Solutions
61+
62+
### Option 1: Manual Helper (Current Approach)
63+
64+
**What:** Provide documented `_run_with_context()` pattern.
65+
66+
**Pros:** Explicit, no magic, works everywhere
67+
**Cons:** Requires user code changes
68+
69+
### Option 2: ContextAwareThreadPoolExecutor
70+
71+
**What:** SDK provides a drop-in executor that auto-propagates context.
72+
73+
```python
74+
from concurrent.futures import ThreadPoolExecutor
75+
import contextvars
76+
77+
class ContextAwareThreadPoolExecutor(ThreadPoolExecutor):
78+
def submit(self, fn, *args, **kwargs):
79+
ctx = contextvars.copy_context()
80+
return super().submit(ctx.run, fn, *args, **kwargs)
81+
```
82+
83+
**Pros:** Clean API, opt-in
84+
**Cons:** User must change imports
85+
86+
### Option 3: Monkey-patch ThreadPoolExecutor
87+
88+
**What:** SDK patches `ThreadPoolExecutor.submit()` globally at initialization.
89+
90+
**Pros:** Zero user code changes
91+
**Cons:**
92+
93+
- High risk of breaking other libraries
94+
- Hidden global side effects
95+
- Performance overhead for all executors (even unrelated ones)
96+
- Debugging becomes harder
97+
98+
**Recommendation:** Not recommended for tracing SDKs.
99+
100+
## Comparison with Node.js SDK
101+
102+
| Aspect | Python | Node.js |
103+
|--------|--------|---------|
104+
| Async context mechanism | `contextvars` (native) | `AsyncLocalStorage` via OpenTelemetry |
105+
| `async/await` propagation | ✅ Automatic | ❌ Requires `context.with()` |
106+
| Thread pools | ❌ Manual propagation | N/A (single-threaded) |
107+
| Callbacks | ❌ Context lost | ❌ Requires `context.bind()` |
108+
109+
Python's native `contextvars` makes async code simpler—no explicit binding needed for `await` chains. However, thread pools and callbacks still require explicit handling in both languages.
110+
111+
## Testing Context Propagation
112+
113+
The FastAPI e2e tests include endpoints that verify context propagation:
114+
115+
- `GET /api/test-async-context` - Verifies context across concurrent async calls
116+
- `GET /api/test-thread-context` - Verifies explicit thread pool propagation
117+
118+
Run the e2e tests to validate:
119+
120+
```bash
121+
cd drift/instrumentation/fastapi/e2e-tests
122+
./run.sh
123+
```
124+
125+
## Edge Cases to Watch For
126+
127+
1. **Libraries using internal thread pools** (e.g., some HTTP clients, database drivers) - May lose context unless the library explicitly supports it
128+
129+
2. **Fire-and-forget async tasks** - `asyncio.create_task()` preserves context, but if the task outlives the parent span, relationships may be unclear
130+
131+
3. **Gevent/eventlet** - Green threads have different context semantics; not currently tested
132+
133+
4. **Multiprocessing** - Context cannot be serialized across process boundaries; each process needs independent tracing setup

0 commit comments

Comments
 (0)