Skip to content

Commit a39b0db

Browse files
committed
Update docs to new test structure
Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com>
1 parent 112cf55 commit a39b0db

2 files changed

Lines changed: 29 additions & 18 deletions

File tree

AGENTS.md

Lines changed: 22 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,9 @@ ext/ # Extension packages (each is a separate PyPI packa
3333
└── flask_dapr/ # Flask integration ← see ext/flask_dapr/AGENTS.md
3434
3535
tests/ # Unit tests (mirrors dapr/ package structure)
36-
examples/ # Integration test suite ← see examples/AGENTS.md
36+
├── examples/ # Output-based tests that run examples and check stdout
37+
├── integration/ # Programmatic SDK tests using DaprClient directly
38+
examples/ # User-facing example applications ← see examples/AGENTS.md
3739
docs/ # Sphinx documentation source
3840
tools/ # Build and release scripts
3941
```
@@ -59,16 +61,21 @@ Each extension is a **separate PyPI package** with its own `setup.cfg`, `setup.p
5961
| `dapr-ext-langgraph` | `dapr.ext.langgraph` | LangGraph checkpoint persistence to Dapr state store | Moderate |
6062
| `dapr-ext-strands` | `dapr.ext.strands` | Strands agent session management via Dapr state store | New |
6163

62-
## Examples (integration test suite)
64+
## Examples and testing
6365

64-
The `examples/` directory serves as both user-facing documentation and the project's integration test suite. Examples are validated by pytest-based integration tests in `tests/integration/`.
66+
The `examples/` directory contains user-facing example applications. These are validated by two test suites:
67+
68+
- **`tests/examples/`** — Output-based tests that run examples via `dapr run` and check stdout for expected strings. Uses a `DaprRunner` helper to manage process lifecycle.
69+
- **`tests/integration/`** — Programmatic SDK tests that call `DaprClient` methods directly and assert on return values, gRPC status codes, and SDK types. More reliable than output-based tests since they don't depend on print statement formatting.
6570

6671
**See `examples/AGENTS.md`** for the full guide on example structure and how to add new examples.
6772

6873
Quick reference:
6974
```bash
70-
tox -e examples # Run all examples (needs Dapr runtime)
71-
tox -e examples -- test_state_store.py # Run a single example
75+
tox -e examples # Run output-based example tests
76+
tox -e examples -- test_state_store.py # Run a single example test
77+
tox -e integration # Run programmatic SDK tests
78+
tox -e integration -- test_state_store.py # Run a single integration test
7279
```
7380

7481
## Python version support
@@ -106,8 +113,11 @@ tox -e ruff
106113
# Run type checking
107114
tox -e type
108115

109-
# Run examples tests / validate examples (requires Dapr runtime)
116+
# Run output-based example tests (requires Dapr runtime)
110117
tox -e examples
118+
119+
# Run programmatic integration tests (requires Dapr runtime)
120+
tox -e integration
111121
```
112122

113123
To run tests directly without tox:
@@ -189,8 +199,8 @@ When completing any task on this project, work through this checklist. Not every
189199
### Examples (integration tests)
190200

191201
- [ ] If you added a new user-facing feature or building block, add or update an example in `examples/`
192-
- [ ] Add a corresponding pytest integration test in `tests/integration/`
193-
- [ ] If you changed output format of existing functionality, update expected output in the affected integration tests
202+
- [ ] Add a corresponding pytest test in `tests/examples/` (output-based) and/or `tests/integration/` (programmatic)
203+
- [ ] If you changed output format of existing functionality, update expected output in `tests/examples/`
194204
- [ ] See `examples/AGENTS.md` for full details on writing examples
195205

196206
### Documentation
@@ -202,7 +212,7 @@ When completing any task on this project, work through this checklist. Not every
202212

203213
- [ ] Run `tox -e ruff` — linting must be clean
204214
- [ ] Run `tox -e py311` (or your Python version) — all unit tests must pass
205-
- [ ] If you touched examples: `tox -e integration -- test_<example-name>.py` to validate locally
215+
- [ ] If you touched examples: `tox -e examples -- test_<example-name>.py` to validate locally
206216
- [ ] Commits must be signed off for DCO: `git commit -s`
207217

208218
## Important files
@@ -217,7 +227,8 @@ When completing any task on this project, work through this checklist. Not every
217227
| `dev-requirements.txt` | Development/test dependencies |
218228
| `dapr/version/__init__.py` | SDK version string |
219229
| `ext/*/setup.cfg` | Extension package metadata and dependencies |
220-
| `tests/integration/` | Pytest-based integration tests that validate examples |
230+
| `tests/examples/` | Output-based tests that validate examples by checking stdout |
231+
| `tests/integration/` | Programmatic SDK tests using DaprClient directly |
221232

222233
## Gotchas
223234

@@ -226,6 +237,6 @@ When completing any task on this project, work through this checklist. Not every
226237
- **Extension independence**: Each extension is a separate PyPI package. Core SDK changes should not break extensions; extension changes should not require core SDK changes unless intentional.
227238
- **DCO signoff**: PRs will be blocked by the DCO bot if commits lack `Signed-off-by`. Always use `git commit -s`.
228239
- **Ruff version pinned**: Dev requirements pin `ruff === 0.14.1`. Use this exact version to match CI.
229-
- **Examples are integration tests**: Changing output format (log messages, print statements) can break integration tests. Always check expected output in `tests/integration/` when modifying user-visible output.
240+
- **Examples are tested by output matching**: Changing output format (log messages, print statements) can break `tests/examples/`. Always check expected output there when modifying user-visible output.
230241
- **Background processes in examples**: Examples that start background services (servers, subscribers) must include a cleanup step to stop them, or CI will hang.
231242
- **Workflow is the most active area**: See `ext/dapr-ext-workflow/AGENTS.md` for workflow-specific architecture and constraints.

examples/AGENTS.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
# AGENTS.md — Dapr Python SDK Examples
22

3-
The `examples/` directory serves as both **user-facing documentation** and the project's **integration test suite**. Each example is a self-contained application validated by pytest-based integration tests in `tests/integration/`.
3+
The `examples/` directory serves as both **user-facing documentation** and the project's **integration test suite**. Each example is a self-contained application validated by pytest-based tests in `tests/examples/`.
44

55
## How validation works
66

7-
1. Each example has a corresponding test file in `tests/integration/` (e.g., `test_state_store.py`)
8-
2. Tests use a `DaprRunner` helper (defined in `conftest.py`) that wraps `dapr run` commands
7+
1. Each example has a corresponding test file in `tests/examples/` (e.g., `test_state_store.py`)
8+
2. Tests use a `DaprRunner` helper (defined in `tests/examples/conftest.py`) that wraps `dapr run` commands
99
3. `DaprRunner.run()` executes a command and captures stdout; `DaprRunner.start()`/`stop()` manage background services
1010
4. Tests assert that expected output lines appear in the captured output
1111

@@ -132,17 +132,17 @@ The `workflow` example includes: `simple.py`, `task_chaining.py`, `fan_out_fan_i
132132
2. Add Python source files and a `requirements.txt` referencing the needed SDK packages
133133
3. Add Dapr component YAMLs in a `components/` subdirectory if the example uses state, pubsub, etc.
134134
4. Write a `README.md` with introduction, pre-requisites, install instructions, and running instructions
135-
5. Add a corresponding test in `tests/integration/test_<example_name>.py`:
135+
5. Add a corresponding test in `tests/examples/test_<example_name>.py`:
136136
- Use the `@pytest.mark.example_dir('<example-name>')` marker to set the working directory
137137
- Use `dapr.run()` for scripts that exit on their own, `dapr.start()`/`dapr.stop()` for long-running services
138138
- Assert expected output lines appear in the captured output
139-
6. Test locally: `tox -e integration -- test_<example_name>.py`
139+
6. Test locally: `tox -e examples -- test_<example_name>.py`
140140

141141
## Gotchas
142142

143-
- **Output format changes break tests**: If you modify print statements or log output in SDK code, check whether any integration test's expected lines depend on that output.
143+
- **Output format changes break tests**: If you modify print statements or log output in SDK code, check whether any test's expected lines in `tests/examples/` depend on that output.
144144
- **Background processes must be cleaned up**: The `DaprRunner` fixture handles cleanup on teardown, but tests should still call `dapr.stop()` to capture output.
145145
- **Dapr prefixes output**: Application stdout appears as `== APP == <line>` when run via `dapr run`.
146146
- **Redis is available in CI**: The CI environment has Redis running on `localhost:6379` — most component YAMLs use this.
147147
- **Some examples need special setup**: `crypto` generates keys, `configuration` seeds Redis, `conversation` needs LLM config — check individual READMEs.
148-
- **Infinite-loop example scripts**: Some example scripts (e.g., `invoke-caller.py`) have `while True` loops for demo purposes. Integration tests must either bypass these with HTTP API calls or use `dapr.run(until=...)` for early termination.
148+
- **Infinite-loop example scripts**: Some example scripts (e.g., `invoke-caller.py`) have `while True` loops for demo purposes. Tests must either bypass these with HTTP API calls or use `dapr.run(until=...)` for early termination.

0 commit comments

Comments
 (0)