Skip to content

Commit 6ba947e

Browse files
sehervCasperGN
andauthored
Add missing sync and new async integration tests (#996)
* Move old integration tests to examples/ Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Test DaprClient directly Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Update docs to new test structure Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address Copilot comments (1) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address Copilot comments (2) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address Copilot comments (3) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Replace sleep() with polls when possible Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address Copilot comments (4) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address Copilot comments (5) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Update README to include both test suites Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Document wait_until() in AGENTS.md Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Update CLAUDE.md Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Fix package name Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Clean up entire process group Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * PR cleanup (1) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Fix possible race running example test Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Refactor functions common to example tests and integration tests Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Add regression test for config race Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Add tests for uncovered components Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Add async tests Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Update docs Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Merge resources/ change from main Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Add Redis to deps for regression test Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Create crypto keys at runtime Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Fix buffering issue on pubsub example test Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address Copilot comments Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address Copilot comments (2) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address Copilot comments (3) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Create keys before loading dapr Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Reorder tests to fail faster Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Re-raise exception explicitly Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Refactor Ollama fixture and create Ollama tests Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address PR comments Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Set up Ollama before integration tests Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Run linter Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Refactor unique name generation Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address PR feedback (2) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Address PR feedback (3) Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Add license headers to all new files Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Remove shell loop in build.yaml Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> * Run unit tests with pytest Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> --------- Signed-off-by: Sergio Herrera <627709+seherv@users.noreply.github.com> Co-authored-by: Casper Nielsen <casper@diagrid.io>
1 parent 3b26a1a commit 6ba947e

49 files changed

Lines changed: 1980 additions & 478 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/build-push-to-main.yaml

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -52,11 +52,7 @@ jobs:
5252
run: uv run mypy
5353
- name: Run unit-tests
5454
run: |
55-
uv run coverage run -m unittest discover -v ./tests
56-
for dir in ext/*/tests; do
57-
uv run coverage run -a -m unittest discover -v "./$dir"
58-
done
59-
uv run coverage run -a -m pytest -m "not e2e" ./ext/dapr-ext-workflow/tests/durabletask/
55+
uv run coverage run -m pytest tests ext -m "not e2e" --ignore=tests/integration --ignore=tests/examples --import-mode=importlib
6056
uv run coverage xml
6157
- name: Upload test coverage
6258
uses: codecov/codecov-action@v6

.github/workflows/build-tag.yaml

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -52,11 +52,7 @@ jobs:
5252
run: uv run mypy
5353
- name: Run unit-tests
5454
run: |
55-
uv run coverage run -m unittest discover -v ./tests
56-
for dir in ext/*/tests; do
57-
uv run coverage run -a -m unittest discover -v "./$dir"
58-
done
59-
uv run coverage run -a -m pytest -m "not e2e" ./ext/dapr-ext-workflow/tests/durabletask/
55+
uv run coverage run -m pytest tests ext -m "not e2e" --ignore=tests/integration --ignore=tests/examples --import-mode=importlib
6056
uv run coverage xml
6157
- name: Upload test coverage
6258
uses: codecov/codecov-action@v6

.github/workflows/build.yaml

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -58,11 +58,7 @@ jobs:
5858
run: uv run mypy
5959
- name: Run unit-tests
6060
run: |
61-
uv run coverage run -m unittest discover -v ./tests
62-
for dir in ext/*/tests; do
63-
uv run coverage run -a -m unittest discover -v "./$dir"
64-
done
65-
uv run coverage run -a -m pytest -m "not e2e" ./ext/dapr-ext-workflow/tests/durabletask/
61+
uv run coverage run -m pytest tests ext -m "not e2e" --ignore=tests/integration --ignore=tests/examples --import-mode=importlib
6662
uv run coverage xml
6763
- name: Upload test coverage
6864
uses: codecov/codecov-action@v6

.github/workflows/run-tests.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
name: validate-examples
1+
name: run-tests
22

33
on:
44
push:
@@ -86,7 +86,7 @@ jobs:
8686
- name: Install uv
8787
uses: astral-sh/setup-uv@v7
8888
- name: Install dependencies
89-
run: uv sync --frozen --all-packages --group examples
89+
run: uv sync --frozen --all-packages --group tests
9090
- name: Set up Dapr CLI
9191
uses: dapr/.github/.github/actions/setup-dapr-cli@main
9292
with:
@@ -103,9 +103,9 @@ jobs:
103103
nohup ollama serve &
104104
sleep 10
105105
ollama pull llama3.2:latest
106-
- name: Check examples
107-
run: |
108-
uv run pytest tests/examples/
109106
- name: Run integration tests
110107
run: |
111108
uv run pytest tests/integration/
109+
- name: Validate examples
110+
run: |
111+
uv run pytest tests/examples/

.gitignore

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,3 +136,9 @@ coverage.lcov
136136

137137
# macOS specific files
138138
.DS_Store
139+
140+
# Integration test scratch dirs
141+
tests/integration/.binding-data/
142+
143+
# Crypto test material generated at test time (see tests/crypto_utils.py)
144+
tests/integration/keys/

CLAUDE.md

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,20 @@
11
@AGENTS.md
22

3-
Use pathlib instead of os.path.
4-
Use httpx instead of urllib.
5-
subprocess(`shell=True`) is used only when it makes the code more readable. Use either shlex or args lists.
6-
subprocess calls should have a reasonable timeout.
7-
Use modern Python (3.10+) features.
83
Make all code strongly typed.
94
Keep conditional nesting to a minimum, and use guard clauses when possible.
10-
Aim for medium "visual complexity": use intermediate variables to store results of nested/complex function calls, but don't create a new variable for everything.
11-
Avoid comments unless there is a gotcha, a complex algorithm or anything an experienced code reviewer needs to be aware of. Focus on making better Google-style docstrings instead.
5+
Aim for medium visual complexity: use intermediate variables to store results of nested/complex function calls. A complex function call could be:
6+
- `f(Object(a=1, b=2, c=3))`, the inner object has more than 2 meaningful args
7+
- `f(Object((a, b)))`, 2 levels of nesting or anything with a long chain of closing parens
8+
- `small_transformation(ImportantObject())`, the object itself is the main subject of the function but the transformation steals the focus
9+
Use descriptive, self-documenting names for these intermediate variables.
10+
Closely related variable names should share a root and use different suffixes. For example, `request_original` and `request_clean`, but not `clean_request`.
11+
Avoid comments unless there is a gotcha, a complex algorithm or anything an experienced code reviewer needs to be aware of. Focus on making short but descriptive Google-style docstrings instead.
1212

13-
The user is not always right. Be skeptical and do not blindly comply if something doesn't make sense.
13+
Use modern Python (3.10+) features.
14+
Use pathlib instead of os.path.
15+
Use httpx instead of urllib.
16+
`subprocess(shell=True)` is used only when it makes the code more readable. Use either shlex or args lists.
17+
Anything that can have an explicit timeout should have one.
1418
Code should be cross-platform and production ready.
19+
20+
The user is not always right. Be skeptical and do not blindly comply if something doesn't make sense.

examples/pubsub-streaming-async/publisher.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ async def publish_events():
4444
)
4545

4646
# Print the request
47-
print(req_data, flush=True)
47+
print(req_data)
4848

4949
await asyncio.sleep(1)
5050

examples/pubsub-streaming-async/subscriber.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ def process_message(message):
1919
global counter
2020
counter += 1
2121
# Process the message here
22-
print(f'Processing message: {message.data()} from {message.topic()}...', flush=True)
22+
print(f'Processing message: {message.data()} from {message.topic()}...')
2323
return 'success'
2424

2525

pyproject.toml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,14 +71,17 @@ dev = [
7171
"opentelemetry-exporter-zipkin~=1.11.1",
7272
"httpx~=0.28.1",
7373
"pyOpenSSL~=26.0.0",
74+
"cryptography>=42.0.0",
75+
"redis>=7.4.0",
7476
"Flask~=3.1.3",
7577
"pytest~=9.0.2",
78+
"pytest-asyncio>=0.23",
7679
"ruff==0.14.1",
7780
"python-dotenv~=1.2.2",
7881
"pydantic~=2.13.3",
7982
"PyYAML~=6.0.3",
8083
]
81-
examples = [
84+
tests = [
8285
{include-group = "dev"},
8386
"mechanical-markdown~=0.8.0",
8487
"langchain-ollama~=1.0.1",
@@ -142,3 +145,4 @@ markers = [
142145
'example_dir(name): set the example directory for the dapr fixture',
143146
]
144147
pythonpath = ["."]
148+
asyncio_mode = "auto"

tests/conftest.py

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# -*- coding: utf-8 -*-
2+
3+
"""
4+
Copyright 2026 The Dapr Authors
5+
Licensed under the Apache License, Version 2.0 (the "License");
6+
you may not use this file except in compliance with the License.
7+
You may obtain a copy of the License at
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
Unless required by applicable law or agreed to in writing, software
10+
distributed under the License is distributed on an "AS IS" BASIS,
11+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
See the License for the specific language governing permissions and
13+
limitations under the License.
14+
"""
15+
16+
import subprocess
17+
from typing import Callable, Iterator
18+
19+
import pytest
20+
21+
from tests.ollama_utils import DEFAULT_MODEL, model_available, ollama_ready
22+
from tests.process_utils import get_kwargs_for_process_group, terminate_process_group
23+
from tests.wait_utils import wait_until
24+
25+
REDIS_CONTAINER = 'dapr_redis'
26+
27+
28+
@pytest.fixture(scope='session')
29+
def flush_redis() -> None:
30+
"""Flush the ``dapr_redis`` container once per session."""
31+
subprocess.run(
32+
args=('docker', 'exec', REDIS_CONTAINER, 'redis-cli', 'FLUSHDB'),
33+
check=True,
34+
capture_output=True,
35+
timeout=10,
36+
)
37+
38+
39+
@pytest.fixture(scope='session')
40+
def redis_set_config() -> Callable[[str, str, int], None]:
41+
"""Dapr encodes values in the config store as ``value||version``"""
42+
43+
def _set(key: str, value: str, version: int = 1) -> None:
44+
subprocess.run(
45+
args=(
46+
'docker',
47+
'exec',
48+
REDIS_CONTAINER,
49+
'redis-cli',
50+
'SET',
51+
key,
52+
f'{value}||{version}',
53+
),
54+
check=True,
55+
capture_output=True,
56+
timeout=10,
57+
)
58+
59+
return _set
60+
61+
62+
@pytest.fixture(scope='session')
63+
def ollama() -> Iterator[None]:
64+
"""Ensure an Ollama server with the default model is running for the session."""
65+
started: subprocess.Popen[str] | None = None
66+
try:
67+
if not ollama_ready():
68+
try:
69+
started = subprocess.Popen(
70+
['ollama', 'serve'],
71+
stdout=subprocess.DEVNULL,
72+
stderr=subprocess.DEVNULL,
73+
text=True,
74+
**get_kwargs_for_process_group(),
75+
)
76+
except FileNotFoundError as exc:
77+
pytest.fail(f'ollama CLI is not installed: {exc}')
78+
wait_until(ollama_ready, timeout=30.0, interval=0.5)
79+
80+
if not model_available(DEFAULT_MODEL):
81+
subprocess.run(['ollama', 'pull', DEFAULT_MODEL], check=True, capture_output=True)
82+
83+
yield
84+
finally:
85+
if started and started.poll() is None:
86+
terminate_process_group(started)
87+
try:
88+
started.wait(timeout=10)
89+
except subprocess.TimeoutExpired:
90+
terminate_process_group(started, force=True)
91+
started.wait()

0 commit comments

Comments
 (0)