Skip to content

Commit 261637b

Browse files
zhongkechenzhongkechen
committed
Merge branch 'testing'
2 parents a65fb4e + 3c7c79b commit 261637b

108 files changed

Lines changed: 43248 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎packages/aws-durable-execution-sdk-python-testing/Dockerfile‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
FROM python:3.13-slim
2+
3+
# Copy and install the wheel
4+
COPY dist/*.whl /tmp/
5+
RUN pip install --no-cache-dir /tmp/*.whl && rm -rf /tmp/*.whl
6+
7+
# AWS credentials (required for boto3)
8+
ENV AWS_ACCESS_KEY_ID=foo \
9+
AWS_SECRET_ACCESS_KEY=bar \
10+
AWS_DEFAULT_REGION=us-east-1
11+
12+
EXPOSE 9014
13+
14+
CMD ["dex-local-runner", "start-server", \
15+
"--host", "0.0.0.0", \
16+
"--port", "9014", \
17+
"--log-level", "DEBUG", \
18+
"--lambda-endpoint", "http://host.docker.internal:3001", \
19+
"--store-type", "sqlite", \
20+
"--store-path", "/tmp/.durable-executions-local/durable-executions.db"]

‎packages/aws-durable-execution-sdk-python-testing/LICENSE‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
2+
Apache License
3+
Version 2.0, January 2004
4+
http://www.apache.org/licenses/
5+
6+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7+
8+
1. Definitions.
9+
10+
"License" shall mean the terms and conditions for use, reproduction,
11+
and distribution as defined by Sections 1 through 9 of this document.
12+
13+
"Licensor" shall mean the copyright owner or entity authorized by
14+
the copyright owner that is granting the License.
15+
16+
"Legal Entity" shall mean the union of the acting entity and all
17+
other entities that control, are controlled by, or are under common
18+
control with that entity. For the purposes of this definition,
19+
"control" means (i) the power, direct or indirect, to cause the
20+
direction or management of such entity, whether by contract or
21+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
22+
outstanding shares, or (iii) beneficial ownership of such entity.
23+
24+
"You" (or "Your") shall mean an individual or Legal Entity
25+
exercising permissions granted by this License.
26+
27+
"Source" form shall mean the preferred form for making modifications,
28+
including but not limited to software source code, documentation
29+
source, and configuration files.
30+
31+
"Object" form shall mean any form resulting from mechanical
32+
transformation or translation of a Source form, including but
33+
not limited to compiled object code, generated documentation,
34+
and conversions to other media types.
35+
36+
"Work" shall mean the work of authorship, whether in Source or
37+
Object form, made available under the License, as indicated by a
38+
copyright notice that is included in or attached to the work
39+
(an example is provided in the Appendix below).
40+
41+
"Derivative Works" shall mean any work, whether in Source or Object
42+
form, that is based on (or derived from) the Work and for which the
43+
editorial revisions, annotations, elaborations, or other modifications
44+
represent, as a whole, an original work of authorship. For the purposes
45+
of this License, Derivative Works shall not include works that remain
46+
separable from, or merely link (or bind by name) to the interfaces of,
47+
the Work and Derivative Works thereof.
48+
49+
"Contribution" shall mean any work of authorship, including
50+
the original version of the Work and any modifications or additions
51+
to that Work or Derivative Works thereof, that is intentionally
52+
submitted to Licensor for inclusion in the Work by the copyright owner
53+
or by an individual or Legal Entity authorized to submit on behalf of
54+
the copyright owner. For the purposes of this definition, "submitted"
55+
means any form of electronic, verbal, or written communication sent
56+
to the Licensor or its representatives, including but not limited to
57+
communication on electronic mailing lists, source code control systems,
58+
and issue tracking systems that are managed by, or on behalf of, the
59+
Licensor for the purpose of discussing and improving the Work, but
60+
excluding communication that is conspicuously marked or otherwise
61+
designated in writing by the copyright owner as "Not a Contribution."
62+
63+
"Contributor" shall mean Licensor and any individual or Legal Entity
64+
on behalf of whom a Contribution has been received by Licensor and
65+
subsequently incorporated within the Work.
66+
67+
2. Grant of Copyright License. Subject to the terms and conditions of
68+
this License, each Contributor hereby grants to You a perpetual,
69+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70+
copyright license to reproduce, prepare Derivative Works of,
71+
publicly display, publicly perform, sublicense, and distribute the
72+
Work and such Derivative Works in Source or Object form.
73+
74+
3. Grant of Patent License. Subject to the terms and conditions of
75+
this License, each Contributor hereby grants to You a perpetual,
76+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77+
(except as stated in this section) patent license to make, have made,
78+
use, offer to sell, sell, import, and otherwise transfer the Work,
79+
where such license applies only to those patent claims licensable
80+
by such Contributor that are necessarily infringed by their
81+
Contribution(s) alone or by combination of their Contribution(s)
82+
with the Work to which such Contribution(s) was submitted. If You
83+
institute patent litigation against any entity (including a
84+
cross-claim or counterclaim in a lawsuit) alleging that the Work
85+
or a Contribution incorporated within the Work constitutes direct
86+
or contributory patent infringement, then any patent licenses
87+
granted to You under this License for that Work shall terminate
88+
as of the date such litigation is filed.
89+
90+
4. Redistribution. You may reproduce and distribute copies of the
91+
Work or Derivative Works thereof in any medium, with or without
92+
modifications, and in Source or Object form, provided that You
93+
meet the following conditions:
94+
95+
(a) You must give any other recipients of the Work or
96+
Derivative Works a copy of this License; and
97+
98+
(b) You must cause any modified files to carry prominent notices
99+
stating that You changed the files; and
100+
101+
(c) You must retain, in the Source form of any Derivative Works
102+
that You distribute, all copyright, patent, trademark, and
103+
attribution notices from the Source form of the Work,
104+
excluding those notices that do not pertain to any part of
105+
the Derivative Works; and
106+
107+
(d) If the Work includes a "NOTICE" text file as part of its
108+
distribution, then any Derivative Works that You distribute must
109+
include a readable copy of the attribution notices contained
110+
within such NOTICE file, excluding those notices that do not
111+
pertain to any part of the Derivative Works, in at least one
112+
of the following places: within a NOTICE text file distributed
113+
as part of the Derivative Works; within the Source form or
114+
documentation, if provided along with the Derivative Works; or,
115+
within a display generated by the Derivative Works, if and
116+
wherever such third-party notices normally appear. The contents
117+
of the NOTICE file are for informational purposes only and
118+
do not modify the License. You may add Your own attribution
119+
notices within Derivative Works that You distribute, alongside
120+
or as an addendum to the NOTICE text from the Work, provided
121+
that such additional attribution notices cannot be construed
122+
as modifying the License.
123+
124+
You may add Your own copyright statement to Your modifications and
125+
may provide additional or different license terms and conditions
126+
for use, reproduction, or distribution of Your modifications, or
127+
for any such Derivative Works as a whole, provided Your use,
128+
reproduction, and distribution of the Work otherwise complies with
129+
the conditions stated in this License.
130+
131+
5. Submission of Contributions. Unless You explicitly state otherwise,
132+
any Contribution intentionally submitted for inclusion in the Work
133+
by You to the Licensor shall be under the terms and conditions of
134+
this License, without any additional terms or conditions.
135+
Notwithstanding the above, nothing herein shall supersede or modify
136+
the terms of any separate license agreement you may have executed
137+
with Licensor regarding such Contributions.
138+
139+
6. Trademarks. This License does not grant permission to use the trade
140+
names, trademarks, service marks, or product names of the Licensor,
141+
except as required for reasonable and customary use in describing the
142+
origin of the Work and reproducing the content of the NOTICE file.
143+
144+
7. Disclaimer of Warranty. Unless required by applicable law or
145+
agreed to in writing, Licensor provides the Work (and each
146+
Contributor provides its Contributions) on an "AS IS" BASIS,
147+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148+
implied, including, without limitation, any warranties or conditions
149+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150+
PARTICULAR PURPOSE. You are solely responsible for determining the
151+
appropriateness of using or redistributing the Work and assume any
152+
risks associated with Your exercise of permissions under this License.
153+
154+
8. Limitation of Liability. In no event and under no legal theory,
155+
whether in tort (including negligence), contract, or otherwise,
156+
unless required by applicable law (such as deliberate and grossly
157+
negligent acts) or agreed to in writing, shall any Contributor be
158+
liable to You for damages, including any direct, indirect, special,
159+
incidental, or consequential damages of any character arising as a
160+
result of this License or out of the use or inability to use the
161+
Work (including but not limited to damages for loss of goodwill,
162+
work stoppage, computer failure or malfunction, or any and all
163+
other commercial damages or losses), even if such Contributor
164+
has been advised of the possibility of such damages.
165+
166+
9. Accepting Warranty or Additional Liability. While redistributing
167+
the Work or Derivative Works thereof, You may choose to offer,
168+
and charge a fee for, acceptance of support, warranty, indemnity,
169+
or other liability obligations and/or rights consistent with this
170+
License. However, in accepting such obligations, You may act only
171+
on Your own behalf and on Your sole responsibility, not on behalf
172+
of any other Contributor, and only if You agree to indemnify,
173+
defend, and hold each Contributor harmless for any liability
174+
incurred by, or claims asserted against, such Contributor by reason
175+
of your accepting any such warranty or additional liability.

‎packages/aws-durable-execution-sdk-python-testing/NOTICE‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.

‎packages/aws-durable-execution-sdk-python-testing/README.md‎

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
# AWS Durable Execution Testing SDK for Python
2+
3+
[![PyPI - Version](https://img.shields.io/pypi/v/aws-durable-execution-sdk-python-testing.svg)](https://pypi.org/project/aws-durable-execution-sdk-python-testing)
4+
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/aws-durable-execution-sdk-python-testing.svg)](https://pypi.org/project/aws-durable-execution-sdk-python-testing)
5+
6+
7+
[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/aws/aws-durable-execution-sdk-python-testing/badge)](https://scorecard.dev/viewer/?uri=github.com/aws/aws-durable-execution-sdk-python-testing)
8+
9+
-----
10+
11+
## Table of Contents
12+
13+
- [Installation](#installation)
14+
- [Quick Start](#quick-start)
15+
- [Architecture](#architecture)
16+
- [Documentation](#documentation)
17+
- [Developer Guide](#developers)
18+
- [License](#license)
19+
20+
## Installation
21+
22+
```console
23+
pip install aws-durable-execution-sdk-python-testing
24+
```
25+
26+
## Overview
27+
28+
Use the AWS Durable Execution Testing SDK for Python to test your Python durable functions locally.
29+
30+
The test framework contains a local runner, so you can run and test your durable function locally
31+
before you deploy it.
32+
33+
## Quick Start
34+
35+
### A durable function under test
36+
37+
```python
38+
from aws_durable_execution_sdk_python.context import (
39+
DurableContext,
40+
durable_step,
41+
durable_with_child_context,
42+
)
43+
from aws_durable_execution_sdk_python.execution import durable_execution
44+
from aws_durable_execution_sdk_python.config import Duration
45+
46+
47+
@durable_step
48+
def one(a: int, b: int) -> str:
49+
return f"{a} {b}"
50+
51+
@durable_step
52+
def two_1(a: int, b: int) -> str:
53+
return f"{a} {b}"
54+
55+
@durable_step
56+
def two_2(a: int, b: int) -> str:
57+
return f"{b} {a}"
58+
59+
@durable_with_child_context
60+
def two(ctx: DurableContext, a: int, b: int) -> str:
61+
two_1_result: str = ctx.step(two_1(a, b))
62+
two_2_result: str = ctx.step(two_2(a, b))
63+
return f"{two_1_result} {two_2_result}"
64+
65+
@durable_step
66+
def three(a: int, b: int) -> str:
67+
return f"{a} {b}"
68+
69+
@durable_execution
70+
def function_under_test(event: Any, context: DurableContext) -> list[str]:
71+
results: list[str] = []
72+
73+
result_one: str = context.step(one(1, 2))
74+
results.append(result_one)
75+
76+
context.wait(Duration.from_seconds(1))
77+
78+
result_two: str = context.run_in_child_context(two(3, 4))
79+
results.append(result_two)
80+
81+
result_three: str = context.step(three(5, 6))
82+
results.append(result_three)
83+
84+
return results
85+
```
86+
87+
### Your test code
88+
89+
```python
90+
from aws_durable_execution_sdk_python.execution import InvocationStatus
91+
from aws_durable_execution_sdk_python_testing.runner import (
92+
ContextOperation,
93+
DurableFunctionTestResult,
94+
DurableFunctionTestRunner,
95+
StepOperation,
96+
)
97+
98+
def test_my_durable_functions():
99+
with DurableFunctionTestRunner(handler=function_under_test) as runner:
100+
result: DurableFunctionTestResult = runner.run(input="input str", timeout=10)
101+
102+
assert result.status is InvocationStatus.SUCCEEDED
103+
assert result.result == '["1 2", "3 4 4 3", "5 6"]'
104+
105+
one_result: StepOperation = result.get_step("one")
106+
assert one_result.result == '"1 2"'
107+
108+
two_result: ContextOperation = result.get_context("two")
109+
assert two_result.result == '"3 4 4 3"'
110+
111+
three_result: StepOperation = result.get_step("three")
112+
assert three_result.result == '"5 6"'
113+
```
114+
## Architecture
115+
![Durable Functions Python Test Framework Architecture](assets/dar-python-test-framework-architecture.svg)
116+
117+
## Event Flow
118+
![Event Flow Sequence Diagram](assets/dar-python-test-framework-event-flow.svg)
119+
120+
1. **DurableTestRunner** starts execution via **Executor**
121+
2. **Executor** creates **Execution** and schedules initial invocation
122+
3. During execution, checkpoints are processed by **CheckpointProcessor**
123+
4. **Individual Processors** transform operation updates and may trigger events
124+
5. **ExecutionNotifier** broadcasts events to **Executor** (observer)
125+
6. **Executor** updates **Execution** state based on events
126+
7. **Execution** completion triggers final event notifications
127+
8. **DurableTestRunner** run() blocks until it receives completion event, and then returns `DurableFunctionTestResult`.
128+
129+
## Major Components
130+
131+
### Core Execution Flow
132+
- **DurableTestRunner** - Main entry point that orchestrates test execution
133+
- **Executor** - Manages execution lifecycle. Mutates Execution.
134+
- **Execution** - Represents the state and operations of a single durable execution
135+
136+
### Service Client Integration
137+
- **InMemoryServiceClient** - Replaces AWS Lambda service client for local testing. Injected into SDK via `DurableExecutionInvocationInputWithClient`
138+
139+
### Checkpoint Processing Pipeline
140+
- **CheckpointProcessor** - Orchestrates operation transformations and validation
141+
- **Individual Validators** - Validate operation updates and state transitions
142+
- **Individual Processors** - Transform operation updates into operations (step, wait, callback, context, execution)
143+
144+
### Execution status changes (Observer Pattern)
145+
- **ExecutionNotifier** - Notifies observers of execution events
146+
- **ExecutionObserver** - Interface for receiving execution lifecycle events
147+
- **Executor** implements `ExecutionObserver` to handle completion events
148+
149+
## Component Relationships
150+
151+
### 1. DurableTestRunner → Executor → Execution
152+
- **DurableTestRunner** serves as the main API entry point and sets up all components
153+
- **Executor** manages the execution lifecycle, handling invocations and state transitions
154+
- **Execution** maintains the state of operations and completion status
155+
156+
### 2. Service Client Injection
157+
- **DurableTestRunner** creates **InMemoryServiceClient** with **CheckpointProcessor**
158+
- **InProcessInvoker** injects the service client into SDK via `DurableExecutionInvocationInputWithClient`
159+
- When durable functions call checkpoint operations, they're intercepted by **InMemoryServiceClient**
160+
- **InMemoryServiceClient** delegates to **CheckpointProcessor** for local processing
161+
162+
### 3. CheckpointProcessor → Individual Validators → Individual Processors
163+
- **CheckpointProcessor** orchestrates the checkpoint processing pipeline
164+
- **Individual Validators** (CheckpointValidator, TransitionsValidator, and operation-specific validators) ensure operation updates are valid
165+
- **Individual Processors** (StepProcessor, WaitProcessor, etc.) transform `OperationUpdate` into `Operation`
166+
167+
### 4. Observer Pattern Flow
168+
The observer pattern enables loose coupling between checkpoint processing and execution management:
169+
170+
1. **CheckpointProcessor** processes operation updates
171+
2. **Individual Processors** detect state changes (completion, failures, timer scheduling)
172+
3. **ExecutionNotifier** broadcasts events to registered observers
173+
4. **Executor** (as ExecutionObserver) receives notifications and updates **Execution** state
174+
5. **Execution** complete_* methods finalize the execution state
175+
176+
177+
## Documentation
178+
179+
### Error Handling
180+
181+
The testing framework implements AWS-compliant error responses that match the exact format expected by boto3 and AWS services. For detailed information about error response formats, exception types, and troubleshooting, see:
182+
183+
- [Error Response Documentation](docs/error-responses.md)
184+
185+
Key features:
186+
- **AWS-compliant JSON format**: Matches boto3 expectations exactly
187+
- **Smithy model compliance**: Field names follow AWS Smithy definitions
188+
- **HTTP status code mapping**: Standard AWS service status codes
189+
- **Boto3 compatibility**: Seamless integration with boto3 error handling
190+
191+
## Developers
192+
Please see [CONTRIBUTING.md](CONTRIBUTING.md). It contains the testing guide, sample commands and instructions
193+
for how to contribute to this package.
194+
195+
tldr; use `hatch` and it will manage virtual envs and dependencies for you, so you don't have to do it manually.
196+
197+
## License
198+
199+
This project is licensed under the [Apache-2.0 License](LICENSE).

‎packages/aws-durable-execution-sdk-python-testing/assets/dar-python-test-framework-architecture.svg‎

Lines changed: 1 addition & 0 deletions
Loading

0 commit comments

Comments
 (0)