Skip to content

Commit bd3b7b0

Browse files
authored
ci: parallelize integration tests into matrix jobs (#269)
* ci: run all integration tests, not just runtime Expand the integration test workflow to exercise evaluation, memory, identity, and async test directories in addition to runtime. The tools/ directory is excluded because its test files execute AWS API calls at module level during import. - Add strands-agents-evals dependency for evaluation tests - Broaden pytest target from tests_integ/runtime to tests_integ/ - Ignore tests_integ/tools (unsafe module-level code) - Increase timeout from 10 to 30 minutes for memory and eval tests * ci: parallelize integration tests into matrix jobs Split the single sequential test job into three parallel matrix jobs (runtime, memory, evaluation) to reduce wall-clock time. Each group gets its own timeout and only installs the dependencies it needs.
1 parent 7cf134f commit bd3b7b0

1 file changed

Lines changed: 23 additions & 3 deletions

File tree

.github/workflows/integration-testing.yml

Lines changed: 23 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,7 @@ jobs:
9999
}
100100
101101
check-access-and-checkout:
102+
name: Test (${{ matrix.group }})
102103
runs-on: ubuntu-latest
103104
needs: authorization-check
104105
if: needs.authorization-check.outputs.should-run == 'true'
@@ -107,6 +108,25 @@ jobs:
107108
id-token: write # Required for AWS OIDC
108109
pull-requests: read # Required to read PR info
109110
contents: read # Required to checkout code
111+
strategy:
112+
fail-fast: false
113+
matrix:
114+
include:
115+
- group: runtime
116+
path: tests_integ/runtime
117+
timeout: 10
118+
extra-deps: ""
119+
ignore: ""
120+
- group: memory
121+
path: tests_integ/memory
122+
timeout: 20
123+
extra-deps: ""
124+
ignore: ""
125+
- group: evaluation
126+
path: tests_integ/evaluation
127+
timeout: 15
128+
extra-deps: "strands-agents-evals"
129+
ignore: ""
110130
steps:
111131
- name: Configure Credentials
112132
uses: aws-actions/configure-aws-credentials@v5
@@ -130,16 +150,16 @@ jobs:
130150
- name: Install dependencies
131151
run: |
132152
pip install -e .
133-
pip install --no-cache-dir pytest requests strands-agents uvicorn httpx starlette websockets
153+
pip install --no-cache-dir pytest requests strands-agents uvicorn httpx starlette websockets ${{ matrix.extra-deps }}
134154
135155
- name: Run integration tests
136156
env:
137157
AWS_REGION: us-west-2
138158
PYTHONUNBUFFERED: 1
139159
id: tests
140-
timeout-minutes: 10
160+
timeout-minutes: ${{ matrix.timeout }}
141161
run: |
142-
pytest tests_integ/runtime -s --log-cli-level=INFO
162+
pytest ${{ matrix.path }} ${{ matrix.ignore }} -s --log-cli-level=INFO
143163
144164
safety-gate:
145165
runs-on: ubuntu-latest

0 commit comments

Comments
 (0)