Skip to content

Commit 3705c8a

Browse files
NeethuESimNeethuESim
committed
doc: add testcontainers learning path
1 parent 3410a2c commit 3705c8a

7 files changed

Lines changed: 732 additions & 0 deletions

File tree

‎content/learning-paths/cross-platform/automate-mcp-with-testcontainers/_index.md‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
title: Automate MCP Server testing using Pytest and Testcontainers
3+
4+
minutes_to_complete: 60
5+
6+
who_is_this_for: This is an intermediate topic for anyone interested in automating their end-to-end testing of MCP (Model Context Protocol) servers using specialized software packages called TestContainers and PyTest.
7+
8+
learning_objectives:
9+
- Understand how TestContainers work for containerized testing of MCP servers.
10+
- Write integration tests for MCP servers using PyTest and TestContainers.
11+
- Configure GitHub Actions to run MCP integration tests in CI/CD pipelines.
12+
13+
prerequisites:
14+
- A machine that can run Python 3 and Docker.
15+
- Familiarity with [Docker](/install-guides/docker/) and container concepts.
16+
- Basic knowledge of Python and Pytest.
17+
- The [Arm MCP Server](https://github.com/arm/mcp) running with AI assistant client.
18+
19+
author: Neethu Elizabeth Simon
20+
21+
### Tags
22+
skilllevels: Introductory
23+
subjects: CI-CD
24+
armips:
25+
- Neoverse
26+
- Cortex-A
27+
operatingsystems:
28+
- Linux
29+
- macOS
30+
- Windows
31+
tools_software_languages:
32+
- Python
33+
- Pytest
34+
- Docker
35+
- GitHub Actions
36+
- Testcontainers
37+
- MCP
38+
39+
further_reading:
40+
- resource:
41+
title: Arm MCP Server GitHub Repository
42+
link: https://github.com/arm/mcp
43+
type: website
44+
- resource:
45+
title: Testcontainers for Python Documentation
46+
link: https://testcontainers-python.readthedocs.io/
47+
type: documentation
48+
- resource:
49+
title: Model Context Protocol Specification
50+
link: https://modelcontextprotocol.io/
51+
type: website
52+
- resource:
53+
title: PyTest Documentation
54+
link: https://docs.pytest.org/
55+
type: documentation
56+
57+
### FIXED, DO NOT MODIFY
58+
# ================================================================================
59+
weight: 1 # _index.md always has weight of 1 to order correctly
60+
layout: "learningpathall" # All files under learning paths have this same wrapper
61+
learning_path_main_page: "yes" # This should be surfaced when looking for related content. Only set for _index.md of learning path content.
62+
---

‎content/learning-paths/cross-platform/automate-mcp-with-testcontainers/_next-steps.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
---
2+
# ================================================================================
3+
# FIXED, DO NOT MODIFY THIS FILE
4+
# ================================================================================
5+
weight: 21 # The weight controls the order of the pages. _index.md always has weight 1.
6+
title: "Next Steps" # Always the same, html page title.
7+
layout: "learningpathall" # All files under learning paths have this same wrapper for Hugo processing.
8+
---

‎content/learning-paths/cross-platform/automate-mcp-with-testcontainers/github-actions-ci.md‎

Lines changed: 197 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,197 @@
1+
---
2+
title: Configure GitHub Actions for CI/CD
3+
weight: 5
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## Why use GitHub Actions for MCP testing?
10+
11+
GitHub Actions provide automated CI/CD directly in your repository. For MCP server testing, it offers:
12+
13+
- **Arm runner support**: GitHub provides native Arm64 runners for building and testing.
14+
- **Docker integration**: Runners come with Docker pre-installed.
15+
- **Automatic triggers**: Tests run on every push and pull request.
16+
- **Parallel execution**: Multiple jobs can run simultaneously.
17+
18+
## Create the workflow file
19+
20+
Create a GitHub Actions workflow file at `.github/workflows/integration-tests.yml`:
21+
22+
```yaml
23+
name: Integration Tests
24+
25+
on:
26+
push:
27+
pull_request:
28+
29+
jobs:
30+
integration-tests:
31+
runs-on: ubuntu-24.04-arm
32+
steps:
33+
- name: Checkout
34+
uses: actions/checkout@v4
35+
36+
- name: Set up Python
37+
uses: actions/setup-python@v5
38+
with:
39+
python-version: "3.11"
40+
cache: pip
41+
42+
- name: Install test dependencies
43+
run: |
44+
python -m pip install --upgrade pip
45+
pip install -r mcp-local/tests/requirements.txt
46+
47+
- name: Build MCP Docker image
48+
run: docker buildx build -f mcp-local/Dockerfile -t arm-mcp .
49+
50+
- name: Run integration tests
51+
env:
52+
MCP_IMAGE: arm-mcp:latest
53+
run: pytest -v mcp-local/tests/test_mcp.py
54+
```
55+
56+
## Understand the workflow configuration
57+
58+
The workflow uses the `ubuntu-24.04-arm` runner, which is a GitHub-hosted Arm64 runner. This ensures that both the Docker build and the tests execute natively on Arm hardware.
59+
60+
Key aspects of the configuration:
61+
62+
### Trigger events
63+
64+
```yaml
65+
on:
66+
push:
67+
pull_request:
68+
```
69+
70+
This configuration triggers the workflow on every push to any branch and on pull request events. You can restrict this to specific branches if needed:
71+
72+
```yaml
73+
on:
74+
push:
75+
branches: [main, develop]
76+
pull_request:
77+
branches: [main]
78+
```
79+
80+
### Python caching
81+
82+
```yaml
83+
- name: Set up Python
84+
uses: actions/setup-python@v5
85+
with:
86+
python-version: "3.11"
87+
cache: pip
88+
```
89+
90+
The `cache: pip` option caches Python packages between runs, significantly speeding up subsequent workflow executions.
91+
92+
### Environment variables
93+
94+
```yaml
95+
- name: Run integration tests
96+
env:
97+
MCP_IMAGE: arm-mcp:latest
98+
run: pytest -v mcp-local/tests/test_mcp.py
99+
```
100+
101+
The `MCP_IMAGE` environment variable tells the test suite which Docker image to use. The test code reads this with:
102+
103+
```python
104+
image = os.getenv("MCP_IMAGE", constants.MCP_DOCKER_IMAGE)
105+
```
106+
107+
## Add test result artifacts
108+
109+
Enhance the workflow to save test results as artifacts for debugging:
110+
111+
```yaml
112+
- name: Run integration tests
113+
env:
114+
MCP_IMAGE: arm-mcp:latest
115+
run: pytest -v mcp-local/tests/test_mcp.py --junitxml=test-results.xml
116+
117+
- name: Upload test results
118+
uses: actions/upload-artifact@v4
119+
if: always()
120+
with:
121+
name: test-results
122+
path: test-results.xml
123+
```
124+
125+
The `if: always()` condition ensures test results upload even when tests fail.
126+
127+
## Add a build matrix for multiple platforms
128+
129+
To test on both Arm64 and x86_64, use a matrix strategy:
130+
131+
```yaml
132+
jobs:
133+
integration-tests:
134+
strategy:
135+
matrix:
136+
runner: [ubuntu-24.04-arm, ubuntu-latest]
137+
runs-on: ${{ matrix.runner }}
138+
steps:
139+
# ... same steps as before
140+
```
141+
142+
This runs the integration tests in parallel on both architectures.
143+
144+
## Monitor workflow runs
145+
146+
After pushing the workflow file, navigate to the Actions tab in your GitHub repository. Each workflow run shows:
147+
148+
- Build steps and their status
149+
- Execution time for each step
150+
- Log output for debugging failures
151+
- Artifacts for download
152+
153+
## Troubleshoot common issues
154+
155+
If the workflow fails, check these common causes:
156+
157+
**Docker build timeout**: The initial image build can take 10+ minutes. GitHub Actions has a default timeout of 360 minutes per job, but individual steps might need explicit timeouts:
158+
159+
```yaml
160+
- name: Build MCP Docker image
161+
timeout-minutes: 30
162+
run: docker buildx build -f mcp-local/Dockerfile -t arm-mcp .
163+
```
164+
165+
**Container startup issues**: If tests fail with timeout errors, the MCP server might not be starting correctly. Add debug output:
166+
167+
```yaml
168+
- name: Run integration tests
169+
env:
170+
MCP_IMAGE: arm-mcp:latest
171+
run: |
172+
docker run --rm arm-mcp:latest echo "Container starts successfully"
173+
pytest -v -s mcp-local/tests/test_mcp.py
174+
```
175+
176+
**Rate limiting**: If tests query external services, you might encounter rate limits. Consider adding retry logic or using mock responses for CI environments.
177+
178+
## What you've accomplished and what's next
179+
180+
In this section:
181+
- You created a GitHub Actions workflow for automated testing.
182+
- You learned how to use Arm64 runners for native execution.
183+
- You added test artifacts and multi-platform support.
184+
- You explored troubleshooting techniques for CI failures.
185+
186+
You now have a complete CI/CD pipeline that automatically tests your MCP server on every code change.
187+
188+
## Summary
189+
190+
In this Learning Path, you learned how to:
191+
192+
- Set up testcontainers for Docker-based integration testing.
193+
- Write pytest tests that communicate with MCP servers over stdio transport.
194+
- Parse MCP JSON-RPC responses and validate tool outputs.
195+
- Configure GitHub Actions with Arm64 runners for automated testing.
196+
197+
These techniques apply to any MCP server implementation, not just the Arm MCP Server. Use this foundation to build comprehensive test suites that ensure your MCP tools work correctly across updates and deployments.

‎content/learning-paths/cross-platform/automate-mcp-with-testcontainers/introduction.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
---
2+
title: Introduction to MCP Server Testing
3+
weight: 2
4+
5+
### FIXED, DO NOT MODIFY
6+
layout: learningpathall
7+
---
8+
9+
## What is MCP?
10+
11+
The Model Context Protocol (MCP) is an open standard that enables AI assistants to securely connect to external data sources and tools. MCP servers provide AI models with context aware capabilities, such as code analysis, knowledge base lookups, and system introspection.
12+
13+
The Arm MCP Server is a reference implementation that provides AI assistants with tools and knowledge specifically for Arm architecture development, migration, and optimization. It includes capabilities like container architecture checking, code analysis with LLVM-MCA, and access to Arm Learning Paths.
14+
15+
## Why automate MCP server testing?
16+
17+
MCP servers expose multiple tools that AI assistants can invoke. As these tools evolve, you need reliable automated tests to:
18+
19+
- Verify that each tool responds correctly to valid requests.
20+
- Catch regressions when updating server code or dependencies.
21+
- Validate container startup and communication protocols.
22+
- Ensure compatibility across different environments.
23+
24+
## Understanding Testcontainers
25+
26+
Testcontainers is a Python library that provides lightweight, throwaway instances of Docker containers for testing. Instead of mocking your MCP server, you can spin up the actual Docker container, run tests against it, and tear it down automatically.
27+
28+
![testcontainers alt-text#center](testcontainers.png "Figure 1. Testcontainers Flow")
29+
30+
This approach offers several benefits:
31+
32+
- **Realistic testing**: Tests run against the actual server implementation.
33+
- **Isolation**: Each test run gets a fresh container instance.
34+
- **Reproducibility**: Tests behave consistently across development machines and CI environments.
35+
- **No external dependencies**: Tests don't require a pre-deployed server.
36+
37+
## What you will build
38+
39+
In this Learning Path, you will create an integration test suite that:
40+
41+
1. Starts the Arm MCP server in a Docker container using Testcontainers.
42+
2. Communicates with the server using the MCP stdio transport protocol.
43+
3. Tests multiple MCP tools including container image checking, knowledge base search, and code analysis.
44+
4. Integrates with GitHub Actions for continuous testing.
45+
46+
## What you've accomplished and what's next
47+
48+
In this section:
49+
- You learned what MCP servers are and why automated testing matters.
50+
- You discovered how Testcontainers enable realistic integration testing.
51+
52+
In the next section, you will set up your development environment and install the required dependencies.

0 commit comments

Comments
 (0)