Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions .github/workflows/integration.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Integration Tests

on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
verify:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4

- uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
version: 10

- name: Use Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
cache: pnpm

- name: Install mcp-recorder
run: pip install -r integration/requirements.txt

- name: Install dependencies
run: pnpm install

- name: Build
run: pnpm run build

- name: Start MCP server
run: FIRECRAWL_API_KEY=test HTTP_STREAMABLE_SERVER=true node dist/index.js &

- name: Wait for server
run: |
for i in $(seq 1 10); do
curl -sf http://localhost:3000/health && exit 0
sleep 1
done
echo "Server failed to start" && exit 1

- name: Verify protocol cassette
run: |
mcp-recorder verify \
--cassette integration/cassettes/protocol_and_errors.json \
--target http://localhost:3000 \
--verbose
56 changes: 56 additions & 0 deletions integration/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Integration Tests

Snapshot-based regression tests for the MCP protocol surface. A golden cassette records the `initialize` handshake, all tool schemas via `tools/list`, and error handling. CI verifies that every PR still matches. If a tool is renamed, removed, or has its schema changed, the diff shows exactly what broke.

Uses [mcp-recorder](https://github.com/devhelmhq/mcp-recorder) for recording and verification.

## What's tested

| Cassette | What it guards |
|---|---|
| `protocol_and_errors.json` | Protocol version, capabilities, all tool schemas, error responses |

## Setup

```bash
pip install -r integration/requirements.txt
```

## Verify locally

```bash
pnpm build
FIRECRAWL_API_KEY=test HTTP_STREAMABLE_SERVER=true node dist/index.js &

mcp-recorder verify \
--cassette integration/cassettes/protocol_and_errors.json \
--target http://localhost:3000 \
--verbose
```

All 4 interactions should pass. Kill the server when done.

## Update cassettes after intentional changes

When you've changed a tool schema or added a new tool, update the cassette with one command:

```bash
# Start the server, then:
mcp-recorder verify \
--cassette integration/cassettes/protocol_and_errors.json \
--target http://localhost:3000 \
--update
```

This replays the recorded requests, accepts the new responses, and writes them back to the cassette. Commit the updated cassette with your PR — the diff makes the schema change visible in review.

## Add new test scenarios

Edit `scenarios.yml` and re-record:

```bash
mcp-recorder record-scenarios integration/scenarios.yml \
--output-dir integration/cassettes
```

See the [mcp-recorder docs](https://github.com/devhelmhq/mcp-recorder) for supported actions.
Loading