Skip to content

Commit ec4e244

Browse files
slinkydeveloperslinkydeveloper
committed
E2E related skills and scripts
1 parent 1133e80 commit ec4e244

3 files changed

Lines changed: 195 additions & 0 deletions

File tree

‎.claude/skills/run-sdk-tests/SKILL.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
---
2+
name: run-sdk-tests
3+
description: Run the Restate SDK conformance test suite locally against this SDK's Docker image. Use when the user wants to run sdk tests, run conformance tests, or verify an implementation.
4+
user-invocable: true
5+
---
6+
7+
# Running SDK Conformance Tests Locally
8+
9+
## Quick start
10+
11+
```bash
12+
# Build image + run all default suite tests
13+
./.tools/run-sdk-tests.sh
14+
15+
# Skip rebuild if service code hasn't changed
16+
./.tools/run-sdk-tests.sh --skip-build
17+
18+
# Run a single test class
19+
./.tools/run-sdk-tests.sh --skip-build --test-suite=default --test-name=Combinators
20+
```
21+
22+
## Prerequisites
23+
24+
- Java 21+
25+
- Podman or Docker
26+
27+
## What the script does
28+
29+
1. Reads the suite version from `.github/workflows/integration.yaml` (single source of truth)
30+
2. Builds the service image using **Jib** (`./gradlew :test-services:jibBuildTar` → `podman/docker load`)
31+
3. Downloads and caches `sdk-tests.jar` in `tmp/` (version-pinned)
32+
4. Pulls the Restate runtime image explicitly
33+
5. Runs `java -jar sdk-tests.jar run ...` directly on the host
34+
35+
## Jib + Podman note
36+
37+
Jib's `jibBuildTar` creates a tarball which is then loaded via `podman load` / `docker load`. This avoids needing a Docker socket. If `jibDockerBuild` is needed instead, set up the Podman Docker socket compatibility:
38+
39+
```bash
40+
export DOCKER_HOST=unix:///run/user/$(id -u)/podman/podman.sock
41+
```
42+
43+
## Key flags (passed through to the runner)
44+
45+
| Flag | Purpose |
46+
|------|---------|
47+
| `--test-suite=default` | Which suite to run (`default`, `alwaysSuspending`, `threeNodes`, etc.) |
48+
| `--test-name=ClassName` | Run only one test class (requires `--test-suite`) |
49+
| `--exclusions-file=path` | YAML file listing tests to skip |
50+
51+
## Reading results
52+
53+
Logs are written to `tmp/test-report/<timestamp>/<suite>/<TestClass>/`:
54+
- `testRunner.log` — client-side request/response logs
55+
- `runtime_0.log` — Restate server log
56+
- `default-service_0.log` — SDK service container log

‎.claude/skills/update-sdk-test-contracts/SKILL.md‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
---
2+
name: update-sdk-test-contracts
3+
description: Update the SDK test service implementations to match a new version of the e2e conformance contracts. Use when the user says "update sdk tests", "update test contracts", or gives a specific e2e release tag.
4+
user-invocable: true
5+
---
6+
7+
# Updating SDK Test Service Implementations to a New Contract Version
8+
9+
## Step 1: Read the release notes
10+
11+
The user will provide a release tag (e.g. `v2.0`). Fetch the full release notes from GitHub:
12+
13+
```bash
14+
gh release view v2.0 --repo restatedev/e2e
15+
```
16+
17+
The release body describes every contract change — new commands, removed handlers, field names, return value conventions. **Read it entirely before writing any code.**
18+
19+
The release notes document is also available in the e2e repo at `sdk-tests/releases/<version>.md`.
20+
21+
## Step 2: Find the test service implementations
22+
23+
This SDK's test services live under `test-services/`. Search for the relevant files:
24+
25+
```bash
26+
# Find VirtualObjectCommandInterpreter implementation
27+
grep -r "VirtualObjectCommandInterpreter" test-services/ -l
28+
29+
# Find TestUtilsService implementation
30+
grep -r "TestUtilsService" test-services/ -l
31+
```
32+
33+
## Step 3: Implement the changes
34+
35+
For each contract change described in the release notes, update the corresponding implementation file. The release notes specify:
36+
- Exact JSON `type` discriminator strings for new commands (match them exactly — they are case-sensitive)
37+
- Exact JSON field names for request bodies
38+
- Return value conventions (pipe-joining, `ok:`/`err:` prefixes, etc.)
39+
40+
## Step 4: Update the version in the script
41+
42+
After implementing, bump the version in `.github/workflows/integration.yaml` to the new tag. The script `.tools/run-sdk-tests.sh` reads from there automatically.
43+
44+
## Step 5: Build and verify
45+
46+
```bash
47+
# Build and run all new test classes mentioned in the release notes
48+
./.tools/run-sdk-tests.sh --test-suite=default --test-name=<TestClass>
49+
50+
# Or run the full default suite
51+
./.tools/run-sdk-tests.sh
52+
```
53+
54+
All tests must pass before the update is complete.

‎.tools/run-sdk-tests.sh‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
# Run the Restate SDK conformance test suite locally.
5+
#
6+
# Prerequisites:
7+
# - Java 21+
8+
# - podman or docker
9+
# - Gradle wrapper (./gradlew) available
10+
#
11+
# Usage:
12+
# ./.tools/run-sdk-tests.sh # build image + run all default suite tests
13+
# ./.tools/run-sdk-tests.sh --skip-build # skip image build, reuse existing
14+
# ./.tools/run-sdk-tests.sh --test-suite=default --test-name=Combinators
15+
#
16+
# Any unknown flags are passed through to the test runner.
17+
18+
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
19+
20+
# ---- Version: single source of truth is the workflow file ----
21+
SDK_TEST_SUITE_VERSION="$(grep -m1 'uses: restatedev/e2e/sdk-tests@' \
22+
"${REPO_ROOT}/.github/workflows/integration.yaml" | sed 's/.*@//' | tr -d ' ')"
23+
24+
JAR_PATH="${REPO_ROOT}/tmp/sdk-tests-${SDK_TEST_SUITE_VERSION}.jar"
25+
JAR_URL="https://github.com/restatedev/e2e/releases/download/${SDK_TEST_SUITE_VERSION}/sdk-tests.jar"
26+
RESTATE_IMAGE="${RESTATE_CONTAINER_IMAGE:-ghcr.io/restatedev/restate:main}"
27+
REPORT_DIR="${REPO_ROOT}/tmp/test-report"
28+
SERVICE_IMAGE="localhost/e2e-java-test-services:local"
29+
30+
# ---- Detect container runtime ----
31+
if command -v podman &>/dev/null; then
32+
DOCKER=podman
33+
elif command -v docker &>/dev/null; then
34+
DOCKER=docker
35+
else
36+
echo "Error: neither podman nor docker found" >&2
37+
exit 1
38+
fi
39+
40+
# ---- Parse args ----
41+
SKIP_BUILD=false
42+
PASSTHROUGH=()
43+
44+
for arg in "$@"; do
45+
case "$arg" in
46+
--skip-build) SKIP_BUILD=true ;;
47+
*) PASSTHROUGH+=("$arg") ;;
48+
esac
49+
done
50+
51+
# ---- 1. Build the service image via Jib ----
52+
if [ "$SKIP_BUILD" = false ]; then
53+
echo "==> Building ${SERVICE_IMAGE} via Jib..."
54+
cd "${REPO_ROOT}"
55+
./gradlew -Djib.console=plain "-Djib.to.image=${SERVICE_IMAGE}" :test-services:jibBuildTar
56+
"${DOCKER}" load < test-services/build/jib-image.tar
57+
fi
58+
59+
# ---- 2. Download the test suite JAR (cached by version) ----
60+
mkdir -p "$(dirname "$JAR_PATH")"
61+
if [ ! -f "$JAR_PATH" ]; then
62+
echo "==> Downloading sdk-test-suite ${SDK_TEST_SUITE_VERSION}..."
63+
curl -fSL -o "$JAR_PATH" "$JAR_URL"
64+
else
65+
echo "==> Using cached sdk-test-suite ${SDK_TEST_SUITE_VERSION}"
66+
fi
67+
68+
# ---- 3. Pull the Restate runtime image ----
69+
echo "==> Pulling Restate image: ${RESTATE_IMAGE}..."
70+
"${DOCKER}" pull "${RESTATE_IMAGE}"
71+
72+
# ---- 4. Run the tests ----
73+
echo "==> Running integration tests (suite ${SDK_TEST_SUITE_VERSION})..."
74+
rm -rf "${REPORT_DIR}"
75+
mkdir -p "${REPORT_DIR}"
76+
77+
RESTATE_CONTAINER_IMAGE="${RESTATE_IMAGE}" java -jar "${JAR_PATH}" run \
78+
--sequential \
79+
--image-pull-policy=CACHED \
80+
--report-dir="${REPORT_DIR}" \
81+
--service-container-image="${SERVICE_IMAGE}" \
82+
"${PASSTHROUGH[@]+"${PASSTHROUGH[@]}"}"
83+
84+
echo ""
85+
echo "==> Done. Report: ${REPORT_DIR}"

0 commit comments

Comments
 (0)