Skip to content

Commit 2c0ac13

Browse files
SodawyxSodawyx
authored
Merge pull request #98 from Serverless-Devs/feat/e2e-oidc-ci
ci: add e2e workflow with OIDC keyless auth for Alibaba Cloud
2 parents aa2b77f + f7111f0 commit 2c0ac13

2 files changed

Lines changed: 259 additions & 0 deletions

File tree

‎.github/workflows/e2e.yml‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
name: E2E Tests
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
branches: [main]
8+
workflow_dispatch:
9+
10+
# OIDC token generation requires id-token:write permission
11+
permissions:
12+
id-token: write
13+
contents: read
14+
15+
# Prevent parallel e2e runs from conflicting on shared test resources
16+
concurrency:
17+
group: e2e-${{ github.ref }}
18+
cancel-in-progress: true
19+
20+
jobs:
21+
e2e:
22+
runs-on: ubuntu-latest
23+
# Skip e2e for PRs from forks (they cannot access OIDC secrets)
24+
if: >-
25+
github.event_name == 'workflow_dispatch' ||
26+
github.event_name == 'push' ||
27+
(github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository)
28+
strategy:
29+
matrix:
30+
python-version: ['3.10']
31+
32+
steps:
33+
- name: Checkout code
34+
uses: actions/checkout@v4
35+
36+
- name: Set up Python ${{ matrix.python-version }}
37+
uses: actions/setup-python@v5
38+
with:
39+
python-version: ${{ matrix.python-version }}
40+
41+
- name: Install dependencies
42+
run: |
43+
make setup PYTHON_VERSION=${{ matrix.python-version }}
44+
45+
# Obtain temporary Alibaba Cloud credentials via OIDC (keyless)
46+
# This action exchanges the GitHub OIDC token for temporary AK/SK/SecurityToken
47+
# and exports them as ALIBABA_CLOUD_ACCESS_KEY_ID, ALIBABA_CLOUD_ACCESS_KEY_SECRET,
48+
# ALIBABA_CLOUD_SECURITY_TOKEN environment variables.
49+
- name: Configure Alibaba Cloud credentials (OIDC)
50+
uses: aliyun/configure-aliyun-credentials-action@v1
51+
with:
52+
role-to-assume: ${{ secrets.ALIBABA_CLOUD_OIDC_ROLE_ARN }}
53+
oidc-provider-arn: ${{ secrets.ALIBABA_CLOUD_OIDC_PROVIDER_ARN }}
54+
role-session-name: agentrun-e2e-${{ github.run_id }}
55+
role-session-expiration: 3600
56+
57+
- name: Run E2E tests
58+
env:
59+
# Credentials are auto-injected by configure-aliyun-credentials-action:
60+
# ALIBABA_CLOUD_ACCESS_KEY_ID
61+
# ALIBABA_CLOUD_ACCESS_KEY_SECRET
62+
# ALIBABA_CLOUD_SECURITY_TOKEN
63+
AGENTRUN_ACCOUNT_ID: ${{ secrets.AGENTRUN_ACCOUNT_ID }}
64+
AGENTRUN_REGION: ${{ secrets.AGENTRUN_REGION }}
65+
AGENTRUN_CONTROL_ENDPOINT: ${{ secrets.AGENTRUN_CONTROL_ENDPOINT }}
66+
AGENTRUN_DATA_ENDPOINT: ${{ secrets.AGENTRUN_DATA_ENDPOINT }}
67+
# API_KEY and AGENTRUN_TEST_WORKSPACE_ID are intentionally NOT set.
68+
# Tests requiring them are excluded via --ignore and -k filters below.
69+
run: |
70+
uv run pytest tests/e2e/ -v --tb=short \
71+
--ignore=tests/e2e/integration \
72+
--ignore=tests/e2e/test_agent_ruintime.py \
73+
--ignore=tests/e2e/test_workspace_id.py \
74+
--ignore=tests/e2e/test_sandbox_browser.py \
75+
--ignore=tests/e2e/test_sandbox_code_interpreter.py \
76+
-k "not (invoke or with_credential or model_proxy or process_get or delete_sandbox or delete_nonexistent_sandbox or connect_nonexistent or connect_sandbox_async or sandbox_lifecycle or connect_with_wrong_template or template_validation_code_interpreter_network)"
77+
78+
- name: E2E Summary
79+
if: always()
80+
run: |
81+
echo "## E2E Test Results" >> $GITHUB_STEP_SUMMARY
82+
echo "" >> $GITHUB_STEP_SUMMARY
83+
echo "- **Trigger:** ${{ github.event_name }}" >> $GITHUB_STEP_SUMMARY
84+
echo "- **Branch:** ${{ github.ref_name }}" >> $GITHUB_STEP_SUMMARY
85+
echo "- **Commit:** ${{ github.sha }}" >> $GITHUB_STEP_SUMMARY
86+
echo "- **Auth:** OIDC keyless (temporary credentials)" >> $GITHUB_STEP_SUMMARY

‎docs/e2e-oidc-setup.md‎

Lines changed: 173 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,173 @@
1+
# E2E Tests: OIDC Keyless Authentication Setup
2+
3+
This guide explains how to configure GitHub Actions OIDC (OpenID Connect) for
4+
running E2E tests against Alibaba Cloud without storing long-lived AK/SK credentials.
5+
6+
## How It Works
7+
8+
```
9+
GitHub Actions ──OIDC token──> Alibaba Cloud RAM (OIDC Provider)
10+
│
11+
▼
12+
STS AssumeRoleWithOIDC
13+
│
14+
▼
15+
Temporary AK/SK/SecurityToken
16+
│
17+
▼
18+
E2E tests use temp creds
19+
```
20+
21+
1. GitHub Actions generates an OIDC token containing repository and workflow metadata.
22+
2. The `aliyun/configure-aliyun-credentials-action` exchanges that token with Alibaba Cloud STS.
23+
3. STS returns temporary credentials (valid for 1 hour) scoped to a specific RAM Role.
24+
4. E2E tests use those credentials — no permanent secrets stored anywhere.
25+
26+
## Prerequisites
27+
28+
- Alibaba Cloud account with RAM admin access
29+
- GitHub repository admin access (to configure secrets)
30+
31+
## Step 1: Create OIDC Identity Provider in RAM
32+
33+
1. Go to [RAM Console > SSO Management > OIDC](https://ram.console.aliyun.com/providers/oidc)
34+
2. Click **Create OIDC Provider**
35+
3. Fill in the form:
36+
37+
| Field | Value |
38+
|-------|-------|
39+
| Provider Name | `github-actions` |
40+
| Issuer URL | `https://token.actions.githubusercontent.com` |
41+
| Client ID (Audience) | `sts.aliyuncs.com` |
42+
| Description | GitHub Actions OIDC for agentrun-sdk e2e tests |
43+
44+
4. Click **OK** to create.
45+
5. Copy the **Provider ARN**, it looks like:
46+
```
47+
acs:ram::<ACCOUNT_ID>:oidc-provider/github-actions
48+
```
49+
50+
## Step 2: Create a RAM Role for E2E Tests
51+
52+
1. Go to [RAM Console > Identities > Roles](https://ram.console.aliyun.com/roles)
53+
2. Click **Create Role** > **IdP** > **OIDC**
54+
3. Fill in the form:
55+
56+
| Field | Value |
57+
|-------|-------|
58+
| Role Name | `github-actions-e2e` |
59+
| Select OIDC Provider | `github-actions` (created in Step 1) |
60+
| Condition | See trust policy below |
61+
62+
4. Use this **Trust Policy** (edit the role after creation if the console doesn't allow full customization):
63+
64+
```json
65+
{
66+
"Statement": [
67+
{
68+
"Action": "sts:AssumeRoleWithOIDC",
69+
"Condition": {
70+
"StringEquals": {
71+
"oidc:aud": "sts.aliyuncs.com"
72+
},
73+
"StringLike": {
74+
"oidc:sub": "repo:Serverless-Devs/agentrun-sdk-python:*"
75+
}
76+
},
77+
"Effect": "Allow",
78+
"Principal": {
79+
"Federated": [
80+
"acs:ram::<ACCOUNT_ID>:oidc-provider/github-actions"
81+
]
82+
}
83+
}
84+
],
85+
"Version": "1"
86+
}
87+
```
88+
89+
> Replace `<ACCOUNT_ID>` with your Alibaba Cloud account ID.
90+
>
91+
> The `oidc:sub` condition restricts access to this specific repository.
92+
> You can narrow it further: `repo:Serverless-Devs/agentrun-sdk-python:ref:refs/heads/main`
93+
> would limit to the main branch only.
94+
95+
5. Copy the **Role ARN**, it looks like:
96+
```
97+
acs:ram::<ACCOUNT_ID>:role/github-actions-e2e
98+
```
99+
100+
## Step 3: Attach Permission Policy to the Role
101+
102+
The RAM Role needs permissions to call AgentRun / Function Compute APIs
103+
used by the E2E tests. Create a custom policy or attach existing ones:
104+
105+
**Recommended minimum permissions:**
106+
- `AliyunFCReadOnlyAccess` (read-only FC access for test verification)
107+
- A custom policy for AgentRun API operations used in tests
108+
109+
Example custom policy:
110+
```json
111+
{
112+
"Version": "1",
113+
"Statement": [
114+
{
115+
"Effect": "Allow",
116+
"Action": [
117+
"fc:*"
118+
],
119+
"Resource": [
120+
"acs:fc:<REGION>:<ACCOUNT_ID>:*"
121+
]
122+
}
123+
]
124+
}
125+
```
126+
127+
> Adjust the `Action` and `Resource` scope based on what your E2E tests actually call.
128+
> Principle of least privilege: grant only what the tests need.
129+
130+
## Step 4: Configure GitHub Secrets
131+
132+
Go to **GitHub repo > Settings > Secrets and variables > Actions** and add:
133+
134+
### OIDC Secrets (required for authentication)
135+
136+
| Secret Name | Description | Example |
137+
|-------------|-------------|---------|
138+
| `ALIBABA_CLOUD_OIDC_PROVIDER_ARN` | OIDC Provider ARN from Step 1 | `acs:ram::1234567890:oidc-provider/github-actions` |
139+
| `ALIBABA_CLOUD_OIDC_ROLE_ARN` | RAM Role ARN from Step 2 | `acs:ram::1234567890:role/github-actions-e2e` |
140+
141+
### Test Configuration Secrets (required for e2e tests)
142+
143+
| Secret Name | Description | Example |
144+
|-------------|-------------|---------|
145+
| `AGENTRUN_ACCOUNT_ID` | Alibaba Cloud account ID | `1234567890` |
146+
| `AGENTRUN_REGION` | Region for test resources | `cn-hangzhou` |
147+
| `AGENTRUN_CONTROL_ENDPOINT` | AgentRun control API endpoint | `https://agentrun.cn-hangzhou.aliyuncs.com` |
148+
| `AGENTRUN_DATA_ENDPOINT` | AgentRun data API endpoint | `https://1234567890.agentrun-data.cn-hangzhou.aliyuncs.com` |
149+
150+
151+
> `API_KEY` and `AGENTRUN_TEST_WORKSPACE_ID` use hardcoded placeholders in the
152+
> workflow and do not need to be configured as secrets.
153+
154+
## Step 5: Verify
155+
156+
1. Push a commit to `main` or trigger the workflow manually via **Actions > E2E Tests > Run workflow**.
157+
2. Check the workflow run — the "Configure Alibaba Cloud credentials (OIDC)" step should succeed.
158+
3. E2E tests should run with temporary credentials.
159+
160+
## Troubleshooting
161+
162+
### "AssumeRoleWithOIDC failed"
163+
- Verify the OIDC Provider issuer URL is exactly `https://token.actions.githubusercontent.com`
164+
- Verify the Role trust policy `oidc:sub` matches your repo: `repo:Serverless-Devs/agentrun-sdk-python:*`
165+
- Check the audience is `sts.aliyuncs.com`
166+
167+
### "Permission denied" during tests
168+
- The RAM Role needs the right policies attached (Step 3)
169+
- Check if the region in the policy matches `AGENTRUN_REGION`
170+
171+
### E2E tests skip on fork PRs
172+
- This is intentional: fork PRs cannot access OIDC secrets
173+
- Only PRs from the same repo, pushes to main, and manual triggers run e2e

0 commit comments

Comments
 (0)