Skip to content

Draft: Add mechanism to manually run tests on PRs from forks #8227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Ephem wants to merge 3 commits into
base: main
Choose a base branch
from
+346 −0
Open

Draft: Add mechanism to manually run tests on PRs from forks #8227

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6c69227
Implement integration tests for forks
Ephem Apr 2, 2026
b5bf7d5
Add integration-tests-gate to be able to share common Required check
Ephem Apr 2, 2026
5fa2191
Remove test branch from workflow as that method of testing wont work
Ephem Apr 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
313 changes: 313 additions & 0 deletions .github/workflows/ci-fork-integration.yml
View file
Open in desktop
Copy link
Copy Markdown
Member Author

@Ephem Ephem Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's plenty of opportunity to de-duplicate parts of this with ci.yml later if we want, but it made sense to me to keep them separate for now to allow for easier reviewing and testing.

Original file line number Diff line number Diff line change
@@ -0,0 +1,313 @@
# Note: There is a copy of this workflow in ci.yml that is
# used for triggering integration tests on forked PRs. If you tweak this
# workflow, see if that also needs to be updated.

name: 'CI: Fork Integration Tests'

on:
pull_request_target:
types: [labeled, synchronize]
branches:
- main

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}
cancel-in-progress: true

jobs:
remove-label-on-push:
if: >-
${{ github.event.action == 'synchronize'
&& github.event.pull_request.head.repo.full_name != github.repository }}
runs-on: 'blacksmith-2vcpu-ubuntu-2204'
permissions:
# Needed for removing the "safe-to-test" label
pull-requests: write
name: Remove safe-to-test label
steps:
- name: Remove safe-to-test label
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
gh pr edit "$PR_NUMBER" --repo "$REPO" --remove-label "safe-to-test" || true

check-label-permissions:
if: >-
${{ github.event.action == 'labeled'
&& github.event.label.name == 'safe-to-test'
&& github.event.pull_request.head.repo.full_name != github.repository }}
runs-on: 'blacksmith-2vcpu-ubuntu-2204'
permissions:
pull-requests: write
name: Check Label Sender Permissions
steps:
- name: Verify label sender has write access
id: checkAccess
uses: actions-cool/check-user-permission@v2
with:
require: write
Copy link
Copy Markdown
Contributor

@dominic-clerk dominic-clerk Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about doing it like

javascript/.github/workflows/release.yml

Line 276 in b80eafb

- name: Limit action to Clerk members
where we limit to our team members. Possibly checking both if we have team members without write access.

I'd like the security gate to have as little 3rd party as possible and the way we do it in the release workflow uses only dependencies provided by GitHub.

I'm also wary of bots or 3rd party integrations that might have write access (dependabot for example) that might be hijacked... somehow. This is the paranoia speaking :)

username: ${{ github.event.sender.login }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Comment on lines +49 to +56
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

git ls-files .github/workflows/ | head -20

Repository: clerk/javascript

Length of output: 494

🏁 Script executed:

cat -n .github/workflows/ci-fork-integration.yml | head -80

Repository: clerk/javascript

Length of output: 3546

🏁 Script executed:

rg "actions-cool/check-user-permission" .github/workflows/

Repository: clerk/javascript

Length of output: 230

🏁 Script executed:

rg "@v[0-9]+" .github/workflows/ | head -30

Repository: clerk/javascript

Length of output: 2253

🌐 Web query:

GitHub Actions pinning to commit SHA security best practice pull_request_target

💡 Result:

Yes, pinning GitHub Actions to full-length commit SHAs is a security best practice, especially critical for workflows using the pull_request_target trigger. Pinning to commit SHAs creates immutable references, preventing supply chain attacks where mutable tags like @v4 or @main are hijacked to point to malicious code. GitHub official documentation explicitly recommends: "Pin actions to a full-length commit SHA. Pinning an action to a full-length commit SHA is currently the only way to use an action as an immutable release." This mitigates risks from bad actors adding backdoors, as they would need a SHA-1 collision. For pull_request_target specifically, this trigger runs workflows in the base repository context with access to secrets and write permissions (via GITHUB_TOKEN), even for forks. Without pinning, untrusted actions could execute malicious code with elevated privileges, leading to RCE, secret exfiltration, or repository compromise. Best practices include: - Always pin to full SHA (40 characters), e.g., uses: actions/checkout@692973e # v4.1.7 - Verify SHA from the official repository (not forks) - Add comments for readability (e.g., # v4.1.7) - Use GitHub policies to enforce SHA pinning at repo/org level - For pull_request_target: Never checkout untrusted PR code (github.event.pull_request.head.sha); use base ref only. Restrict to trusted repos/forks, limit permissions to read-only, split workflows (unprivileged pull_request -> privileged via artifacts). Tools like pin-github-action or StepSecurity automate pinning. GitHub provides org/repo policies for enforcement and Dependabot for updates. Recent changes (2025) force pull_request_target to use default branch workflows, aiding fixes but not eliminating need for pinning.

Citations:

Pin actions-cool/check-user-permission to a full commit SHA.

This step gates a pull_request_target workflow (which runs with elevated privileges and repo credentials) using a mutable third-party tag. GitHub security guidance requires pinning actions to immutable commit SHAs to prevent supply chain attacks. Replace @v2 with the full 40-character commit SHA and add a comment noting the version for reference (e.g., # v2.x.x).

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci-fork-integration.yml around lines 51 - 58, Replace the
mutable tag for the GitHub Action used in the checkAccess step with an immutable
commit SHA: change uses: actions-cool/check-user-permission@v2 to uses:
actions-cool/check-user-permission@<40-char-commit-sha> (keep the step id
"checkAccess" and other fields unchanged) and add an inline comment above or
beside the uses line noting the original version tag for reference (e.g., #
v2.x.x).

- name: Remove unauthorized label and fail
if: ${{ steps.checkAccess.outputs.require-result == 'false' }}
Copy link
Copy Markdown
Contributor

@dominic-clerk dominic-clerk Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs claim true | false are the only results, but I wonder what happens if it errors out for whatever reason? Let's fail on "not true" instead of "false".

Suggested change
if: ${{ steps.checkAccess.outputs.require-result == 'false' }}
if: ${{ steps.checkAccess.outputs.require-result != 'true' }}

env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
REPO: ${{ github.repository }}
run: |
echo "::error::User ${{ github.event.sender.login }} added safe-to-test but only has '${{ steps.checkAccess.outputs.user-permission }}' permission (write required)."
gh pr edit "$PR_NUMBER" --repo "$REPO" --remove-label "safe-to-test" || true
exit 1

integration-tests:
needs: [check-label-permissions]
if: >-
${{ github.event.action == 'labeled'
&& github.event.label.name == 'safe-to-test'
&& github.event.pull_request.head.repo.full_name != github.repository
&& github.event.pull_request.draft == false }}
name: Integration Tests (${{ matrix.test-name }}, ${{ matrix.test-project }}${{ matrix.next-version && format(', {0}', matrix.next-version) || '' }})
permissions:
contents: read
actions: write
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
defaults:
run:
shell: bash
timeout-minutes: ${{ vars.TIMEOUT_MINUTES_LONG && fromJSON(vars.TIMEOUT_MINUTES_LONG) || 15 }}

strategy:
fail-fast: false
# This needs to be kept in sync with ci.yml
matrix:
test-name:
[
'generic',
'express',
'fastify',
'ap-flows',
'localhost',
'sessions',
'sessions:staging',
'handshake',
'handshake:staging',
'astro',
'tanstack-react-start',
'vue',
'nuxt',
'react-router',
'custom',
'hono',
'chrome-extension',
]
test-project: ['chrome']
include:
- test-name: 'billing'
test-project: 'chrome'
- test-name: 'machine'
test-project: 'chrome'
- test-name: 'nextjs'
test-project: 'chrome'
next-version: '15'
- test-name: 'nextjs'
test-project: 'chrome'
next-version: '16'
- test-name: 'quickstart'
test-project: 'chrome'
next-version: '15'
- test-name: 'quickstart'
test-project: 'chrome'
next-version: '16'
- test-name: 'cache-components'
test-project: 'chrome'
next-version: '16'

steps:
# Phase 1: Check out the base branch so that the composite action
# at .github/actions/init-blacksmith comes from trusted code.
- name: Checkout base branch
uses: actions/checkout@v4
with:
fetch-depth: 1
fetch-tags: false
filter: 'blob:none'
show-progress: false

- name: Setup (from base branch)
id: config
uses: ./.github/actions/init-blacksmith
with:
turbo-signature: ${{ secrets.TURBO_REMOTE_CACHE_SIGNATURE_KEY }}
Copy link
Copy Markdown
Contributor

@dominic-clerk dominic-clerk Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a separate cache entirely to make cache poisoning completely impossible? I know it shouldn't be possible already and would require a hash collision but it would make it completely impossible and protect us against a bug in turbo itself.

Copy link
Copy Markdown
Member Author

@Ephem Ephem Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also just not use the turbo cache. These runs don't happen often so I think it's fine if they take extra time.

Copy link
Copy Markdown
Contributor

@dominic-clerk dominic-clerk Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even better!

turbo-team: ${{ vars.TURBO_TEAM }}
turbo-token: ${{ secrets.TURBO_TOKEN }}
playwright-enabled: true

# Phase 2: Overlay the fork code at the exact SHA frozen in the event payload.
# This SHA was captured when the label was applied and cannot be changed by
# subsequent pushes to the fork branch.
- name: Checkout fork PR (SHA-pinned)
uses: actions/checkout@v4
with:
repository: ${{ github.event.pull_request.head.repo.full_name }}
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 1
fetch-tags: false
filter: 'blob:none'
show-progress: false
clean: false

- name: Re-install dependencies for fork code
run: pnpm install
Copy link
Copy Markdown
Contributor

@dominic-clerk dominic-clerk Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case the review misses a change in config that had allowed a post-install script

Suggested change
run: pnpm install
run: pnpm install --ignore-scripts

Comment on lines +142 to +166
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# First, let's locate and examine the init-blacksmith action
find . -path "*/.github/actions/init-blacksmith/action.yml" -type f

Repository: clerk/javascript

Length of output: 104

🏁 Script executed:

#!/bin/bash
# Also check the structure of .github directory
ls -la .github/actions/ 2>/dev/null || echo "actions directory not found"

Repository: clerk/javascript

Length of output: 415

🏁 Script executed:

#!/bin/bash
# Let's look at the ci-fork-integration.yml file in context
head -200 .github/workflows/ci-fork-integration.yml | tail -100

Repository: clerk/javascript

Length of output: 3571

🏁 Script executed:

#!/bin/bash
# Examine the init-blacksmith action
cat .github/actions/init-blacksmith/action.yml

Repository: clerk/javascript

Length of output: 4592

🏁 Script executed:

#!/bin/bash
# Let's verify the context around the fork checkout and see if there are any mitigation measures
grep -A 30 "Checkout fork PR" .github/workflows/ci-fork-integration.yml | head -40

Repository: clerk/javascript

Length of output: 1100

🏁 Script executed:

#!/bin/bash
# Let's check if there are any other security-related configurations in the workflow
grep -i "secret\|credential\|token\|clean" .github/workflows/ci-fork-integration.yml

Repository: clerk/javascript

Length of output: 777

Run the fork checkout in a fresh workspace.

The init-blacksmith action exports TURBO_TOKEN, TURBO_REMOTE_CACHE_SIGNATURE_KEY, and other environment variables via core.exportVariable(), which persist throughout the job. With clean: false on the fork checkout, these variables—including secrets—remain accessible to the untrusted fork code. Fork scripts (like pnpm install postinstall hooks) can access and exfiltrate TURBO_TOKEN. Use a separate job or checkout path and clear credentials before checking out fork code.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci-fork-integration.yml around lines 144 - 168, The job
currently runs the init-blacksmith action (init-blacksmith) which exports
secrets into the environment, then performs the fork checkout step
(actions/checkout@v4 with clean: false) and runs pnpm install; to prevent leaked
secrets, run the fork checkout in an isolated context: either move the fork
checkout and subsequent commands (the "Checkout fork PR (SHA-pinned)" step and
"Re-install dependencies for fork code"/pnpm install) into a separate job that
does NOT call init-blacksmith, or perform the checkout into a separate
workspace/path and explicitly clear/unset exported variables (unset TURBO_TOKEN,
TURBO_REMOTE_CACHE_SIGNATURE_KEY and any other exported envs from
init-blacksmith) before running pnpm install; also remove or change clean: false
so the checkout does not inherit the runner workspace state if keeping the same
job.


- name: Verify jq is installed
shell: bash
run: |
if ! command -v jq &> /dev/null; then
echo "jq not found, installing..."
sudo apt-get update && sudo apt-get install -y jq
fi
jq --version

- name: Validate turbo task
env:
E2E_APP_CLERK_JS_DIR: ${{runner.temp}}
E2E_APP_CLERK_UI_DIR: ${{runner.temp}}
E2E_CLERK_JS_VERSION: 'latest'
E2E_CLERK_UI_VERSION: 'latest'
E2E_NEXTJS_VERSION: ${{ matrix.next-version }}
E2E_PROJECT: ${{ matrix.test-project }}
INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
run: |
TASK_NAME="test:integration:${{ matrix.test-name }}"
TURBO_STDERR=$(mktemp)
if ! TURBO_JSON=$(pnpm turbo run "$TASK_NAME" --dry=json 2>"$TURBO_STDERR"); then
echo "::error::Turbo task '$TASK_NAME' failed validation"
cat "$TURBO_STDERR"
exit 1
fi

if ! TASK_COUNT=$(jq -er '.tasks | length' <<< "$TURBO_JSON"); then
echo "::error::Turbo task '$TASK_NAME' returned invalid JSON or missing .tasks"
printf '%s\n' "$TURBO_JSON"
exit 1
fi

if [ "$TASK_COUNT" -eq 0 ]; then
echo "::error::Turbo task '$TASK_NAME' returned 0 tasks"
exit 1
fi

echo "Task '$TASK_NAME' validated ($TASK_COUNT tasks in graph)"

- name: Build packages
run: pnpm turbo build $TURBO_ARGS --only

- name: Publish to local registry
run: pkglab pub --force

- name: Edit .npmrc [link-workspace-packages=false]
run: sed -i -E 's/link-workspace-packages=(deep|true)/link-workspace-packages=false/' .npmrc

- name: Install @clerk/clerk-js in os temp
working-directory: ${{runner.temp}}
run: |
mkdir clerk-js && cd clerk-js
pnpm init
pkglab add @clerk/clerk-js

- name: Install @clerk/ui in os temp
working-directory: ${{runner.temp}}
run: |
mkdir clerk-ui && cd clerk-ui
pnpm init
pkglab add @clerk/ui

- name: Copy components @clerk/astro
if: ${{ matrix.test-name == 'astro' }}
run: cd packages/astro && pnpm copy:components

- name: Write all ENV certificates to files in integration/certs
uses: actions/github-script@v7
env:
INTEGRATION_CERTS: '${{secrets.INTEGRATION_CERTS}}'
INTEGRATION_ROOT_CA: '${{secrets.INTEGRATION_ROOT_CA}}'
with:
script: |
const fs = require('fs');
const path = require('path');
const rootCa = process.env.INTEGRATION_ROOT_CA;
console.log('rootCa', rootCa);
fs.writeFileSync(path.join(process.env.GITHUB_WORKSPACE, 'integration/certs', 'rootCA.pem'), rootCa);
const certs = JSON.parse(process.env.INTEGRATION_CERTS);
for (const [name, cert] of Object.entries(certs)) {
fs.writeFileSync(path.join(process.env.GITHUB_WORKSPACE, 'integration/certs', name), cert);
}

Comment on lines +235 to +251
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Remove the root CA debug log.

Line 247 prints INTEGRATION_ROOT_CA into the workflow logs. Even if masking helps, secret material should not be emitted at all.

🛡️ Minimal fix 
-            console.log('rootCa', rootCa);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Write all ENV certificates to files in integration/certs
uses: actions/github-script@v7
env:
INTEGRATION_CERTS: '${{secrets.INTEGRATION_CERTS}}'
INTEGRATION_ROOT_CA: '${{secrets.INTEGRATION_ROOT_CA}}'
with:
script: |
const fs = require('fs');
const path = require('path');
const rootCa = process.env.INTEGRATION_ROOT_CA;
console.log('rootCa', rootCa);
fs.writeFileSync(path.join(process.env.GITHUB_WORKSPACE, 'integration/certs', 'rootCA.pem'), rootCa);
const certs = JSON.parse(process.env.INTEGRATION_CERTS);
for (const [name, cert] of Object.entries(certs)) {
fs.writeFileSync(path.join(process.env.GITHUB_WORKSPACE, 'integration/certs', name), cert);
}
- name: Write all ENV certificates to files in integration/certs
uses: actions/github-script@v7
env:
INTEGRATION_CERTS: '${{secrets.INTEGRATION_CERTS}}'
INTEGRATION_ROOT_CA: '${{secrets.INTEGRATION_ROOT_CA}}'
with:
script: |
const fs = require('fs');
const path = require('path');
const rootCa = process.env.INTEGRATION_ROOT_CA;
fs.writeFileSync(path.join(process.env.GITHUB_WORKSPACE, 'integration/certs', 'rootCA.pem'), rootCa);
const certs = JSON.parse(process.env.INTEGRATION_CERTS);
for (const [name, cert] of Object.entries(certs)) {
fs.writeFileSync(path.join(process.env.GITHUB_WORKSPACE, 'integration/certs', name), cert);
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci-fork-integration.yml around lines 237 - 253, The
workflow currently logs the secret via the console.log('rootCa', rootCa)
statement; remove that console.log so the INTEGRATION_ROOT_CA value is never
emitted to workflow logs, keeping the rest of the script that reads
process.env.INTEGRATION_ROOT_CA and writes files using
fs.writeFileSync(path.join(process.env.GITHUB_WORKSPACE, 'integration/certs',
'rootCA.pem'), rootCa) intact.

- name: LS certs
working-directory: ./integration/certs
run: ls -la && pwd

- name: Run Integration Tests
id: integration-tests
timeout-minutes: 25
run: pnpm turbo test:integration:${{ matrix.test-name }} $TURBO_ARGS
env:
E2E_DEBUG: '1'
E2E_APP_CLERK_JS_DIR: ${{runner.temp}}
E2E_APP_CLERK_UI_DIR: ${{runner.temp}}
E2E_CLERK_JS_VERSION: 'latest'
E2E_CLERK_UI_VERSION: 'latest'
E2E_NEXTJS_VERSION: ${{ matrix.next-version }}
E2E_PROJECT: ${{ matrix.test-project }}
E2E_CLERK_ENCRYPTION_KEY: ${{ matrix.clerk-encryption-key }}
INTEGRATION_INSTANCE_KEYS: ${{ secrets.INTEGRATION_INSTANCE_KEYS }}
NODE_EXTRA_CA_CERTS: ${{ github.workspace }}/integration/certs/rootCA.pem
VERCEL_AUTOMATION_BYPASS_SECRET: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}
Comment on lines +256 to +271
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/ci-fork-integration.yml

Repository: clerk/javascript

Length of output: 13598

🏁 Script executed:

grep -n "clerk-encryption-key" .github/workflows/ci.yml | head -20

Repository: clerk/javascript

Length of output: 134

🏁 Script executed:

sed -n '1,150p' .github/workflows/ci.yml | grep -A 100 "strategy:" | head -80

Repository: clerk/javascript

Length of output: 42

🏁 Script executed:

sed -n '80,200p' .github/workflows/ci.yml

Repository: clerk/javascript

Length of output: 3860

🏁 Script executed:

sed -n '200,500p' .github/workflows/ci.yml | grep -B 5 -A 120 "integration-tests:"

Repository: clerk/javascript

Length of output: 4504

🏁 Script executed:

sed -n '400,500p' .github/workflows/ci.yml

Repository: clerk/javascript

Length of output: 3996

Add clerk-encryption-key to the matrix or remove this env var.

E2E_CLERK_ENCRYPTION_KEY references ${{ matrix.clerk-encryption-key }} (line 270), but clerk-encryption-key is not defined in any matrix entry or include block. This causes the environment variable to be empty at runtime. The same issue exists in ci.yml.

🧰 Tools
🪛 actionlint (1.7.11)

[error] 270-270: property "clerk-encryption-key" is not defined in object type {next-version: number; test-name: string; test-project: string}

(expression)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/ci-fork-integration.yml around lines 258 - 273, The env
var E2E_CLERK_ENCRYPTION_KEY in the "Run Integration Tests" job is populated
from matrix.clerk-encryption-key but that matrix key does not exist; either add
clerk-encryption-key to the matrix/include entries (set a value per matrix
entry) so matrix.clerk-encryption-key resolves, or remove
E2E_CLERK_ENCRYPTION_KEY from the env block; apply the same fix to the
corresponding job in ci.yml to keep both workflows consistent (search for "Run
Integration Tests" or the job id integration-tests and the env name
E2E_CLERK_ENCRYPTION_KEY).


- name: Sanitize artifact name
if: ${{ cancelled() || failure() }}
id: sanitize
run: |
SANITIZED="${TEST_NAME//:/-}"
echo "artifact-suffix=${SANITIZED}" >> $GITHUB_OUTPUT
env:
TEST_NAME: ${{ matrix.test-name }}

- name: Upload test-results
if: ${{ cancelled() || failure() }}
uses: actions/upload-artifact@v4
with:
name: playwright-traces-fork-${{ github.run_id }}-${{ github.run_attempt }}-${{ steps.sanitize.outputs.artifact-suffix }}${{ matrix.next-version && format('-next{0}', matrix.next-version) || '' }}
path: test-results
retention-days: 1

integration-tests-gate:
needs: [integration-tests]
if: ${{ always() && needs.integration-tests.result != 'skipped' }}
runs-on: 'blacksmith-2vcpu-ubuntu-2204'
permissions:
statuses: write
name: Fork Integration Tests Gate
steps:
- name: Report integration test status
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RESULT: ${{ needs.integration-tests.result }}
SHA: ${{ github.event.pull_request.head.sha }}
REPO: ${{ github.repository }}
run: |
if [ "$RESULT" = "success" ]; then
STATE="success"
else
STATE="failure"
fi
gh api "repos/$REPO/statuses/$SHA" \
-f state="$STATE" \
-f context="integration-tests" \
-f description="Fork integration tests: $RESULT"
33 changes: 33 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# Note: There is a copy of this workflow in ci-fork-integration.yml that is
# used for triggering integration tests on forked PRs. If you tweak this
# workflow, see if that also needs to be updated.

name: CI

on:
Expand Down Expand Up @@ -289,6 +293,7 @@ jobs:

strategy:
fail-fast: false
# This needs to be kept in sync with ci-fork-integration.yml
matrix:
test-name:
[
Expand Down Expand Up @@ -476,6 +481,34 @@ jobs:
path: test-results
retention-days: 1

integration-tests-gate:
needs: [integration-tests]
if: >-
${{ always()
&& github.event_name == 'pull_request'
&& needs.integration-tests.result != 'skipped' }}
runs-on: "blacksmith-2vcpu-ubuntu-2204"
permissions:
statuses: write
name: Integration Tests Gate
steps:
- name: Report integration test status
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RESULT: ${{ needs.integration-tests.result }}
SHA: ${{ github.event.pull_request.head.sha }}
REPO: ${{ github.repository }}
run: |
if [ "$RESULT" = "success" ]; then
STATE="success"
else
STATE="failure"
fi
gh api "repos/$REPO/statuses/$SHA" \
-f state="$STATE" \
-f context="integration-tests" \
Copy link
Copy Markdown
Member Author

@Ephem Ephem Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These gates in the two files post to the same context, that way we can add that as a required check and it will pass if either of the jobs pass.

-f description="Integration tests: $RESULT"

pkg-pr-new:
name: Publish with pkg-pr-new
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.draft == false }}
Expand Down
Loading