Skip to content

Commit cb24bb4

Browse files
authored
ci(playwright): add orchestrator workflow and wire up CI pipeline (#41720)
## Description Wire up the full Playwright CI pipeline with a new orchestrator workflow and updates to existing workflows: - **`playwright-e2e.yml`** (new): Top-level orchestrator that runs path-filter → client/server/rts builds → Docker image → Playwright tests. Supports `workflow_dispatch` for manual runs and `workflow_call` for PR comment triggers. Skips builds when `docker_image_name` is provided (pull existing image instead). - **`ci-test-playwright.yml`**: Refactored from standalone `workflow_dispatch` to `workflow_call` only (invoked by orchestrator). Adds TED + cloud-services containers, Depot CLI, DockerHub login, disk cleanup. Moves Node.js/Yarn/Playwright install after Appsmith is ready (parallel with container startup). - **`build-docker-image.yml`**: Adds `branch` input so the orchestrator can pass a specific branch for checkout when `pr` is 0. - **`pw-test-command.yml`**: Routes `/test-pw` PR comments through `playwright-e2e.yml` instead of directly to `ci-test-playwright.yml`, passing `pr`, `branch`, and `docker_image_name`. ## Communication Should the DevRel and Marketing teams inform users about this change? - [ ] Yes - [x] No Made with [Cursor](https://cursor.com) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added a dedicated Playwright end-to-end workflow to orchestrate conditional builds and E2E tests. * **Chores** * Reworked CI test workflow to support reusable invocation, new inputs (PR/branch/tag), and improved checkout handling. * Added/updated build orchestration and Docker image handling, container setup, caching, and test-run wiring to improve reliability. * **Chores** * Updated test-run command to invoke the new Playwright E2E workflow. <!-- end of auto-generated comment: release notes by coderabbit.ai --> <!-- This is an auto-generated comment: Cypress test results --> > [!WARNING] > Tests have not run on the HEAD 870c9b6 yet > <hr>Tue, 07 Apr 2026 15:49:52 UTC <!-- end of auto-generated comment: Cypress test results -->
1 parent cff46fa commit cb24bb4

4 files changed

Lines changed: 288 additions & 55 deletions

File tree

.github/workflows/build-docker-image.yml

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@ on:
55
workflow_dispatch:
66
workflow_call:
77
inputs:
8+
branch:
9+
description: "Branch to checkout when pr is 0 (must match client/server/rts builds)"
10+
required: false
11+
type: string
12+
default: ""
813
pr:
914
description: "This is the PR number in case the workflow is being called in a pull request"
1015
required: false
@@ -33,8 +38,14 @@ jobs:
3338

3439
# Checkout the code in the current branch in case the workflow is called because of a branch push event
3540
- name: Checkout the head commit of the branch
36-
if: inputs.pr == 0
41+
if: inputs.pr == 0 && inputs.branch == ''
42+
uses: actions/checkout@v4
43+
44+
- name: Checkout the specified branch
45+
if: inputs.pr == 0 && inputs.branch != ''
3746
uses: actions/checkout@v4
47+
with:
48+
ref: ${{ inputs.branch }}
3849

3950
- name: Download the client build artifact
4051
if: steps.run_result.outputs.run_result != 'success'

.github/workflows/ci-test-playwright.yml

Lines changed: 128 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -1,58 +1,48 @@
11
name: Appsmith Playwright E2E
22

3+
# Invoked only via playwright-e2e.yml (or another orchestrator) so Docker image + run_id align.
34
on:
4-
workflow_dispatch:
5+
workflow_call:
56
inputs:
67
project:
7-
description: "Playwright project (smoke, sanity, regression, regression-git, chromium, chromium-ee). Empty = all projects."
8+
description: "Playwright project (smoke, sanity, regression, chromium). Empty = all."
89
required: false
910
type: string
1011
default: ""
1112
feature:
12-
description: "Feature subdirectory within tier (e.g. git, datasource). Only used with tier projects."
13+
description: "Feature subdirectory (e.g. git)."
1314
required: false
1415
type: string
1516
default: ""
1617
flag_overrides:
17-
description: 'JSON feature flag overrides merged on top of real server response (e.g. {"flag":true})'
18+
description: "JSON feature flag overrides"
1819
required: false
1920
type: string
2021
default: ""
2122
docker_image_name:
22-
description: Docker image name
23+
description: "Non-empty = docker pull this image; empty = load cicontainer from same-run cache"
2324
required: false
2425
type: string
2526
default: ""
26-
workflow_call:
27-
inputs:
28-
project:
29-
description: "Playwright project (smoke, sanity, regression, regression-git, chromium, chromium-ee). Empty = all projects."
27+
pr:
28+
description: "PR number for merge ref checkout"
3029
required: false
31-
type: string
32-
default: ""
33-
feature:
34-
description: "Feature subdirectory within tier (e.g. git, datasource)"
30+
type: number
31+
default: 0
32+
branch:
33+
description: "Branch when pr is 0"
3534
required: false
3635
type: string
3736
default: ""
38-
flag_overrides:
39-
description: "JSON feature flag overrides"
40-
required: false
41-
type: string
42-
default: ""
43-
docker_image_name:
44-
description: Docker image name
37+
ted_tag:
38+
description: "test-event-driver image tag"
4539
required: false
4640
type: string
47-
default: ""
41+
default: "latest"
4842

4943
jobs:
5044
playwright-test:
5145
runs-on: warp-ubuntu-latest-x64-4x
52-
if: |
53-
github.event.pull_request.head.repo.full_name == github.repository ||
54-
github.event_name == 'push' ||
55-
github.event_name == 'workflow_dispatch'
5646
defaults:
5747
run:
5848
shell: bash
@@ -68,38 +58,39 @@ jobs:
6858
- 27017:27017
6959

7060
steps:
71-
- name: Checkout
61+
- name: Checkout PR merge
62+
if: inputs.pr != 0
7263
uses: actions/checkout@v4
7364
with:
74-
ref: ${{ inputs.pr != 0 && format('refs/pull/{0}/merge', inputs.pr) || '' }}
65+
ref: refs/pull/${{ inputs.pr }}/merge
7566

76-
- name: Set up Node.js
77-
uses: actions/setup-node@v4
67+
- name: Checkout branch
68+
if: inputs.pr == 0 && inputs.branch != ''
69+
uses: actions/checkout@v4
7870
with:
79-
node-version-file: app/client/package.json
71+
ref: ${{ inputs.branch }}
8072

81-
- name: Restore Yarn cache
82-
uses: actions/cache@v4
83-
with:
84-
path: |
85-
app/client/.yarn/cache
86-
key: v1-yarn3-${{ hashFiles('app/client/yarn.lock') }}
73+
- name: Checkout default ref
74+
if: inputs.pr == 0 && inputs.branch == ''
75+
uses: actions/checkout@v4
8776

88-
- name: Install dependencies
89-
working-directory: app/client
90-
run: yarn install --immutable
77+
- name: Clean up disk
78+
run: |
79+
sudo rm -rf /usr/share/dotnet
80+
sudo rm -rf /opt/ghc
81+
sudo rm -rf "/usr/local/share/boost"
82+
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
9183
92-
- name: Install Playwright Chromium
93-
working-directory: app/client
94-
run: npx playwright install chromium --with-deps
84+
- name: Set up Depot CLI
85+
uses: depot/setup-action@v1
9586

96-
- name: Cache Playwright browsers
97-
uses: actions/cache@v4
87+
- name: Login to DockerHub
88+
uses: docker/login-action@v3
9889
with:
99-
path: ~/.cache/ms-playwright
100-
key: playwright-browsers-${{ hashFiles('app/client/yarn.lock') }}
90+
username: ${{ secrets.DOCKER_HUB_USERNAME }}
91+
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
10192

102-
- name: Get Docker image
93+
- name: Get Docker image id
10394
run: |
10495
if [[ '${{ inputs.docker_image_name }}' != '' ]]; then
10596
echo 'docker_container_name=${{ inputs.docker_image_name }}' >> "$GITHUB_ENV"
@@ -108,29 +99,87 @@ jobs:
10899
fi
109100
110101
- name: Restore Docker image cache
102+
if: inputs.docker_image_name == ''
111103
uses: actions/cache@v4
112104
with:
113-
path: ${{ env.docker_container_name }}.tar.gz
105+
path: cicontainer.tar.gz
114106
key: docker-image-${{ github.run_id }}
115107

116108
- name: Load Docker image
117109
run: |
118110
if [[ '${{ inputs.docker_image_name }}' == '' ]]; then
119-
gunzip ${{ env.docker_container_name }}.tar.gz
120-
docker load -i ${{ env.docker_container_name }}.tar
111+
gunzip cicontainer.tar.gz
112+
docker load -i cicontainer.tar
121113
else
122114
docker pull ${{ env.docker_container_name }}
123115
fi
124116
125-
- name: Start Appsmith container
117+
- name: Create Appsmith stacks folder
118+
working-directory: "."
119+
run: mkdir -p cicontainerlocal/stacks/configuration/
120+
121+
- name: Set license for Appsmith container
122+
run: echo "APPSMITH_LICENSE_KEY=VALID-LICENSE-KEY" >> "$GITHUB_ENV"
123+
124+
- name: Run Appsmith, CS & TED docker images
125+
env:
126+
LAUNCHDARKLY_BUSINESS_FLAGS_SERVER_KEY: ${{ secrets.LAUNCHDARKLY_BUSINESS_FLAGS_SERVER_KEY }}
127+
APPSMITH_CARBON_API_KEY: ${{ secrets.CYPRESS_APPSMITH_CARBON_API_KEY }}
128+
OPENAI_API_KEY: ${{ secrets.CYPRESS_OPENAI_API_KEY }}
129+
OPENAI_ASSISTANT_ID: ${{ secrets.CYPRESS_OPENAI_ASSISTANT_ID }}
130+
working-directory: "."
126131
run: |
127-
mkdir -p cicontainerlocal/stacks/configuration/
132+
sudo /etc/init.d/ssh stop || true
133+
sudo systemctl disable --now ssh.socket || true
134+
mkdir -p ~/git-server/keys
135+
ted_tag="${{ inputs.ted_tag }}"
136+
docker run --name test-event-driver -d -p 22:22 -p 5001:5001 -p 3306:3306 \
137+
-p 5433:5432 -p 28017:27017 -p 25:25 -p 4200:4200 -p 5000:5000 -p 3001:3000 -p 6001:6001 -p 8001:8000 --privileged --pid=host --ipc=host --volume /:/host -v ~/git-server/keys:/git-server/keys \
138+
"appsmith/test-event-driver:${ted_tag:-latest}"
139+
docker run --name cloud-services -d -p 8000:80 -p 8090:8090 \
140+
--privileged --pid=host --ipc=host --add-host=host.docker.internal:host-gateway\
141+
-e KEYGEN_API_URL=http://host.docker.internal:5001 \
142+
-e KEYGEN_BYPASS_SIGNATURE_VERIFICATION=true \
143+
-e CREATE_DUMMY_LICENSES=true \
144+
-e APPSMITH_CLOUD_SERVICES_MONGODB_URI=mongodb://host.docker.internal:27017 \
145+
-e APPSMITH_CLOUD_SERVICES_MONGODB_DATABASE=cs \
146+
-e APPSMITH_CLOUD_SERVICES_MONGODB_AUTH_DATABASE=admin \
147+
-e APPSMITH_REDIS_URL=redis://host.docker.internal:6379/ \
148+
-e APPSMITH_APPS_API_KEY=dummy-api-key \
149+
-e APPSMITH_REMOTE_API_KEY=dummy-api-key \
150+
-e APPSMITH_GITHUB_API_KEY=dummy-appsmith-gh-api-key \
151+
-e APPSMITH_JWT_SECRET=appsmith-cloud-services-jwt-secret-dummy-key \
152+
-e APPSMITH_ENCRYPTION_SALT=encryption-salt \
153+
-e APPSMITH_ENCRYPTION_PASSWORD=encryption-password \
154+
-e APPSMITH_CUSTOMER_PORTAL_URL=https://dev.appsmith.com \
155+
-e APPSMITH_CLOUD_SERVICES_BASE_URL=https://cs-dev.appsmith.com \
156+
-e AUTH0_ISSUER_URL=https://login.release-customer.appsmith.com/ \
157+
-e AUTH0_CLIENT_ID=dummy-client-id \
158+
-e AUTH0_CLIENT_SECRET=dummy-secret-id \
159+
-e AUTH0_AUDIENCE_URL=https://login.local-customer.appsmith.com/ \
160+
-e CLOUDSERVICES_URL=cs-dev.appsmith.com \
161+
-e CUSTOMER_URL=dev.appsmith.com \
162+
-e ENTERPRISE_USER_NAME=ent-user@appsmith.com \
163+
-e ENTERPRISE_USER_PASSWORD=ent_user_password \
164+
-e ENTERPRISE_ADMIN_NAME=ent-admin@appsmith.com \
165+
-e ENTERPRISE_ADMIN_PASSWORD=ent_admin_password \
166+
-e LAUNCHDARKLY_BUSINESS_FLAGS_SERVER_KEY=$LAUNCHDARKLY_BUSINESS_FLAGS_SERVER_KEY \
167+
appsmith/cloud-services:release
128168
cd cicontainerlocal
129169
docker run -d --name appsmith -p 80:80 \
130170
-v "$PWD/stacks:/appsmith-stacks" \
171+
-e APPSMITH_LICENSE_KEY="$APPSMITH_LICENSE_KEY" \
131172
-e APPSMITH_DISABLE_TELEMETRY=true \
173+
-e APPSMITH_INTERCOM_APP_ID=DUMMY_VALUE \
132174
-e APPSMITH_CLOUD_SERVICES_BASE_URL=http://host.docker.internal:5001 \
133-
--add-host=host.docker.internal:host-gateway \
175+
-e APPSMITH_CLOUD_SERVICES_SIGNATURE_BASE_URL=http://host.docker.internal:8090 \
176+
-e APPSMITH_RATE_LIMIT=1000 \
177+
-e APPSMITH_CARBON_API_BASE_PATH=https://carbon.appsmith.com \
178+
-e APPSMITH_CARBON_API_KEY="$APPSMITH_CARBON_API_KEY" \
179+
-e APPSMITH_AI_SERVER_MANAGED_HOSTING=true \
180+
-e OPENAI_ASSISTANT_ID="$OPENAI_ASSISTANT_ID" \
181+
-e OPENAI_API_KEY="$OPENAI_API_KEY" \
182+
--add-host=host.docker.internal:host-gateway --add-host=api.segment.io:host-gateway --add-host=t.appsmith.com:host-gateway \
134183
${{ env.docker_container_name }}
135184
136185
- name: Wait for Appsmith to be ready
@@ -145,6 +194,32 @@ jobs:
145194
sleep 10
146195
done
147196
197+
- name: Set up Node.js
198+
uses: actions/setup-node@v4
199+
with:
200+
node-version-file: app/client/package.json
201+
202+
- name: Restore Yarn cache
203+
uses: actions/cache@v4
204+
with:
205+
path: |
206+
app/client/.yarn/cache
207+
key: v1-yarn3-${{ hashFiles('app/client/yarn.lock') }}
208+
209+
- name: Install dependencies
210+
working-directory: app/client
211+
run: yarn install --immutable
212+
213+
- name: Install Playwright Chromium
214+
working-directory: app/client
215+
run: npx playwright install chromium --with-deps
216+
217+
- name: Cache Playwright browsers
218+
uses: actions/cache@v4
219+
with:
220+
path: ~/.cache/ms-playwright
221+
key: playwright-browsers-${{ hashFiles('app/client/yarn.lock') }}
222+
148223
- name: Run Playwright tests (attempt 1)
149224
id: attempt1
150225
continue-on-error: true

0 commit comments

Comments
 (0)