Skip to content

Commit 2f45f9a

Browse files
committed
test(expo): reorient Maestro e2e to the bridge layer + add run_e2e toggle
The native UI is covered by clerk-ios/clerk-android's own tests; the Expo Maestro suite now targets only what those can't reach — the bridge/integration layer. - Cut native-UI-content flows (edit-first-name, email-verification, open-inline-profile). - Un-skip the re-auth cycle; add session/persists-across-restart (token-cache rehydration across a cold restart — a pure-Expo concern). - Un-skip theming flows and rework verification: Maestro's JS sandbox can't decode PNGs, so the in-flow runScript never worked. check-theme-color.js now counts themed-color pixels (resolution-robust) and runs as a Node post-step via verify-themes.sh, wired into the runners + a CI 'Verify theming' step. - Add run_e2e workflow input (default true) gating both jobs, so e2e can be toggled off for unit-only runs once the suite is trusted.
1 parent 1aab538 commit 2f45f9a

14 files changed

Lines changed: 233 additions & 174 deletions

.github/workflows/mobile-e2e.yml

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,11 @@ on:
3939
description: "Optional: when set together with clerk_android_ref, the workflow checks out clerk-android at that ref, bumps CLERK_*_VERSION by appending this suffix (e.g. '-expo-compat-123'), publishes to mavenLocal, and pins @clerk/expo to the resulting version. Used by clerk-android's expo-compat release gate."
4040
required: false
4141
default: ""
42+
run_e2e:
43+
description: "Run the e2e build + Maestro jobs. Set false to skip e2e entirely (the fast JS unit tests still run separately in ci.yml). Default true while we validate the suite; flip to false for unit-only once the suite is trusted."
44+
required: false
45+
type: boolean
46+
default: true
4247
workflow_call:
4348
inputs:
4449
quickstart_ref:
@@ -65,6 +70,10 @@ on:
6570
type: string
6671
required: false
6772
default: ""
73+
run_e2e:
74+
type: boolean
75+
required: false
76+
default: true
6877

6978
env:
7079
EXPO_INSTANCE_NAME: clerkstage-with-native-components
@@ -91,7 +100,7 @@ jobs:
91100
# clerk-ios's expo-compat release gate (passes clerk_ios_ref without
92101
# clerk_android_ref). Manual dispatches without ref inputs still run
93102
# both jobs.
94-
if: inputs.clerk_ios_ref == '' || inputs.clerk_android_ref != ''
103+
if: inputs.run_e2e && (inputs.clerk_ios_ref == '' || inputs.clerk_android_ref != '')
95104
runs-on: 'blacksmith-8vcpu-ubuntu-2204'
96105
timeout-minutes: 45
97106
defaults:
@@ -588,6 +597,15 @@ jobs:
588597
| while read f; do grep -qE "^[[:space:]]*-[[:space:]]*(${pattern})[[:space:]]*$" "$f" || printf '%s\n' "$f"; done
589598
| xargs -n 1 -I FLOW bash -c 'flow="$1"; for a in 1 2; do if maestro test --env CLERK_TEST_EMAIL="$CLERK_TEST_EMAIL" --env CLERK_TEST_PASSWORD="$CLERK_TEST_PASSWORD" --flatten-debug-output "$flow"; then exit 0; fi; if [ "$a" -eq 2 ]; then echo "::error::Flow $flow failed after 2 attempts"; exit 1; fi; echo "::warning::Flow $flow failed attempt $a, retrying after 10s..."; adb shell am force-stop com.clerk.clerkexpoquickstart >/dev/null 2>&1 || true; sleep 10; done' _ FLOW
590599
600+
# Theme verification is a Node post-step (Maestro can't decode PNGs in its
601+
# JS sandbox). The theming flows wrote theme-light.png / theme-dark.png to
602+
# integration/mobile; this asserts each contains a large region of the
603+
# expected themed color. Runs on success OR failure so a flaked flow
604+
# doesn't hide a theming regression; skips cleanly if no theme shots exist.
605+
- name: Verify Android theming
606+
if: success() || failure()
607+
run: bash integration/mobile/scripts/verify-themes.sh
608+
591609
- name: Upload Maestro artifacts on failure or cancel
592610
if: failure() || cancelled()
593611
uses: actions/upload-artifact@v4
@@ -616,7 +634,7 @@ jobs:
616634
# clerk-android's expo-compat release gate, which passes clerk_android_ref
617635
# (and clerk_android_snapshot_suffix) but not clerk_ios_ref. Manual
618636
# dispatches without ref inputs still run both jobs.
619-
if: inputs.clerk_android_ref == '' || inputs.clerk_ios_ref != ''
637+
if: inputs.run_e2e && (inputs.clerk_android_ref == '' || inputs.clerk_ios_ref != '')
620638
# GitHub-hosted runner. Blacksmith macOS-15 didn't pick up the job in
621639
# our trial (capacity / label availability) — revisit when Blacksmith
622640
# exposes a label we know is provisioned on this account.
@@ -1021,6 +1039,13 @@ jobs:
10211039
done
10221040
' _ FLOW
10231041
1042+
# Theme verification is a Node post-step (Maestro can't decode PNGs in its
1043+
# JS sandbox). iOS runs the light theme only (dark-mode is androidOnly);
1044+
# verify-themes.sh skips any theme shot that wasn't produced.
1045+
- name: Verify iOS theming
1046+
if: success() || failure()
1047+
run: bash integration/mobile/scripts/verify-themes.sh
1048+
10241049
- name: Upload Maestro artifacts on failure or cancel
10251050
if: failure() || cancelled()
10261051
uses: actions/upload-artifact@v4

integration/mobile/flows/cycles/sign-in-sign-out-sign-in.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@
1616
appId: com.clerk.clerkexpoquickstart
1717
tags:
1818
- regression
19-
- skip
2019
---
2120
- runFlow: ../common/open-app.yaml
2221
# First sign-in

integration/mobile/flows/profile/edit-first-name.yaml

Lines changed: 0 additions & 38 deletions
This file was deleted.

integration/mobile/flows/profile/open-inline-profile.yaml

Lines changed: 0 additions & 8 deletions
This file was deleted.
Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
# PERSISTENCE: the Expo token cache (expo-secure-store) must rehydrate the
2+
# Clerk session across a full app restart. This is a pure-Expo concern — the
3+
# native SDKs don't own JS-side session persistence — so it's exactly the
4+
# bridge/integration coverage the native UI suites (Sam/Sean) can't provide.
5+
#
6+
# Sign in, FULLY kill the app (cold start, not background), relaunch WITHOUT
7+
# clearing state, and assert we land on the signed-in home screen without
8+
# re-entering credentials. If the token cache regresses, this lands on the
9+
# signed-out AuthView instead.
10+
appId: com.clerk.clerkexpoquickstart
11+
tags:
12+
- regression
13+
- persistence
14+
---
15+
# Clean start → sign in.
16+
- runFlow: ../common/open-app.yaml
17+
- runFlow: ../common/assert-signed-out.yaml
18+
- runFlow: ../common/sign-in-email-password.yaml
19+
- runFlow: ../common/assert-signed-in.yaml
20+
# Fully terminate the app to force a cold start that must rehydrate the
21+
# session from secure-store rather than from in-memory state.
22+
- stopApp
23+
- waitForAnimationToEnd:
24+
timeout: 2000
25+
# Relaunch WITHOUT clearState — the persisted session must survive.
26+
- launchApp
27+
- waitForAnimationToEnd:
28+
timeout: 5000
29+
# Dev build: reconnect to the dev server if the launcher reappears after a
30+
# cold start (iOS: localhost, Android: 10.0.2.2 — both end in :8081).
31+
- runFlow:
32+
when:
33+
visible: "Development Build"
34+
commands:
35+
- tapOn:
36+
text: ".*:8081"
37+
- waitForAnimationToEnd:
38+
timeout: 10000
39+
- runFlow:
40+
when:
41+
visible: ".*developer menu.*"
42+
commands:
43+
- tapOn:
44+
text: "Close"
45+
optional: true
46+
- waitForAnimationToEnd:
47+
timeout: 3000
48+
# The whole point: signed in WITHOUT re-entering credentials. extendedWaitUntil
49+
# tolerates cold-start rehydration time; if the cache regressed we'd instead
50+
# see the AuthView's "Welcome! Sign in to continue".
51+
- extendedWaitUntil:
52+
visible: "Sign Out"
53+
timeout: 30000
54+
- takeScreenshot: session-persisted

integration/mobile/flows/sign-up/email-verification.yaml

Lines changed: 0 additions & 44 deletions
This file was deleted.
Lines changed: 15 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,33 +1,25 @@
1-
# REGRESSION: Native theming was not applied because Clerk.initialize() on
2-
# Android was resetting customTheme to its default (null) parameter.
1+
# REGRESSION: native theming wasn't reaching the rendered component. This has
2+
# broken two ways, both invisible to the native SDK's own tests:
3+
# 1. The Expo config plugin not embedding clerk-theme.json (the app.json
4+
# plugin entry was missing the `theme` prop) — so Info.plist / Android
5+
# assets never carried the theme.
6+
# 2. Android's Clerk.initialize() resetting customTheme to its default.
7+
# Either way the AuthView renders with default styling instead of the theme.
38
#
4-
# This flow takes a screenshot of the AuthView and uses scripts/check-theme-color.js
5-
# to assert that a sampled pixel matches the expected primary color.
9+
# This flow only CAPTURES the signed-out AuthView screenshot. The actual color
10+
# assertion runs as a Node POST-STEP in the runner (scripts/check-theme-color.js)
11+
# — Maestro's runScript sandbox is GraalJS and can't decode a PNG. The post-step
12+
# asserts the screenshot contains a large region of the themed primary (#FF4444),
13+
# which is resolution-independent (no fragile pixel coordinates).
614
appId: com.clerk.clerkexpoquickstart
7-
# TODO: the check-theme-color.js script requires pngjs (install in the
8-
# quickstart app) and the NativeComponentQuickstart currently doesn't
9-
# bundle the clerk-theme.json for iOS — so the iOS AuthView renders
10-
# with default purple not the themed red. Skip until theming is wired
11-
# through to iOS in the quickstart.
1215
tags:
1316
- regression
1417
- theming
15-
- skip
1618
---
1719
- runFlow: ../common/open-app.yaml
1820
- runFlow: ../common/assert-signed-out.yaml
1921
- waitForAnimationToEnd:
2022
timeout: 3000
21-
# Take a screenshot of the AuthView
22-
- takeScreenshot: theme-screenshot
23-
# Run the pixel-check helper. Coordinates target the primary "Continue" button.
24-
# The script exits non-zero if the sampled pixel is more than tolerance away
25-
# from the expected color.
26-
- runScript:
27-
file: ../../scripts/check-theme-color.js
28-
env:
29-
THEME_IMAGE: theme-screenshot.png
30-
THEME_X: "200"
31-
THEME_Y: "560"
32-
THEME_EXPECTED: "#FF4444"
33-
THEME_TOLERANCE: "20"
23+
# Written to integration/mobile/theme-light.png (maestro's CWD). The runner's
24+
# theme-check post-step reads this file.
25+
- takeScreenshot: theme-light
Lines changed: 10 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,13 @@
1-
# Theming: verify dark mode applies the darkColors from clerk-theme.json.
2-
# Android only: iOS hex colors are static for v1 of the theming plugin.
1+
# Theming: dark mode applies the darkColors from clerk-theme.json (primary
2+
# #FF6666). Android only — iOS theme colors are static in v1 of the plugin.
3+
#
4+
# Captures the signed-out AuthView screenshot under dark mode. The color
5+
# assertion is a Node post-step in the runner (scripts/check-theme-color.js),
6+
# which asserts a large region of the dark-mode primary (#FF6666) is present.
37
appId: com.clerk.clerkexpoquickstart
4-
# TODO: check-theme-color.js requires pngjs (install in the quickstart app).
5-
# Skip until theming tooling is wired up alongside custom-theme-applied.
68
tags:
79
- theming
810
- androidOnly
9-
- skip
1011
---
1112
# Force the device to dark mode before launching the app.
1213
- launchApp:
@@ -15,7 +16,7 @@ tags:
1516
darkMode: true
1617
- waitForAnimationToEnd:
1718
timeout: 5000
18-
# Dev build: tap the dev server URL to connect (iOS: localhost, Android: 10.0.2.2).
19+
# Dev build: tap the dev server URL to connect (Android: 10.0.2.2:8081).
1920
- runFlow:
2021
when:
2122
visible: "Development Build"
@@ -34,13 +35,6 @@ tags:
3435
- waitForAnimationToEnd:
3536
timeout: 3000
3637
- runFlow: ../common/assert-signed-out.yaml
37-
- takeScreenshot: dark-theme-screenshot
38-
# Sample a pixel from the primary button. The dark theme primary is #FF6666.
39-
- runScript:
40-
file: ../../scripts/check-theme-color.js
41-
env:
42-
THEME_IMAGE: dark-theme-screenshot.png
43-
THEME_X: "200"
44-
THEME_Y: "560"
45-
THEME_EXPECTED: "#FF6666"
46-
THEME_TOLERANCE: "20"
38+
# Written to integration/mobile/theme-dark.png (maestro's CWD). The runner's
39+
# theme-check post-step reads this file.
40+
- takeScreenshot: theme-dark
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# npm regenerates these whenever verify-themes.sh runs `npm install`. This is a
2+
# standalone helper dir (not a pnpm workspace package), so its lockfile/modules
3+
# are runtime-only and intentionally not tracked.
4+
node_modules/
5+
package-lock.json

0 commit comments

Comments
 (0)