Skip to content

Commit 78eba95

Browse files
Add E2E matrix launch smoke foundation
Assisted-By: devx/6c1e3ad5-96c8-4972-b087-da7ff7b195c3
1 parent 87bbef5 commit 78eba95

8 files changed

Lines changed: 384 additions & 11 deletions

File tree

e2e/README.md

Lines changed: 65 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,70 @@
11
# Checkout Kit End-to-End Tests
22

3-
This directory is reserved for cross-platform end-to-end tests. There is no runnable e2e suite checked in yet.
3+
This directory contains shared Maestro end-to-end test configuration for Checkout Kit sample apps.
44

5-
Planned coverage:
5+
The current runnable suite is a minimal launch smoke. It launches a sample app and waits for the shared ready marker exposed by that app.
66

7-
- Swift checkout presentation and protocol lifecycle.
8-
- Android checkout presentation and protocol lifecycle.
9-
- React Native wrapper behavior.
10-
- Web component open/close and `checkout:*` events.
7+
## Matrix
118

12-
Until this directory contains test code, use the platform test suites and sample apps described in each platform README.
9+
E2E runs are described by `config/matrix.yml`. The matrix expands applications, OS tracks, and suites into BrowserStack Maestro run rows.
10+
11+
Current applications:
12+
13+
- React Native iOS sample app
14+
- React Native Android sample app
15+
16+
Current OS tracks:
17+
18+
- `latest`
19+
20+
Current suites:
21+
22+
- `tests/shared/launch-smoke.yaml`
23+
24+
Validate the matrix:
25+
26+
```bash
27+
ruby e2e/scripts/e2e_matrix validate
28+
```
29+
30+
Expand all run rows:
31+
32+
```bash
33+
ruby e2e/scripts/e2e_matrix expand
34+
```
35+
36+
Expand a single run row by index:
37+
38+
```bash
39+
ruby e2e/scripts/e2e_matrix expand --index 0
40+
```
41+
42+
## Run locally
43+
44+
Install Maestro locally, launch the target sample app, then run the shared smoke flow with the same environment contract used by CI.
45+
46+
React Native iOS:
47+
48+
```bash
49+
E2E_APP_ID=com.shopify.checkoutkit.reactnativedemo \
50+
E2E_READY_MARKER=checkout-kit-sample-ready \
51+
maestro --platform ios test e2e/tests/shared/launch-smoke.yaml
52+
```
53+
54+
React Native Android:
55+
56+
```bash
57+
E2E_APP_ID=com.shopify.checkoutkit.reactnativedemo \
58+
E2E_READY_MARKER=checkout-kit-sample-ready \
59+
maestro --platform android test e2e/tests/shared/launch-smoke.yaml
60+
```
61+
62+
React Native E2E runs should use the released native SDK artifacts declared by the React Native sample configuration, not local in-repo native SDK overrides.
63+
64+
## Shared app contract
65+
66+
Shared flows rely on stable cross-app identifiers. The launch smoke requires each target app to expose this ready marker:
67+
68+
- `checkout-kit-sample-ready`
69+
70+
Future shared flows should add identifiers here before they are used across React Native, Swift, and Android sample apps.

e2e/config/matrix.yml

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
version: 1
2+
applications:
3+
- id: react-native-ios
4+
target: react-native
5+
platform: ios
6+
app_id: com.shopify.checkoutkit.reactnativedemo
7+
artifact_env: E2E_REACT_NATIVE_IOS_APP_PATH
8+
ready_marker: checkout-kit-sample-ready
9+
- id: react-native-android
10+
target: react-native
11+
platform: android
12+
app_id: com.shopify.checkoutkit.reactnativedemo
13+
artifact_env: E2E_REACT_NATIVE_ANDROID_APP_PATH
14+
ready_marker: checkout-kit-sample-ready
15+
os_tracks:
16+
- latest
17+
suites:
18+
- id: launch-smoke
19+
execute: tests/shared/launch-smoke.yaml

e2e/lib/e2e_matrix.rb

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# frozen_string_literal: true
2+
3+
require "yaml"
4+
5+
class E2EMatrix
6+
attr_reader :config_path
7+
8+
def self.load(config_path)
9+
new(config_path, YAML.safe_load_file(config_path))
10+
end
11+
12+
def initialize(config_path, config)
13+
@config_path = config_path
14+
@config = config || {}
15+
end
16+
17+
def expand
18+
applications.flat_map do |application|
19+
os_tracks.flat_map do |os_track|
20+
suites.map do |suite|
21+
build_run(application, os_track, suite)
22+
end
23+
end
24+
end
25+
end
26+
27+
def run_at(index)
28+
runs = expand
29+
runs.fetch(index)
30+
end
31+
32+
def validation_errors
33+
errors = []
34+
errors << "version must be 1" unless @config.fetch("version", nil) == 1
35+
validate_collection(errors, "applications", applications)
36+
validate_collection(errors, "os_tracks", os_tracks)
37+
validate_collection(errors, "suites", suites)
38+
validate_applications(errors)
39+
validate_os_tracks(errors)
40+
validate_suites(errors)
41+
errors
42+
end
43+
44+
private
45+
46+
def build_run(application, os_track, suite)
47+
platform = application.fetch("platform")
48+
os_track_id = os_track_id(os_track)
49+
suite_id = suite.fetch("id")
50+
application_id = application.fetch("id")
51+
52+
{
53+
"id" => "#{application_id}-#{os_track_id}-#{suite_id}",
54+
"application_id" => application_id,
55+
"target" => application.fetch("target"),
56+
"platform" => platform,
57+
"os_track" => os_track_id,
58+
"device_selector" => device_selector(platform, os_track),
59+
"app_id" => application.fetch("app_id"),
60+
"artifact_env" => application.fetch("artifact_env"),
61+
"execute" => suite.fetch("execute"),
62+
"ready_marker" => application.fetch("ready_marker"),
63+
"status_context" => "checkout-kit/e2e/#{application_id}/#{os_track_id}/#{suite_id}"
64+
}
65+
end
66+
67+
def device_selector(platform, os_track)
68+
return os_track.fetch("device_selector") if os_track.is_a?(Hash) && os_track.key?("device_selector")
69+
70+
"#{platform}:phone:#{os_track_id(os_track)}"
71+
end
72+
73+
def os_track_id(os_track)
74+
os_track.is_a?(Hash) ? os_track.fetch("id") : os_track
75+
end
76+
77+
def applications
78+
@config.fetch("applications", []) || []
79+
end
80+
81+
def os_tracks
82+
@config.fetch("os_tracks", []) || []
83+
end
84+
85+
def suites
86+
@config.fetch("suites", []) || []
87+
end
88+
89+
def validate_collection(errors, name, collection)
90+
errors << "#{name} must be a non-empty array" unless collection.is_a?(Array) && !collection.empty?
91+
end
92+
93+
def validate_applications(errors)
94+
return unless applications.is_a?(Array)
95+
96+
validate_unique_ids(errors, "application", applications)
97+
applications.each do |application|
98+
id = application.fetch("id", "<missing>")
99+
required_application_keys.each do |key|
100+
errors << "application #{id} missing #{key}" if application.fetch(key, "").to_s.empty?
101+
end
102+
platform = application.fetch("platform", nil)
103+
errors << "application #{id} platform must be ios or android" unless ["ios", "android"].include?(platform)
104+
end
105+
end
106+
107+
def validate_os_tracks(errors)
108+
return unless os_tracks.is_a?(Array)
109+
110+
ids = os_tracks.map { |os_track| safe_os_track_id(os_track) }
111+
errors << "os_track ids must be unique" unless ids.compact.uniq.length == ids.compact.length
112+
os_tracks.each do |os_track|
113+
id = safe_os_track_id(os_track)
114+
errors << "os_track missing id" if id.to_s.empty?
115+
end
116+
end
117+
118+
def safe_os_track_id(os_track)
119+
os_track_id(os_track)
120+
rescue KeyError
121+
nil
122+
end
123+
124+
def validate_suites(errors)
125+
return unless suites.is_a?(Array)
126+
127+
validate_unique_ids(errors, "suite", suites)
128+
suites.each do |suite|
129+
id = suite.fetch("id", "<missing>")
130+
execute = suite.fetch("execute", "")
131+
errors << "suite #{id} missing execute" if execute.empty?
132+
next if execute.empty?
133+
134+
errors << "suite #{id} execute path does not exist: #{execute}" unless File.exist?(File.join(e2e_root, execute))
135+
end
136+
end
137+
138+
def validate_unique_ids(errors, label, collection)
139+
ids = collection.map { |item| item.fetch("id", nil) }
140+
errors << "#{label} ids must be unique" unless ids.compact.uniq.length == ids.compact.length
141+
end
142+
143+
def required_application_keys
144+
["id", "target", "platform", "app_id", "artifact_env", "ready_marker"]
145+
end
146+
147+
def e2e_root
148+
File.expand_path("..", File.dirname(config_path))
149+
end
150+
end

e2e/scripts/e2e_matrix

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
#!/usr/bin/env ruby
2+
# frozen_string_literal: true
3+
4+
require "json"
5+
require "optparse"
6+
require_relative "../lib/e2e_matrix"
7+
8+
options = {
9+
config: "e2e/config/matrix.yml"
10+
}
11+
12+
parser = OptionParser.new do |opts|
13+
opts.banner = "Usage: e2e/scripts/e2e_matrix validate|expand [options]"
14+
opts.on("--config PATH") { |path| options[:config] = path }
15+
opts.on("--index INDEX", Integer) { |index| options[:index] = index }
16+
end
17+
18+
command = ARGV.shift
19+
parser.parse!(ARGV)
20+
21+
unless ["validate", "expand"].include?(command)
22+
warn parser
23+
exit 1
24+
end
25+
26+
matrix = E2EMatrix.load(options.fetch(:config))
27+
28+
case command
29+
when "validate"
30+
errors = matrix.validation_errors
31+
if errors.empty?
32+
puts "E2E matrix is valid"
33+
else
34+
warn errors.join("\n")
35+
exit 1
36+
end
37+
when "expand"
38+
errors = matrix.validation_errors
39+
unless errors.empty?
40+
warn errors.join("\n")
41+
exit 1
42+
end
43+
44+
output = if options.key?(:index)
45+
matrix.run_at(options.fetch(:index))
46+
else
47+
matrix.expand
48+
end
49+
50+
puts JSON.pretty_generate(output)
51+
end

e2e/test/e2e_matrix_test.rb

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# frozen_string_literal: true
2+
3+
require "fileutils"
4+
require "tmpdir"
5+
require "minitest/autorun"
6+
require_relative "../lib/e2e_matrix"
7+
8+
class E2EMatrixTest < Minitest::Test
9+
def test_expands_initial_matrix_rows
10+
matrix = E2EMatrix.load("e2e/config/matrix.yml")
11+
12+
runs = matrix.expand
13+
14+
assert_equal 2, runs.length
15+
assert_equal "react-native-ios-latest-launch-smoke", runs[0].fetch("id")
16+
assert_equal "react-native-android-latest-launch-smoke", runs[1].fetch("id")
17+
assert_equal ["ios", "android"], runs.map { |run| run.fetch("platform") }
18+
assert_equal ["ios:phone:latest", "android:phone:latest"], runs.map { |run| run.fetch("device_selector") }
19+
assert_equal ["tests/shared/launch-smoke.yaml"], runs.map { |run| run.fetch("execute") }.uniq
20+
end
21+
22+
def test_validates_real_matrix
23+
matrix = E2EMatrix.load("e2e/config/matrix.yml")
24+
25+
assert_empty matrix.validation_errors
26+
end
27+
28+
def test_returns_stable_indexed_run
29+
matrix = E2EMatrix.load("e2e/config/matrix.yml")
30+
31+
run = matrix.run_at(1)
32+
33+
assert_equal "react-native-android-latest-launch-smoke", run.fetch("id")
34+
assert_equal "E2E_REACT_NATIVE_ANDROID_APP_PATH", run.fetch("artifact_env")
35+
end
36+
37+
def test_reports_missing_suite_file
38+
Dir.mktmpdir do |dir|
39+
config_dir = File.join(dir, "config")
40+
FileUtils.mkdir_p(config_dir)
41+
File.write(File.join(config_dir, "matrix.yml"), <<~YAML)
42+
version: 1
43+
applications:
44+
- id: react-native-ios
45+
target: react-native
46+
platform: ios
47+
app_id: com.shopify.checkoutkit.reactnativedemo
48+
artifact_env: E2E_REACT_NATIVE_IOS_APP_PATH
49+
ready_marker: checkout-kit-sample-ready
50+
os_tracks:
51+
- latest
52+
suites:
53+
- id: missing
54+
execute: tests/shared/missing.yaml
55+
YAML
56+
57+
matrix = E2EMatrix.load(File.join(config_dir, "matrix.yml"))
58+
59+
assert_includes matrix.validation_errors, "suite missing execute path does not exist: tests/shared/missing.yaml"
60+
end
61+
end
62+
end
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
# frozen_string_literal: true
2+
3+
require "minitest/autorun"
4+
5+
class ReactNativeSampleE2ETest < Minitest::Test
6+
def test_sample_exposes_shared_launch_smoke_ready_marker
7+
app_source = File.read("platforms/react-native/sample/src/App.tsx")
8+
9+
assert_includes app_source, 'testID="checkout-kit-sample-ready"'
10+
assert_includes app_source, 'accessibilityLabel="checkout-kit-sample-ready"'
11+
end
12+
end

e2e/tests/shared/launch-smoke.yaml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
appId: ${E2E_APP_ID}
2+
---
3+
- launchApp
4+
- extendedWaitUntil:
5+
visible:
6+
id: ${E2E_READY_MARKER}
7+
timeout: 60000

0 commit comments

Comments
 (0)