Soft-launch E2E PR checks

[kieran-osgood-shopify]

Stack Context

This is Phase 7 of the BrowserStack Maestro E2E stack for Checkout Kit, stacked on GitHub E2E reporting.

Parent issue: https://github.com/shop/issues-checkout-kit/issues/1101
Epic: https://github.com/shop/issues-checkout-kit/issues/1084

What?

Adds the soft-launch controls and runbook for PR E2E checks:

• maps Bitrise pull requests to the e2e pipeline in e2e/bitrise.yml
• defaults BrowserStack Maestro execution to E2E_STRICT=false
• keeps assertion failures reportable without blocking initial PR progress
• adds BrowserStack API retry controls for 429/5xx responses
• attempts to stop BrowserStack builds on timeout
• documents retry, timeout, local rerun, and triage behavior in e2e/RUNBOOK.md
• documents soft-launch environment controls in Bitrise setup docs

Why?

The E2E pipeline should start non-blocking while the infrastructure and suite stabilize. Developers still get commit statuses, Check Runs, and actionable failure comments, but PR progress is not blocked until the team intentionally promotes checks to required.

Validation

shadowenv exec -- ruby e2e/test/soft_launch_test.rb
shadowenv exec -- ruby e2e/test/bitrise_pipeline_test.rb
shadowenv exec -- ruby e2e/test/github_reporter_test.rb
shadowenv exec -- ruby e2e/test/e2e_matrix_test.rb
shadowenv exec -- ruby e2e/test/browserstack_device_resolver_test.rb
shadowenv exec -- ruby e2e/test/maestro_suite_package_test.rb
shadowenv exec -- ruby -c e2e/scripts/browserstack_maestro
shadowenv exec -- ruby -c e2e/scripts/report_e2e_results
shadowenv exec -- bitrise validate -c e2e/bitrise.yml
shadowenv exec -- ruby -e 'require "yaml"; YAML.safe_load_file("e2e/bitrise.yml"); puts "e2e bitrise yaml parsed"'

[kieran-osgood-shopify]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• Soft-launch E2E PR checks #364 👈 (View in Graphite)
• Report E2E results to GitHub #363
• Run BrowserStack Maestro from E2E matrix rows #362
• Build React Native iOS E2E IPA artifact #361
• Build React Native Android E2E APK artifact #360
• Add Bitrise E2E pipeline skeleton #357
• Add E2E matrix launch smoke foundation #356
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[bitrise]

Checkout Kit E2E results

Run	Status	Target	Platform	OS track	Device	Suite
react-native-ios-latest-launch-smoke	✅ passed	react-native	ios	latest	ios:phone:latest → iPhone 15-27 Beta	tests/shared/launch-smoke.yaml
react-native-android-latest-launch-smoke	❌ failed	react-native	android	latest	android:phone:latest → Google Pixel 9-17.0	tests/shared/launch-smoke.yaml

Failures

BrowserStack artifacts require BrowserStack access. Sign in to BrowserStack App Automate before opening artifact links.

react-native-android-latest-launch-smoke

• Device selector: android:phone:latest
• Resolved device: Google Pixel 9-17.0
• Suite: tests/shared/launch-smoke.yaml
• BrowserStack build: Open BrowserStack build
• Failed test: tests/shared/launch-smoke (failed)
  • Video: Open video
  • Screenshot: Open screenshot
  • Maestro commands: Open Maestro commands
  • Maestro log: Open Maestro log

[bitrise]

🤖 Bitrise Build Report > Checkout Kit E2E > e2e > #35: ✅ SUCCESS

• Duration: 12m 19s
• Workflows: 6 total / 6 executed / 0 aborted / 0 skipped / 0 failed / 6 successful