GitHub Actions Workflows

This directory contains comprehensive CI/CD workflows using orchestrated testing powered by process-compose for the Devbox mobile plugins and example projects.

Orchestrated Testing

All workflows use process-compose orchestration with:

✅ Automatic status checks - Boot verification, app deployment, process health
✅ Concurrent execution - Independent tests run in parallel
✅ Configurable timeouts - No infinite hangs (BOOT_TIMEOUT, TEST_TIMEOUT)
✅ Better logging - Per-process logs automatically uploaded to artifacts
✅ Proper failure handling - Dependent tasks skip on failure
✅ Unified approach - Consistent test methodology across platforms

Workflows

1. PR Fast Checks (`pr-checks.yml`)

Trigger: Automatically runs on every PR and push to main

Purpose: Fast validation to catch issues early (~30-40 minutes)

What it tests:

Shellcheck linting + GitHub workflow validation (orchestrated)
Plugin unit tests with parallel execution (orchestrated)
Device management functionality
Cache functionality
Quick smoke tests with automatic boot verification (orchestrated)

Jobs:

lint-and-validate (ubuntu-24.04, ~10 min)
- Uses devbox run test:lint (orchestrated)
- Shellcheck on all scripts (parallel)
- GitHub workflow validation via act
android-plugin-tests (ubuntu-24.04, ~15 min)
- Uses devbox run test:android (orchestrated)
- All Android unit tests run in parallel
- Automatic test parallelization
ios-plugin-tests (macos-15, ~15 min)
- Uses devbox run test:ios (orchestrated)
- All iOS unit tests run in parallel
- Automatic test parallelization
android-quick-smoke (ubuntu-24.04 + KVM, ~30 min)
- Uses tests/e2e-android-orchestrated.sh
- Setup → Build → Boot (verified) → Deploy (verified) → Verify app running
- Automatic status checks with 3-second polling
ios-quick-smoke (macos-15, ~20 min)
- Uses tests/e2e-ios-orchestrated.sh
- Setup → Build → Boot (verified) → Deploy (verified) → Verify app running
- Automatic status checks with 3-second polling
android-example-unit-tests, ios-unit-tests, react-native-unit-tests
- Traditional unit tests (independent of orchestration)
status-check - Aggregates results from all jobs

This workflow provides fast feedback with improved reliability through automatic verification at each stage.

2. Full E2E Tests (`e2e-full.yml`)

Trigger:

Manual dispatch via GitHub Actions UI
Weekly schedule (Mondays at 00:00 UTC)

Purpose: Comprehensive end-to-end testing with orchestrated workflows across min/max platform versions (~45-60 minutes per job)

Platform Coverage:

Android

Min: API 21 (Android 5.0 Lollipop) on ubuntu-24.04
Max: API 36 (Android 15) on ubuntu-24.04
Hardware Acceleration: KVM enabled for performance
Orchestrated with automatic boot verification

iOS

Min: iOS 15.4 on macos-14 (first Apple Silicon macOS supporting iOS 15.4)
Max: iOS 26.2 on macos-15 (latest macOS version)
Orchestrated with automatic boot verification

React Native

Tests both Android and iOS builds on min/max versions
Unified job with platform matrix
Orchestrated with full status checking

Jobs:

android-e2e (ubuntu-24.04, ~45 min, matrix: min/max)
- Uses tests/e2e-android-orchestrated.sh
- Full workflow with status verification at each stage
- Configurable timeouts: BOOT_TIMEOUT=240, TEST_TIMEOUT=600
ios-e2e (macos-14/15, ~45 min, matrix: min/max)
- Uses tests/e2e-ios-orchestrated.sh
- Full workflow with status verification at each stage
- Configurable timeouts: BOOT_TIMEOUT=180, TEST_TIMEOUT=600
react-native-e2e (ubuntu/macos, ~60 min, matrix: android/ios × min/max)
- Uses tests/e2e-react-native-orchestrated.sh
- Unified job for both platforms
- Conditional setup based on platform
- Extended timeout: TEST_TIMEOUT=900
e2e-summary - Aggregates all results

Features:

✅ Parallel execution using matrix strategy
✅ Selective execution via workflow inputs
✅ Orchestrated testing with automatic status checks
✅ Process-compose logs uploaded on failure
✅ Configurable timeouts per test stage
✅ Proper failure handling with dependency chaining

Running Tests Manually

Run Full E2E Suite

Go to the Actions tab in GitHub
Select Full E2E Tests workflow
Click Run workflow
(Optional) Uncheck platforms you don't want to test
Click Run workflow

Run Specific Tests

You can selectively run tests by toggling the inputs:

run_android: Test Android native examples
run_ios: Test iOS native examples
run_react_native: Test React Native cross-platform example

Test Matrix

Android E2E

- device: min (API 21) - Android 5.0
- device: max (API 36) - Android 15

iOS E2E

- device: min (iOS 15.4)
- device: max (iOS 26.2)

React Native E2E

- platform: android, device: min (API 21)
- platform: android, device: max (API 36)
- platform: ios, device: min (iOS 15.4)
- platform: ios, device: max (iOS 26.2)

Device Configurations

Device configurations are stored in each example's devbox.d/ directory:

examples/
├── android/devbox.d/android/devices/
│   ├── min.json  (API 21)
│   └── max.json  (API 36)
├── ios/devbox.d/ios/devices/
│   ├── min.json  (iOS 15.4)
│   └── max.json  (iOS 26.2)
└── react-native/devbox.d/
    ├── android/devices/
    │   ├── min.json
    │   └── max.json
    └── ios/devices/
        ├── min.json
        └── max.json

Timeout Settings

PR Fast Checks: 15-30 minutes per job
E2E Tests: 45-60 minutes per job
Includes buffer time for:
- Emulator/simulator boot time
- Dependency installation
- Build processes
- Test execution

Concurrency

PR checks use concurrency groups to cancel outdated runs:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

This saves CI resources by cancelling superseded runs when new commits are pushed.

Failure Handling

fail-fast: false - Tests continue even if one matrix job fails
Artifacts are uploaded on failure for debugging
Summary jobs aggregate results and report overall status

Debugging Failed Tests

When a test fails:

Check the job logs in the Actions tab
Download artifacts (uploaded automatically on failure):
- Process-compose logs per process (setup, build, boot, deploy, verify)
- Android: /tmp/android-e2e-logs/, build outputs
- iOS: /tmp/ios-e2e-logs/, CoreSimulator logs
- React Native: /tmp/rn-e2e-logs/, both platforms

Examine orchestration logs:

# Download and extract artifact
tar -xzf android-smoke-logs.tar.gz

# View specific process logs
cat /tmp/android-e2e-logs/android-emulator.log  # Boot process
cat /tmp/android-e2e-logs/build-app.log         # Build output
cat /tmp/android-e2e-logs/deploy-app.log        # Deployment

Reproduce locally using orchestrated tests:

# Android E2E (with same timeouts as CI)
cd examples/android
BOOT_TIMEOUT=240 TEST_TIMEOUT=600 bash ../../tests/e2e-android-orchestrated.sh

# With interactive TUI for debugging
TEST_TUI=true BOOT_TIMEOUT=240 bash ../../tests/e2e-android-orchestrated.sh

# iOS E2E
cd examples/ios
BOOT_TIMEOUT=180 TEST_TIMEOUT=600 bash ../../tests/e2e-ios-orchestrated.sh

# Legacy approach (still works)
cd examples/android
EMU_HEADLESS=1 devbox run --pure start:emu min

Common Failure Patterns

Emulator boot timeout:

[ERROR] android-emulator failed: readiness probe timeout after 240s
[INFO] deploy-app skipped: dependency android-emulator not healthy

→ Check android-emulator.log, may need increased BOOT_TIMEOUT

Build failure:

[ERROR] build-app failed: exit code 1
[INFO] deploy-app skipped: dependency build-app not completed successfully

→ Check build-app.log for compilation errors

App deployment failure:

[ERROR] deploy-app failed: readiness probe timeout after 60s

→ Check deploy-app.log for ADB/installation errors

Cache Management

Viewing Cache Usage

Check your repository's cache usage:

Go to Settings → Actions → Caches
View cache size and hit rates
Manually delete old caches if needed

Cache Limits

Total size: 10 GB per repository (soft limit)
Retention: 7 days of inactivity
Eviction: Oldest caches deleted first when limit reached

Optimizing Cache Performance

Caches are scoped to branch by default
PR caches can restore from main branch caches
Use restore-keys for fallback cache matching

Adding New Test Cases

To add new test scenarios:

Update the test scripts in plugins/tests/
Add new jobs to pr-checks.yml for fast validation
Add new matrix entries to e2e-full.yml for platform coverage
Update this README with the new test coverage

Runner Requirements

Android Tests

OS: ubuntu-24.04 (with KVM hardware acceleration enabled)
Disk Space: ~20GB per job (emulator images, build artifacts)
Memory: 7GB+ (default for GitHub-hosted Ubuntu runners)
Cost: ~$0.008/minute (standard Linux runner pricing)

iOS Tests

Min Version (iOS 15.4): macos-14 (first Apple Silicon macOS runner supporting iOS 15.4)
Max Version (iOS 26.2): macos-15 (latest macOS supporting iOS 26.2)
Disk Space: ~30GB per job (simulator runtimes, build artifacts)
Memory: 7GB+ (default for GitHub-hosted macOS runners)
Cost: ~$0.08/minute (macOS runner pricing)

Cost Optimization

By running Android tests on Ubuntu instead of macOS:

10x cost reduction for Android tests (Linux vs macOS pricing)
KVM hardware acceleration provides similar performance to macOS
Only iOS tests require expensive macOS runners

Caching Strategy

All workflows use comprehensive caching to speed up builds and reduce CI time:

Devbox Cache

Action: jetify-com/devbox-install-action@v0.14.0
Config: enable-cache: true
Caches: Nix store, devbox packages, and shell environments

Gradle Cache (Android)

Paths: ~/.gradle/caches, ~/.gradle/wrapper
Key: Based on *.gradle* and gradle-wrapper.properties hashes
Speedup: 2-5 minutes per build

CocoaPods Cache (iOS)

Paths: ~/.cocoapods/repos, ~/Library/Caches/CocoaPods
Key: Based on Podfile.lock hash
Speedup: 1-3 minutes per build

Xcode Build Cache (iOS)

Paths: ~/Library/Developer/Xcode/DerivedData
Key: Based on Xcode project/workspace hashes
Speedup: 3-7 minutes per build

Node.js/npm Cache (React Native)

Action: actions/setup-node@v4 with built-in caching
Config: cache: 'npm'
Key: Based on package-lock.json hash
Speedup: 1-2 minutes per build

Cache Benefits

First run: Full build, caches are populated
Subsequent runs: 30-50% faster builds from cache hits
Cache invalidation: Automatic when dependency files change

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GitHub Actions Workflows

Orchestrated Testing

Workflows

1. PR Fast Checks (`pr-checks.yml`)

2. Full E2E Tests (`e2e-full.yml`)

Android

iOS

React Native

Running Tests Manually

Run Full E2E Suite

Run Specific Tests

Test Matrix

Android E2E

iOS E2E

React Native E2E

Device Configurations

Timeout Settings

Concurrency

Failure Handling

Debugging Failed Tests

Common Failure Patterns

Cache Management

Viewing Cache Usage

Cache Limits

Optimizing Cache Performance

Adding New Test Cases

Runner Requirements

Android Tests

iOS Tests

Cost Optimization

Caching Strategy

Devbox Cache

Gradle Cache (Android)

CocoaPods Cache (iOS)

Xcode Build Cache (iOS)

Node.js/npm Cache (React Native)

Cache Benefits

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

GitHub Actions Workflows

Orchestrated Testing

Workflows

1. PR Fast Checks (pr-checks.yml)

2. Full E2E Tests (e2e-full.yml)

Android

iOS

React Native

Running Tests Manually

Run Full E2E Suite

Run Specific Tests

Test Matrix

Android E2E

iOS E2E

React Native E2E

Device Configurations

Timeout Settings

Concurrency

Failure Handling

Debugging Failed Tests

Common Failure Patterns

Cache Management

Viewing Cache Usage

Cache Limits

Optimizing Cache Performance

Adding New Test Cases

Runner Requirements

Android Tests

iOS Tests

Cost Optimization

Caching Strategy

Devbox Cache

Gradle Cache (Android)

CocoaPods Cache (iOS)

Xcode Build Cache (iOS)

Node.js/npm Cache (React Native)

Cache Benefits

1. PR Fast Checks (`pr-checks.yml`)

2. Full E2E Tests (`e2e-full.yml`)