CI/CD Documentation

This document provides comprehensive information about the CI/CD workflows for the devbox mobile plugins repository.

Overview

The repository uses GitHub Actions with three main workflows to ensure code quality and platform compatibility. The CI/CD strategy balances speed and thoroughness by running fast checks on every PR and comprehensive tests on-demand or on a schedule.

Workflow Types

PR Fast Checks - Quick validation for every PR
Full E2E Tests - Comprehensive platform testing

Design Philosophy

The CI/CD system prioritizes:

Fast feedback - PRs get fast feedback
Platform coverage - Tests minimum and maximum supported platform versions
Cost efficiency - Android tests run on Linux, iOS only on macOS
Parallelization - Matrix strategy runs tests concurrently
Reliability - Caching and proper timeouts prevent flaky tests

PR Checks Workflow (`pr-checks.yml`)

Triggers

Runs automatically on:

Pull requests targeting main branch
Direct pushes to main branch

Concurrency Control

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

Automatically cancels outdated workflow runs when new commits are pushed to the same PR or branch, saving CI resources.

Jobs

1. Fast Tests (ubuntu-24.04)

Runs linting, unit tests, and integration tests without device emulation.

devbox run test:fast

What it includes:

Shellcheck linting on all bash scripts
GitHub workflow validation via act --list
Android plugin unit tests (parallel execution via process-compose)
iOS plugin unit tests (parallel execution via process-compose)
Device management tests
Cache functionality tests

Environment: ubuntu-24.04 (Linux)

Artifacts uploaded:

fast-test-reports - Test results and logs from test-results/ and reports/

2. Android E2E (ubuntu-24.04 with KVM, matrix: min/max)

End-to-end tests with real Android emulators.

Matrix strategy:

matrix:
  device: [min, max]

min - API 21 (Android 5.0 Lollipop)
max - API 36 (Android 15)

Prerequisites:

KVM hardware acceleration enabled
Gradle build cache configured

Environment variables:

EMU_HEADLESS: 1          # Run emulator without GUI
BOOT_TIMEOUT: 180        # 3 minutes for emulator boot
TEST_TIMEOUT: 300        # 5 minutes for test execution
ANDROID_DEFAULT_DEVICE: ${{ matrix.device }}
TEST_TUI: false          # Disable interactive terminal UI

Workflow:

Enable KVM for hardware acceleration
Setup Gradle cache for faster builds
Install Devbox with package caching
Run E2E test: devbox run --pure test:e2e
Upload artifacts on success or failure

Artifacts uploaded:

android-{min|max}-reports - Test reports from reports/ and APK outputs

3. iOS E2E (macos-14/15, matrix: min/max)

End-to-end tests with real iOS simulators.

Matrix strategy:

matrix:
  include:
    - device: min
      os: macos-14
    - device: max
      os: macos-15

min - iOS 15.4 on macos-14 (first Apple Silicon macOS supporting iOS 15.4)
max - iOS 26.2 on macos-15 (latest macOS version)

Prerequisites:

CocoaPods cache configured
Xcode build cache configured

Environment variables:

SIM_HEADLESS: 1          # Run simulator without GUI
BOOT_TIMEOUT: 120        # 2 minutes for simulator boot
TEST_TIMEOUT: 300        # 5 minutes for test execution
IOS_DEFAULT_DEVICE: ${{ matrix.device }}
TEST_TUI: false          # Disable interactive terminal UI

Workflow:

Setup CocoaPods and Xcode build caches
Install Devbox with package caching
Run E2E test: devbox run --pure test:e2e
Upload artifacts on success or failure

Artifacts uploaded:

ios-{min|max}-reports - Test reports and CoreSimulator logs

4. React Native E2E (ubuntu/macos, matrix: android/ios × min/max + web)

Cross-platform React Native tests covering Android, iOS, and web.

Matrix strategy:

matrix:
  include:
    # Android tests
    - platform: android, device: min, os: ubuntu-24.04
    - platform: android, device: max, os: ubuntu-24.04
    # iOS tests
    - platform: ios, device: min, os: macos-14
    - platform: ios, device: max, os: macos-15
    # Web test
    - platform: web, device: none, os: ubuntu-24.04

Prerequisites (platform-specific):

Node.js 20 with npm cache
Android: KVM enabled, Gradle cache
iOS: CocoaPods cache, Xcode build cache

Environment variables:

EMU_HEADLESS: ${{ matrix.platform == 'android' && '1' || '0' }}
SIM_HEADLESS: ${{ matrix.platform == 'ios' && '1' || '0' }}
BOOT_TIMEOUT: 240        # 4 minutes for device boot
TEST_TIMEOUT: 600        # 10 minutes for test execution
ANDROID_DEFAULT_DEVICE: ${{ matrix.device }}
IOS_DEFAULT_DEVICE: ${{ matrix.device }}
TEST_TUI: false

Workflow:

Setup Node.js with npm cache
Enable KVM (Android only)
Setup platform-specific caches (Gradle/CocoaPods/Xcode)
Install Devbox with package caching
Run platform-specific test:
- Android: devbox run --pure -e IOS_SKIP_SETUP=1 -e EMU_HEADLESS=1 test:e2e:android
- iOS: devbox run --pure -e ANDROID_SKIP_SETUP=1 -e SIM_HEADLESS=1 test:e2e:ios
- Web: devbox run --pure -e ANDROID_SKIP_SETUP=1 -e IOS_SKIP_SETUP=1 test:e2e:web
Upload artifacts on success or failure

Artifacts uploaded:

react-native-{platform}-{device}-reports - Test reports, build outputs, and logs

5. Status Check (ubuntu-latest)

Aggregates results from all jobs and provides summary status.

Dependencies:

needs: [fast-tests, android-e2e, ios-e2e, react-native-e2e]
if: always()

Outputs:

📊 PR Check Results:
  Fast Tests: success
  Android E2E: success
  iOS E2E: success
  React Native E2E: success

Fails if any dependent job fails, preventing merge of broken code.

Full E2E Workflow (`e2e-full.yml`)

Triggers

Runs on:

Manual dispatch via GitHub Actions UI
Weekly schedule - Mondays at 00:00 UTC

on:
  workflow_dispatch:
    inputs:
      run_android: true
      run_ios: true
      run_react_native: true
  schedule:
    - cron: '0 0 * * 1'

Platform Selection

When triggering manually, you can selectively run tests by toggling inputs:

run_android - Test Android examples (default: true)
run_ios - Test iOS examples (default: true)
run_react_native - Test React Native example (default: true)

Job Configurations

Android E2E (ubuntu-24.04, matrix: min/max)

Similar to PR checks but with extended timeouts for more thorough testing.

Key differences from PR checks:

BOOT_TIMEOUT: 240 (4 minutes vs 3 minutes)
TEST_TIMEOUT: 600 (10 minutes vs 5 minutes)
timeout-minutes: 45 (job timeout vs 30 minutes)

iOS E2E (macos-14/15, matrix: min/max)

Similar to PR checks but with extended timeouts.

Key differences from PR checks:

BOOT_TIMEOUT: 180 (3 minutes vs 2 minutes)
TEST_TIMEOUT: 600 (10 minutes vs 5 minutes)
timeout-minutes: 45 (job timeout vs 25 minutes)

React Native E2E (ubuntu/macos, matrix: android/ios × min/max + web)

Similar to PR checks but with extended timeouts.

Key differences from PR checks:

BOOT_TIMEOUT: 300 (5 minutes vs 4 minutes)
TEST_TIMEOUT: 900 (15 minutes vs 10 minutes)
timeout-minutes: 60 (job timeout vs 45 minutes)

Platform Coverage

Android:

API 21 (Android 5.0 Lollipop) - Minimum supported version
API 36 (Android 15) - Latest stable version

iOS:

iOS 15.4 (on macos-14) - Minimum supported version
iOS 26.2 (on macos-15) - Latest stable version

React Native:

All Android and iOS versions above
Web build (no device needed)

Optimization Strategies

Device Filtering

Both workflows test only min and max devices to minimize CI time while maintaining platform coverage.

Configuration in device lock files:

# examples/android/devbox.d/android/devices/devices.lock
min:abc123...
max:def456...

Lock files are committed to the repository and limit which SDK versions are evaluated by Nix.

Platform-Specific Skipping

Android and iOS initialization scripts check for platform-specific skip flags:

# Skip expensive operations in CI
if [ "${ANDROID_SKIP_SETUP:-0}" = "1" ]; then
  # Skip SDK component downloads
fi

if [ "${IOS_SKIP_SETUP:-0}" = "1" ]; then
  # Skip Xcode setup
fi

These flags are not currently used in workflows but are available for optimization.

Caching Strategy

All workflows use comprehensive caching to speed up builds:

Devbox Cache

Action: jetify-com/devbox-install-action@v0.14.0
Config: enable-cache: true
Caches: Nix store, devbox packages, shell environments

Gradle Cache (Android)

uses: actions/cache@v4
with:
  path: |
    ~/.gradle/caches
    ~/.gradle/wrapper
  key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*', '**/gradle-wrapper.properties') }}
  restore-keys: ${{ runner.os }}-gradle-

CocoaPods Cache (iOS)

uses: actions/cache@v4
with:
  path: |
    ~/.cocoapods/repos
    ~/Library/Caches/CocoaPods
  key: ${{ runner.os }}-pods-${{ hashFiles('**/Podfile.lock') }}
  restore-keys: ${{ runner.os }}-pods-

Xcode Build Cache (iOS)

uses: actions/cache@v4
with:
  path: ~/Library/Developer/Xcode/DerivedData
  key: ${{ runner.os }}-xcode-${{ hashFiles('**/*.xcodeproj/**', '**/*.xcworkspace/**') }}
  restore-keys: ${{ runner.os }}-xcode-

Node.js/npm Cache (React Native)

uses: actions/setup-node@v4
with:
  node-version: '20'
  cache: 'npm'
  cache-dependency-path: examples/react-native/package-lock.json

Matrix Parallelization

Both workflows use matrix strategies to run tests in parallel:

strategy:
  fail-fast: false
  matrix:
    device: [min, max]

Benefits:

Tests complete in time of slowest job, not sum of all jobs
fail-fast: false allows all tests to complete even if one fails
Clear visibility into which specific configurations fail

Lock Files for SDK Optimization

Device lock files contain checksums of device definitions:

min:1a2b3c4d5e6f7g8h9i0j
max:9i8h7g6f5e4d3c2b1a0j

Benefits:

Nix only evaluates SDK versions for devices in the lock file
Skips evaluation of unused intermediate API levels
Significantly reduces Nix flake evaluation time

Maintenance:

# Regenerate after adding/removing devices
devbox run --pure android.sh devices eval
devbox run --pure ios.sh devices eval

Running CI Locally

Using act

The act tool runs GitHub Actions workflows locally using Docker.

Installation:

# Add to your devbox environment
devbox add act

# Or install globally
brew install act  # macOS

Prerequisites:

Docker installed and running
Sufficient disk space (~20GB for Android, ~30GB for iOS)

Running Specific Jobs

# Fast tests (no emulation)
act -j fast-tests

# Android E2E tests
act -j android-e2e

# iOS E2E tests (requires macOS)
act -j ios-e2e

# React Native E2E tests
act -j react-native-e2e

# Status check
act -j status-check

Running Entire Workflows

# PR checks workflow
act -W .github/workflows/pr-checks.yml

# Full E2E workflow
act -W .github/workflows/e2e-full.yml

Testing Workflow Changes

# Validate workflow syntax without running
act --list

# List all jobs in workflow
act -W .github/workflows/pr-checks.yml --list

# Run with specific event trigger
act pull_request -W .github/workflows/pr-checks.yml

# Run with specific matrix values
act -j android-e2e -e matrix='{"device":"min"}'

Limitations

iOS tests cannot run locally:

macOS simulators require macOS host (not Linux)
act runs on Linux Docker containers
Use GitHub Actions runners for iOS testing

Secrets and environment variables:

# Pass secrets via file
act -s GITHUB_TOKEN=your_token

# Or via environment file
echo "GITHUB_TOKEN=your_token" > .secrets
act --secret-file .secrets

Resource requirements:

Android tests need KVM support (Linux only)
Each test may require 10-20GB disk space
Ensure Docker has sufficient memory (8GB+)

Debugging CI Failures

Accessing Workflow Logs

Go to Actions tab in GitHub repository
Select the failed workflow run
Click on the failed job name
Expand the failed step to view logs

Downloading Artifacts

All workflows upload artifacts on failure for debugging.

To download:

Navigate to the failed workflow run
Scroll to Artifacts section at the bottom
Click artifact name to download (e.g., android-min-reports)

Artifact contents:

test-results/ - Process-compose logs, per-process outputs
reports/ - Application logs, test reports
app/build/outputs/ (Android) - APK files, build logs
ios/build/ (iOS) - App bundles, build logs
~/Library/Logs/CoreSimulator/ (iOS) - Simulator logs

Common Failure Patterns

Emulator Boot Timeout

Symptom:

ERROR: Emulator failed to boot within 180 seconds

Possible causes:

KVM not enabled (Android on Linux)
Insufficient system resources
Emulator image download failure
API level incompatibility

Debug steps:

Check android-emulator.log in artifacts
Verify KVM is enabled: ls -l /dev/kvm
Check system resources: free -h, df -h
Try manually: EMU_HEADLESS=1 devbox run start:emu min

Fix:

Increase BOOT_TIMEOUT in workflow
Verify device definition in devices/min.json
Check Nix flake SDK composition

Build Failure

Symptom:

ERROR: Build failed with exit code 1

Possible causes:

Missing dependencies
Gradle/Xcode version incompatibility
Source code compilation errors
Cache corruption

Debug steps:

Check build-app.log in artifacts
Look for compilation errors in logs
Check dependency resolution issues
Try locally: devbox run build in the appropriate example project

Fix:

Update dependencies in build.gradle or Podfile
Clear cache: delete workflow caches in Settings → Actions → Caches
Fix source code errors
Update build tools version

App Deployment Failure

Symptom:

ERROR: Failed to install app on device

Possible causes:

ADB connection issues (Android)
Simulator not ready (iOS)
APK/app bundle not found
Incompatible target device

Debug steps:

Check deploy-app.log in artifacts
Verify emulator is running: check android-emulator.log
Check APK/app path: verify ANDROID_APP_APK or IOS_APP_ARTIFACT
Try manually: adb install app.apk or xcrun simctl install booted app.app

Fix:

Increase BOOT_TIMEOUT to ensure device is ready
Verify app artifact path in configuration
Check device compatibility with app requirements

Test Execution Timeout

Symptom:

ERROR: Test execution exceeded 300 seconds

Possible causes:

Test suite is slow
App is unresponsive
Device performance issues
Infinite loop in test code

Debug steps:

Check test logs in artifacts
Identify which test is hanging
Try locally with debug logging: DEBUG=1 devbox run test:e2e
Profile test execution time

Fix:

Increase TEST_TIMEOUT in workflow
Optimize slow tests
Fix hanging test code
Add timeout assertions in tests

Reproducing Locally

Android Issues

cd examples/android

# Reproduce exact CI environment
EMU_HEADLESS=1 \
BOOT_TIMEOUT=180 \
TEST_TIMEOUT=300 \
ANDROID_DEFAULT_DEVICE=min \
TEST_TUI=false \
devbox run --pure test:e2e

# Or with interactive UI for debugging
BOOT_TIMEOUT=180 \
TEST_TIMEOUT=300 \
ANDROID_DEFAULT_DEVICE=min \
devbox run test:e2e

iOS Issues

cd examples/ios

# Reproduce exact CI environment
SIM_HEADLESS=1 \
BOOT_TIMEOUT=120 \
TEST_TIMEOUT=300 \
IOS_DEFAULT_DEVICE=min \
TEST_TUI=false \
devbox run --pure test:e2e

# Or with interactive UI for debugging
BOOT_TIMEOUT=120 \
TEST_TIMEOUT=300 \
IOS_DEFAULT_DEVICE=min \
devbox run test:e2e

React Native Issues

cd examples/react-native

# Android
devbox run --pure -e IOS_SKIP_SETUP=1 -e EMU_HEADLESS=1 test:e2e:android

# iOS
devbox run --pure -e ANDROID_SKIP_SETUP=1 -e SIM_HEADLESS=1 test:e2e:ios

# Web
devbox run --pure -e ANDROID_SKIP_SETUP=1 -e IOS_SKIP_SETUP=1 test:e2e:web

Examining Logs

Process-compose logs (when using orchestrated tests):

# Download artifact and extract
tar -xzf android-min-reports.tar.gz

# View specific process logs
cat reports/logs/android-emulator.log  # Device boot
cat reports/logs/build-app.log         # Build output
cat reports/logs/deploy-app.log        # Deployment
cat reports/logs/verify-app.log        # App verification

Traditional logs:

# Android emulator logs
cat reports/logs/emulator.log

# Android build logs
cat examples/android/app/build/outputs/logs/build.log

# iOS simulator logs
cat ~/Library/Logs/CoreSimulator/*/system.log

# iOS build logs
cat reports/logs/xcodebuild.log

Adding New Workflows

Workflow File Structure

GitHub Actions workflows are YAML files in .github/workflows/:

name: Workflow Name

on:
  # Triggers
  push:
    branches: [main]
  pull_request:
    branches: [main]

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  job-name:
    name: Job Display Name
    runs-on: ubuntu-24.04
    timeout-minutes: 30
    steps:
      - uses: actions/checkout@v4

      - name: Install Devbox
        uses: jetify-com/devbox-install-action@v0.14.0
        with:
          enable-cache: true

      - name: Run tests
        run: devbox run test

      - name: Upload artifacts
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: reports/

Job Dependencies

Use needs to create job dependencies:

jobs:
  build:
    runs-on: ubuntu-24.04
    steps:
      - run: echo "Building..."

  test:
    needs: build  # Waits for build to succeed
    runs-on: ubuntu-24.04
    steps:
      - run: echo "Testing..."

  deploy:
    needs: [build, test]  # Waits for both
    if: always()          # Runs even if test fails
    runs-on: ubuntu-24.04
    steps:
      - run: echo "Deploying..."

Matrix Strategy

Use matrices to run tests in parallel across configurations:

jobs:
  test:
    strategy:
      fail-fast: false
      matrix:
        platform: [android, ios]
        device: [min, max]
        include:
          - platform: android
            os: ubuntu-24.04
          - platform: ios
            os: macos-15
    runs-on: ${{ matrix.os }}
    steps:
      - run: echo "Testing ${{ matrix.platform }} on ${{ matrix.device }}"

Secrets Management

Store sensitive values as repository secrets:

Adding secrets:

Go to Settings → Secrets and variables → Actions
Click New repository secret
Add name and value

Using secrets in workflows:

steps:
  - name: Deploy
    env:
      NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
      API_KEY: ${{ secrets.API_KEY }}
    run: npm publish

Protected environments:

jobs:
  publish:
    environment:
      name: production
    steps:
      - run: npm publish

Best Practices

Use concurrency control:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

Set reasonable timeouts:

jobs:
  test:
    timeout-minutes: 30  # Prevent infinite hangs

Upload artifacts on failure:

- name: Upload logs
  if: always()  # Run even if previous steps failed
  uses: actions/upload-artifact@v4
  with:
    name: logs
    path: reports/

Use caching:

- name: Cache dependencies
  uses: actions/cache@v4
  with:
    path: ~/.gradle/caches
    key: ${{ runner.os }}-gradle-${{ hashFiles('**/*.gradle*') }}

Validate workflows locally:

# Check syntax
act --list

# Run specific job
act -j test

Follow naming conventions:

Workflow files: {purpose}.yml (e.g., pr-checks.yml)
Job names: {action}-{target} (e.g., test-android)
Step names: Descriptive actions (e.g., "Install dependencies")

Document in README: Update .github/workflows/README.md when adding new workflows with:

Purpose and triggers
Job descriptions
Timing expectations
Environment requirements

Cost Considerations

Runner Pricing (GitHub-hosted)

Linux (ubuntu-latest): $0.008/minute
macOS (macos-latest): $0.08/minute
Windows (windows-latest): $0.016/minute

Current Workflow Costs (estimated per run)

PR Checks:

Fast Tests (Linux): low cost
Android E2E (Linux, 2 matrix jobs): low cost
iOS E2E (macOS, 2 matrix jobs): higher cost due to macOS pricing
React Native E2E (Linux+macOS, 5 matrix jobs): highest cost component

Full E2E:

Similar to PR checks but with longer timeouts

Optimization Impact

Running Android on Linux vs macOS:

Linux: $0.008/minute
macOS: $0.08/minute
Savings: 10x cost reduction for Android tests

Using matrix parallelization:

Without: Jobs run sequentially, total wall time is sum of all jobs
With: Jobs run concurrently, total wall time is the slowest single job
Savings: Significant wall-time reduction, same total compute cost

Using caching:

First run: Full build with no cached dependencies
Cached run: Partial build reusing cached dependencies
Savings: Meaningful reduction in build time per run

Free Tier Limits

GitHub provides free minutes for public repositories:

Public repos: Unlimited free minutes
Private repos: 2,000 free minutes/month (then $0.008/minute for Linux)

FilesExpand file tree

CI-CD.md

Latest commit

History

CI-CD.md

File metadata and controls

CI/CD Documentation

Overview

Workflow Types

Design Philosophy

PR Checks Workflow (pr-checks.yml)

Triggers

Concurrency Control

Jobs

1. Fast Tests (ubuntu-24.04)

2. Android E2E (ubuntu-24.04 with KVM, matrix: min/max)

3. iOS E2E (macos-14/15, matrix: min/max)

4. React Native E2E (ubuntu/macos, matrix: android/ios × min/max + web)

5. Status Check (ubuntu-latest)

Full E2E Workflow (e2e-full.yml)

Triggers

Platform Selection

Job Configurations

Android E2E (ubuntu-24.04, matrix: min/max)

iOS E2E (macos-14/15, matrix: min/max)

React Native E2E (ubuntu/macos, matrix: android/ios × min/max + web)

Platform Coverage

Optimization Strategies

Device Filtering

Platform-Specific Skipping

Caching Strategy

Devbox Cache

Gradle Cache (Android)

CocoaPods Cache (iOS)

Xcode Build Cache (iOS)

Node.js/npm Cache (React Native)

Matrix Parallelization

Lock Files for SDK Optimization

Running CI Locally

Using act

Running Specific Jobs

Running Entire Workflows

Testing Workflow Changes

Limitations

Debugging CI Failures

Accessing Workflow Logs

Downloading Artifacts

Common Failure Patterns

Emulator Boot Timeout

Build Failure

App Deployment Failure

Test Execution Timeout

Reproducing Locally

Android Issues

iOS Issues

React Native Issues

Examining Logs

Adding New Workflows

Workflow File Structure

Job Dependencies

Matrix Strategy

Secrets Management

Best Practices

Cost Considerations

Runner Pricing (GitHub-hosted)

Current Workflow Costs (estimated per run)

Optimization Impact

Free Tier Limits

References

PR Checks Workflow (`pr-checks.yml`)

Full E2E Workflow (`e2e-full.yml`)