Skip to content

Commit 81c22d5

Browse files
msluszniakmsluszniak
authored
ci: per-app build matrix with precise path triggers (#1112)
## Description Replaces the LLM-only build workflows with one matrix-based `build-apps.yml`, gated by `dorny/paths-filter` so each app/platform cell runs only when its code or shared infrastructure changes. Three parallel matrices per push: - a Metro JS bundle (production mode, catches missing imports and transform errors before any native build), - Android native build, - iOS native build. Cells skip in ~30s via content-hash cache markers when their relevant files haven't changed since the last passing run. A lint-time drift check fails PRs that introduce source files not covered by any per-app filter. Adds a Jest smoke test for bare-rn. Skips all build workflows on draft PRs; fires on `ready_for_review`. ### Introduces a breaking change? - [ ] Yes - [x] No ### Type of change - [ ] Bug fix (change which fixes an issue) - [ ] New feature (change which adds functionality) - [ ] Documentation update (improves or adds clarity to existing documentation) - [x] Other (chores, tests, code style improvements etc.) ### Tested on Not applicable ### Testing instructions 1. Open as draft — confirm no build workflows run. 2. Mark ready for review — `Example apps build check` runs; the matrix contains only the apps whose paths actually changed. 3. Touch only `apps/computer-vision/**` — only `computer-vision` cells run (Android + iOS + bundle). 4. Touch only `packages/react-native-executorch/android/**` — only Android native cells run; iOS and bundle skip. 5. Push a no-op touching only the workflow file — every cell hits its cache marker and finishes in ~30s. 6. Add a broken import to one app's `App.tsx` — the bundle cell for that app fails fast before any native build finishes. 7. Add a tracked file under `packages/react-native-executorch/` not matched by any filter — `Check CI filter coverage` in the `lint` job fails and lists the uncovered path. ### Related issues Closes #963. ### Checklist - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have updated the documentation accordingly - [x] My changes generate no new warnings ### Additional notes This PR also fixes notorious iOS CI fails but getting available simulator, not a specific one.
1 parent 73cdedf commit 81c22d5

17 files changed

Lines changed: 943 additions & 146 deletions

‎.github/actions/build-android-app/action.yml‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
name: Build Android app
2+
description: Build an Android demo app from this monorepo (with optional Expo prebuild)
3+
4+
inputs:
5+
app-path:
6+
description: Path to the app workspace, relative to the repo root (e.g. apps/llm)
7+
required: true
8+
expo-prebuild:
9+
description: Whether to run `expo prebuild --platform android` before the Gradle build
10+
required: false
11+
default: "true"
12+
filter-name:
13+
description: dorny/paths-filter filter name for this app+platform (e.g. llm-android). Used as the content-hash cache key so a previously-passing build can be skipped if nothing relevant has changed.
14+
required: true
15+
16+
runs:
17+
using: composite
18+
steps:
19+
- name: Setup Node.js
20+
uses: actions/setup-node@v6
21+
with:
22+
node-version-file: .nvmrc
23+
cache: "yarn"
24+
25+
- name: Install root dependencies
26+
shell: bash
27+
run: yarn install --immutable
28+
29+
- name: Compute build hash
30+
id: hash
31+
shell: bash
32+
run: |
33+
h=$(npx ts-node scripts/compute-app-hash.ts "${{ inputs.filter-name }}")
34+
echo "key=build-${{ inputs.filter-name }}-$h" >> "$GITHUB_OUTPUT"
35+
36+
- name: Lookup pass marker
37+
id: cache
38+
uses: actions/cache/restore@v4
39+
with:
40+
path: ${{ runner.temp }}/ci-marker
41+
key: ${{ steps.hash.outputs.key }}
42+
lookup-only: true
43+
44+
- name: Skip notice
45+
if: steps.cache.outputs.cache-hit == 'true'
46+
shell: bash
47+
run: echo "Skipping build — ${{ inputs.filter-name }} already passed at this content hash."
48+
49+
# ubuntu-latest ships with ~14 GB free, but a full Android build for any of
50+
# the demo apps (Gradle cache + ExecuTorch native build + node_modules +
51+
# third-party libs) regularly exceeds that and fails with "no space left on
52+
# device". Reclaim ~10 GB by removing toolchains we don't use here.
53+
- name: Free disk space
54+
if: steps.cache.outputs.cache-hit != 'true'
55+
shell: bash
56+
run: |
57+
sudo rm -rf /usr/share/dotnet
58+
sudo rm -rf /opt/ghc
59+
sudo rm -rf /opt/hostedtoolcache/CodeQL
60+
sudo docker system prune -af
61+
62+
- name: Setup Java 17
63+
if: steps.cache.outputs.cache-hit != 'true'
64+
uses: actions/setup-java@v5
65+
with:
66+
distribution: "zulu"
67+
java-version: 17
68+
69+
- name: Install Expo CLI
70+
if: steps.cache.outputs.cache-hit != 'true' && inputs.expo-prebuild == 'true'
71+
shell: bash
72+
run: |
73+
npm install -g @expo/cli
74+
echo "$(npm prefix -g)/bin" >> $GITHUB_PATH
75+
76+
- name: Generate native Android project
77+
if: steps.cache.outputs.cache-hit != 'true' && inputs.expo-prebuild == 'true'
78+
working-directory: ${{ inputs.app-path }}
79+
shell: bash
80+
run: |
81+
rm -rf android
82+
npx expo prebuild --platform android --no-install
83+
84+
# Note: not caching ~/.gradle or apps/<app>/android/.gradle on purpose.
85+
# A full Gradle cache for this monorepo runs ~3 GB and competes with the
86+
# 10 GB repo cache quota against the much smaller (~250 B) build markers.
87+
# Markers gating entire build cells beat ~5-10 min of Gradle warm-up.
88+
- name: Build app
89+
if: steps.cache.outputs.cache-hit != 'true'
90+
working-directory: ${{ inputs.app-path }}/android
91+
shell: bash
92+
run: |
93+
./gradlew assembleDebug \
94+
--build-cache \
95+
--parallel \
96+
--daemon \
97+
--configure-on-demand \
98+
-PreactNativeArchitectures=arm64-v8a \
99+
-Dorg.gradle.jvmargs="-Xmx4g -XX:+HeapDumpOnOutOfMemoryError" \
100+
-Dorg.gradle.workers.max=4
101+
102+
- name: Save pass marker
103+
if: steps.cache.outputs.cache-hit != 'true' && success()
104+
shell: bash
105+
run: |
106+
mkdir -p "${{ runner.temp }}/ci-marker"
107+
touch "${{ runner.temp }}/ci-marker/passed"
108+
109+
- name: Cache pass marker
110+
if: steps.cache.outputs.cache-hit != 'true' && success()
111+
uses: actions/cache/save@v4
112+
with:
113+
path: ${{ runner.temp }}/ci-marker
114+
key: ${{ steps.hash.outputs.key }}

‎.github/actions/build-ios-app/action.yml‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
name: Build iOS app
2+
description: Build an iOS demo app from this monorepo (with optional Expo prebuild)
3+
4+
inputs:
5+
app-path:
6+
description: Path to the app workspace, relative to the repo root (e.g. apps/llm)
7+
required: true
8+
expo-prebuild:
9+
description: Whether to run `expo prebuild --platform ios` before pod install
10+
required: false
11+
default: "true"
12+
filter-name:
13+
description: dorny/paths-filter filter name for this app+platform (e.g. llm-ios). Used as the content-hash cache key so a previously-passing build can be skipped if nothing relevant has changed.
14+
required: true
15+
16+
runs:
17+
using: composite
18+
steps:
19+
- name: Setup Node.js
20+
uses: actions/setup-node@v6
21+
with:
22+
node-version-file: .nvmrc
23+
cache: "yarn"
24+
25+
- name: Install root dependencies
26+
shell: bash
27+
run: yarn install --immutable
28+
29+
- name: Compute build hash
30+
id: hash
31+
shell: bash
32+
run: |
33+
h=$(npx ts-node scripts/compute-app-hash.ts "${{ inputs.filter-name }}")
34+
echo "key=build-${{ inputs.filter-name }}-$h" >> "$GITHUB_OUTPUT"
35+
36+
- name: Lookup pass marker
37+
id: cache
38+
uses: actions/cache/restore@v4
39+
with:
40+
path: ${{ runner.temp }}/ci-marker
41+
key: ${{ steps.hash.outputs.key }}
42+
lookup-only: true
43+
44+
- name: Skip notice
45+
if: steps.cache.outputs.cache-hit == 'true'
46+
shell: bash
47+
run: echo "Skipping build — ${{ inputs.filter-name }} already passed at this content hash."
48+
49+
- name: Setup Xcode
50+
if: steps.cache.outputs.cache-hit != 'true'
51+
uses: maxim-lobanov/setup-xcode@v1
52+
with:
53+
xcode-version: latest-stable
54+
55+
- name: Install Expo CLI
56+
if: steps.cache.outputs.cache-hit != 'true' && inputs.expo-prebuild == 'true'
57+
shell: bash
58+
run: |
59+
npm install -g @expo/cli
60+
echo "$(npm prefix -g)/bin" >> $GITHUB_PATH
61+
62+
- name: Generate native iOS project
63+
if: steps.cache.outputs.cache-hit != 'true' && inputs.expo-prebuild == 'true'
64+
working-directory: ${{ inputs.app-path }}
65+
shell: bash
66+
run: |
67+
rm -rf ios
68+
npx expo prebuild --platform ios --no-install
69+
70+
# Note: not caching CocoaPods on purpose. A populated Pods/ runs ~500 MB
71+
# per app — ~2.5 GB across the 5 iOS cells — and competes with the 10 GB
72+
# repo cache quota against the much smaller (~250 B) build markers.
73+
# Markers gating entire build cells beat ~3-5 min of pod-install warm-up.
74+
- name: Install CocoaPods dependencies
75+
if: steps.cache.outputs.cache-hit != 'true'
76+
working-directory: ${{ inputs.app-path }}/ios
77+
shell: bash
78+
run: pod install
79+
80+
- name: Build app
81+
if: steps.cache.outputs.cache-hit != 'true'
82+
working-directory: ${{ inputs.app-path }}/ios
83+
shell: bash
84+
run: |
85+
WORKSPACE=$(ls -d *.xcworkspace | head -n 1)
86+
SCHEME="${WORKSPACE%.xcworkspace}"
87+
set -o pipefail && xcodebuild \
88+
-workspace "$WORKSPACE" \
89+
-scheme "$SCHEME" \
90+
-sdk iphonesimulator \
91+
-configuration Debug \
92+
-destination 'generic/platform=iOS Simulator' \
93+
build \
94+
CODE_SIGNING_ALLOWED=NO \
95+
-jobs $(sysctl -n hw.ncpu) \
96+
COMPILER_INDEX_STORE_ENABLE=NO \
97+
ONLY_ACTIVE_ARCH=YES | xcbeautify
98+
99+
- name: Save pass marker
100+
if: steps.cache.outputs.cache-hit != 'true' && success()
101+
shell: bash
102+
run: |
103+
mkdir -p "${{ runner.temp }}/ci-marker"
104+
touch "${{ runner.temp }}/ci-marker/passed"
105+
106+
- name: Cache pass marker
107+
if: steps.cache.outputs.cache-hit != 'true' && success()
108+
uses: actions/cache/save@v4
109+
with:
110+
path: ${{ runner.temp }}/ci-marker
111+
key: ${{ steps.hash.outputs.key }}

‎.github/workflows/README.md‎

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# CI workflows
2+
3+
Three GitHub Actions workflows run on every PR:
4+
5+
| Workflow | Jobs |
6+
|---|---|
7+
| [`ci.yml`](ci.yml) | `lint` (eslint + `check-ci-filter-coverage.ts` + typecheck) and `build-library` (`yarn workspaces ... prepare`). Skips markdown / docs-only changes via `paths-ignore`. |
8+
| [`test-bare-rn.yml`](test-bare-rn.yml) | Jest smoke test for `apps/bare-rn` |
9+
| [`build-apps.yml`](build-apps.yml) | Per-app build matrix — bundles + native builds for all 5 demo apps on Android + iOS |
10+
11+
All three workflows skip draft PRs via `if: github.event.pull_request.draft != true`. `build-apps.yml` and `test-bare-rn.yml` additionally short-circuit on forks (`github.repository == 'software-mansion/react-native-executorch'`) so the expensive native matrix doesn't run on contributor forks.
12+
13+
The rest of this README explains `build-apps.yml`, which is the expensive one and has the most moving parts.
14+
15+
## The per-app build matrix
16+
17+
A naive matrix runs ~20 min × 10 native cells × every push. Two layers cut that cost:
18+
19+
1. **Path filtering** narrows the matrix per push — only apps whose files actually changed enter the matrix.
20+
2. **Content-hash cache markers** let cells that already passed for the current code skip the build entirely.
21+
22+
Both layers read the same filter file: [`scripts/build-app-filters.yml`](../../scripts/build-app-filters.yml).
23+
24+
```
25+
push → detect-changes job → matrix of (app, platform) cells
26+
│ │
27+
│ └─ each cell: compute content hash → cache lookup → skip if marker exists
28+
│
29+
└─ git diff $before $head | matched against build-app-filters.yml
30+
```
31+
32+
## Filter file: `scripts/build-app-filters.yml`
33+
34+
Single source of truth for "which files belong to which app". Uses YAML anchors so each path is named once and composed up:
35+
36+
```yaml
37+
core-shared: &core-shared # files every app depends on
38+
- packages/react-native-executorch/src/{common,constants,errors,native,types,utils}/**
39+
- "{package.json,yarn.lock}"
40+
# ...
41+
android-shared: &android-shared # Android-only shared
42+
- .github/actions/build-android-app/**
43+
- packages/react-native-executorch/android/**
44+
cv-pkg: &cv-pkg # CV-specific TS modules + C++ models
45+
- packages/react-native-executorch/common/rnexecutorch/models/VisionModel.{cpp,h}
46+
- packages/react-native-executorch/common/rnexecutorch/models/{classification,...,vertical_ocr}/**
47+
- packages/react-native-executorch/src/{modules,hooks}/computer_vision/**
48+
# ...
49+
computer-vision-app: &computer-vision-app
50+
- *core-shared
51+
- *expo-fetcher
52+
- *cv-pkg
53+
- apps/computer-vision/**
54+
computer-vision-android: [*computer-vision-app, *android-shared]
55+
computer-vision-ios: [*computer-vision-app, *ios-shared]
56+
```
57+
58+
The final `<app>-<platform>` entries are what the matrix consumes. Everything else is composition.
59+
60+
### When to update the filter file
61+
62+
| Change | Update |
63+
|---|---|
64+
| New file under `apps/<existing-app>/**` | Nothing — already covered by `<app>-app` |
65+
| New model directory under `packages/react-native-executorch/common/rnexecutorch/models/<domain>/` | `<domain>-pkg` anchor (add the directory name into its brace-expansion list) |
66+
| New TS module under `src/modules/<domain>/` or `src/hooks/<domain>/` | `<domain>-pkg` anchor |
67+
| New file that affects every app (e.g. new utility under `src/common/`) | Usually already covered by `core-shared`'s `src/{common,constants,errors,native,types,utils}/**` |
68+
| New iOS-only / Android-only source under `packages/react-native-executorch/ios/` or `/android/` | Already covered by `ios-shared` / `android-shared` (broad `**` globs) |
69+
| New demo app under `apps/<new-app>/` | New `<new-app>-pkg` + `<new-app>-app` + `<new-app>-android` + `<new-app>-ios` anchors, plus the apps list in `build-apps.yml`'s `Compute matrices` step |
70+
71+
You don't have to remember this list. `scripts/check-ci-filter-coverage.ts` runs in `ci.yml`'s `lint` job and fails the PR if any file under `packages/react-native-executorch/` isn't matched by a filter — you'll get a list of orphans and which anchor to add them to.
72+
73+
If a file genuinely doesn't belong to any build (e.g. `tsconfig.doc.json`), add it to `ALLOWLIST` in that script instead.
74+
75+
## Push-incremental diff (instead of `dorny/paths-filter`)
76+
77+
`dorny/paths-filter@v3` silently ignores its `base:` input on `pull_request` events (you'll see `'base' input parameter is ignored when action is triggered by pull request event` in the log). It always uses GitHub's PR-files API, which returns the *cumulative* PR diff — so every push to a PR re-evaluates filters against the entire PR diff and pulls every app that has ever been touched into the matrix.
78+
79+
We replaced it with [`scripts/detect-changed-filters.ts`](../../scripts/detect-changed-filters.ts), which runs `git diff --name-only $base $head` and matches the result against `build-app-filters.yml` using picomatch. For `pull_request synchronize` events, `base` is `github.event.before` (the previous push SHA), so the diff is just what *this push* changed. For `opened` / `reopened` / `ready_for_review`, `github.event.before` isn't set and the script falls back to `pull_request.base.sha` (full PR diff — the right thing for those events).
80+
81+
## Cache markers
82+
83+
Even when a cell enters the matrix, [`compute-app-hash.ts`](../../scripts/compute-app-hash.ts) hashes every tracked file matched by that cell's filter (excluding `HASH_EXCLUDE`). The hash becomes a cache key: `build-<app>-<platform>-<hash>` (or `bundle-<platform>-<app>-<hash>` for the bundle job). If a marker with that key exists in the GitHub Actions cache, the cell exits before doing any real work.
84+
85+
Markers are tiny (~250 bytes each — they exist only as cache-presence proofs, no payload). They're saved at the end of every successful build and live under the per-PR ref (`refs/pull/N/merge`) plus `refs/heads/main` for main-branch builds.
86+
87+
### `HASH_EXCLUDE`
88+
89+
Editing a file in `core-shared` triggers every app's matrix cell, but most edits to those files don't actually change build behavior — workflow tweaks, filter-file edits, etc. The hash excludes a small allowlist of orchestration files so editing them re-runs the matrix but every cell still hits its existing marker:
90+
91+
```ts
92+
// compute-app-hash.ts
93+
const HASH_EXCLUDE = new Set([
94+
'.github/workflows/build-apps.yml',
95+
'scripts/build-app-filters.yml',
96+
'scripts/detect-changed-filters.ts',
97+
]);
98+
```
99+
100+
If you make a workflow edit that *does* change build behavior — pinning a new action version, changing a `with:` input on a composite, swapping `runs-on:`, editing `.github/actions/setup/` (Node version, install flags) — markers won't reflect it. Force-clear the relevant cache entries from **Actions → Caches** in the repo UI.
101+
102+
Edits inside the per-platform composite actions (`.github/actions/build-android-app/`, `.github/actions/build-ios-app/`) *do* invalidate markers automatically, because those paths are part of `android-shared` / `ios-shared` and contribute to the hash.
103+
104+
## Common scenarios
105+
106+
### "I pushed a one-line tweak to one app and CI fired everything"
107+
108+
It shouldn't — push-incremental diff narrows to just that app's filter. If it didn't, check:
109+
110+
- Did the tweak also touch a file in `core-shared`? `package.json`, `yarn.lock`, and anything under `packages/react-native-executorch/src/{common,constants,errors,native,types,utils}/` fan out to every app.
111+
- Is this the first push to a fresh PR? `pull_request opened` uses the PR base for the diff, which is correct but means the *entire PR diff* matches filters on the first run. Subsequent `synchronize` pushes will narrow.
112+
113+
### "I edited build-apps.yml and now everything is rebuilding from scratch"
114+
115+
`build-apps.yml` is in `HASH_EXCLUDE`, so cells should fan out but hit cache. If they're rebuilding for real:
116+
117+
- Did you also change something else under `core-shared` in the same push? That would invalidate every hash.
118+
- Did the markers get evicted? GitHub's cache quota is 10 GB per repo with LRU eviction. Markers themselves are tiny but the yarn `node_modules` cache (~550 MB × Linux + macOS) sits alongside them. Heavy churn on other branches can evict markers.
119+
120+
### "I want to force a full rebuild"
121+
122+
**Actions → Caches** in the repo UI. Filter by `build-` or `bundle-` prefix and delete the relevant entries. Next push will cache-miss and rebuild.
123+
124+
### "I added a new model directory and the lint job fails saying it's an orphan"
125+
126+
`scripts/check-ci-filter-coverage.ts` walks every tracked file under `packages/react-native-executorch/` and verifies it matches at least one filter. Fix by adding the new path to the right `<domain>-pkg` anchor in `scripts/build-app-filters.yml`. If the file genuinely doesn't belong to any build (it's a doc, a license, etc.), add it to `ALLOWLIST` in `check-ci-filter-coverage.ts`.
127+
128+
### "I'm working on a long-running branch and don't want CI on every push"
129+
130+
Open the PR in draft. All three workflows skip on `draft != true`. Marking the PR as ready_for_review fires the workflows fresh on the current HEAD.

0 commit comments

Comments
 (0)