Skip to content

Commit 28a08fe

Browse files
authored
chore(infra): optimize Android E2E build for lg runner (16 vCPUs, 48GB) (MetaMask#23869)
## **Description** This PR optimizes Android E2E builds to prevent "Gradle Daemon disappeared unexpectedly" crashes on the LG runner (16 vCPUs, 48GB RAM). ### Root Cause Analysis The "Daemon disappeared" symptom is consistent with **process termination under memory pressure** (e.g., OS OOM kill). On **lg (48GB)**, the previous GitHub Actions Gradle config (`-Xmx16g`, `workers.max=6`, `daemon=true`) could overlap with Node/Metro memory usage and native compilation spikes. ### Solution: Optimized Gradle Memory Settings Following [Gradle 8.10.2 Performance Best Practices](https://docs.gradle.org/8.10.2/userguide/performance.html), we tuned `gradle.properties.github` for the 48GB runner. #### Improved Gradle Logging for E2E - E2E builds now run Gradle with **`--stacktrace --info`** via `scripts/build.sh` to provide more actionable logs when the build fails. - Android CI uses **JDK 17** via `setup-e2e-env` defaults: [setup-e2e-env action](https://github.com/MetaMask/github-tools/blob/v1/.github/actions/setup-e2e-env/action.yml) #### JVM Memory Changes | Setting | Before | After | Reason | |---------|--------|-------|--------| | **Heap (`-Xmx`)** | 16GB | **12GB** | Leave room for Node.js/Metro | | **Initial Heap (`-Xms`)** | (none) | **4GB** | Set JVM initial heap to reduce early heap resizing: [Java launcher docs](https://docs.oracle.com/en/java/javase/17/docs/specs/man/java.html) | | **MaxGCPauseMillis** | (none) | **500ms** | Tune G1 pause-time goal for CI throughput: [G1 GC tuning guide](https://docs.oracle.com/en/java/javase/17/gctuning/garbage-first-g1-garbage-collector1.html) | | **ExitOnOutOfMemoryError** | (none) | **enabled** | Fail fast on JVM OOM: [Java launcher docs](https://docs.oracle.com/en/java/javase/17/docs/specs/man/java.html) | | **file.encoding** | (none) | **UTF-8** | Pin JVM default charset (JDK 17 default can depend on OS/locale): [Charset.defaultCharset()](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/nio/charset/Charset.html#defaultCharset()) | | **OptimizeStringConcat** | enabled | **removed** | Remove non-standard `-XX` tuning flag from baseline config (prefer defaults): [Java launcher docs](https://docs.oracle.com/en/java/javase/17/docs/specs/man/java.html) | #### Gradle Settings Changes | Setting | Before | After | Reason | |---------|--------|-------|--------| | **Workers** | 6 | **2** | Prevent memory contention | | **Daemon** | true | **false** | Disable persistent daemon between CI builds: [Gradle Daemon](https://docs.gradle.org/8.10.2/userguide/gradle_daemon.html#sec:disabling_the_daemon) | | **configureondemand** | true | **removed** | Not recommended for modern Gradle + Android builds: [Gradle performance guide](https://docs.gradle.org/8.10.2/userguide/performance.html); [Android/AGP compatibility](https://stackoverflow.com/questions/49990933/configuration-on-demand-is-not-supported-by-the-current-version-of-the-android-g) | #### Unchanged Settings (kept as-is) | Setting | Value | Why Kept | |---------|-------|----------| | `parallel` | true | [Recommended for multi-project builds](https://docs.gradle.org/8.10.2/userguide/performance.html#parallel_execution) | | `caching` | true | [Recommended build cache usage](https://docs.gradle.org/8.10.2/userguide/build_cache.html) | | `vfs.watch` | false | Already disabled in baseline config; we keep it disabled for CI | | `MaxMetaspaceSize` | 1g | Kept from baseline config | | `UseG1GC` | enabled | Kept from baseline config | | `G1HeapRegionSize` | 16m | Kept from baseline config | | `UseStringDeduplication` | enabled | Kept from baseline config | ### Memory Budget (48GB Runner) > Note: rough budgeting (actual usage varies by task and input changes) ``` ┌─────────────────────────────────────────────────────────┐ │ Component │ Allocation │ ├─────────────────────────┼───────────────────────────────┤ │ Gradle Heap │ 12GB │ │ Gradle Metaspace │ 1GB │ │ Node.js (Metro) │ 8GB (--max-old-space-size) │ │ OS + System + native │ remainder │ └─────────────────────────┴───────────────────────────────┘ ``` ### Additional Optimizations - **Skip AAB bundle for E2E** - E2E tests only use APK files. - **Removed AAB references** from the E2E build workflow. - **Runner moved from xl → lg** for this workflow after tuning: lg (48GB) is sufficient. ### Documentation References **Gradle 8.10.2:** - [Performance Guide](https://docs.gradle.org/8.10.2/userguide/performance.html) - [Gradle Daemon](https://docs.gradle.org/8.10.2/userguide/gradle_daemon.html) - [Build Cache](https://docs.gradle.org/8.10.2/userguide/build_cache.html) **Java 17:** - [Java launcher docs (heap, -X/-XX, OOM behavior)](https://docs.oracle.com/en/java/javase/17/docs/specs/man/java.html) - [Charset.defaultCharset()](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/nio/charset/Charset.html#defaultCharset()) - [G1 GC tuning guide](https://docs.oracle.com/en/java/javase/17/gctuning/garbage-first-g1-garbage-collector1.html) ### Official Runner Specs ([source](https://cirrus-runners.app/setup/#__tabbed_3_2)) | Runner | vCPUs | RAM | Disk | |--------|-------|-----|------| | **lg** | 16 | 48 GB | 200 GB | | xl | 32 | 96 GB | 400 GB | ## **Changelog** CHANGELOG entry: null ## **Related issues** Fixes: [INFRA-3174](https://consensyssoftware.atlassian.net/browse/INFRA-3174) ## **Manual testing steps** ```gherkin Feature: Android E2E Build Optimization Scenario: Build completes without daemon disappearance Given the PR uses tuned Gradle memory settings And Gradle runs with --stacktrace --info for E2E When the Android E2E build workflow runs Then the build completes without the "Daemon disappeared" failure And APK artifacts are uploaded successfully ``` ## **Screenshots/Recordings** ### **Before** Build failing with: ``` Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed) ``` ### **After** Build results will be visible in PR checks after workflow runs. ## **Pre-merge author checklist** - [x] I've followed [MetaMask Contributor Docs](https://github.com/MetaMask/contributor-docs) and [MetaMask Mobile Coding Standards](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/CODING_GUIDELINES.md). - [x] I've completed the PR template to the best of my ability - [x] I've included tests if applicable - [x] I've documented my code using [JSDoc](https://jsdoc.app/) format if applicable - [x] I've applied the right labels on the PR (see [labeling guidelines](https://github.com/MetaMask/metamask-mobile/blob/main/.github/guidelines/LABELING_GUIDELINES.md)). Not required for external contributors. ## **Pre-merge reviewer checklist** - [ ] I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed). - [ ] I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots. [INFRA-3174]: https://consensyssoftware.atlassian.net/browse/INFRA-3174?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
1 parent 17ef3b4 commit 28a08fe

4 files changed

Lines changed: 17 additions & 33 deletions

File tree

.github/workflows/build-android-e2e.yml

Lines changed: 1 addition & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,6 @@ on:
99
apk-uploaded:
1010
description: 'Whether the APK was successfully uploaded'
1111
value: ${{ jobs.build-android-apks.outputs.apk-uploaded }}
12-
aab-uploaded:
13-
description: 'Whether the AAB was successfully uploaded'
14-
value: ${{ jobs.build-android-apks.outputs.aab-uploaded }}
1512
inputs:
1613
build_type:
1714
description: 'The type of build to perform'
@@ -32,17 +29,15 @@ on:
3229
jobs:
3330
build-android-apks:
3431
name: Build Android E2E APKs
35-
runs-on: ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-xl # Bumped from lg to xl to prevent Daemon disappearance issue (Daemon OOM issue in CI)
32+
runs-on: ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-lg # lg runner: 16 vCPUs, 48GB RAM
3633
timeout-minutes: 40
3734
env:
3835
GRADLE_USER_HOME: /home/admin/_work/.gradle
3936
CACHE_GENERATION: v1 # Increment this to bust the cache (v1, v2, v3, etc.)
4037
outputs:
4138
apk-uploaded: ${{ steps.upload-apk.outcome == 'success' }}
42-
aab-uploaded: ${{ steps.upload-aab.outcome == 'success' }}
4339
apk-target-path: ${{ steps.determine-target-paths.outputs.apk-target-path }}
4440
test-apk-target-path: ${{ steps.determine-target-paths.outputs.test-apk-target-path }}
45-
aab-target-path: ${{ steps.determine-target-paths.outputs.aab-target-path }}
4641
artifact_name: ${{ steps.determine-target-paths.outputs.artifact_name }}
4742

4843
steps:
@@ -88,14 +83,12 @@ jobs:
8883
{
8984
echo "apk-target-path=android/app/build/outputs/apk/flask/release"
9085
echo "test-apk-target-path=android/app/build/outputs/apk/androidTest/flask/release"
91-
echo "aab-target-path=android/app/build/outputs/bundle/flaskRelease"
9286
echo "artifact_name=app-flask-release"
9387
} >> "$GITHUB_OUTPUT"
9488
elif [[ "${{ inputs.build_type }}" == "main" ]]; then
9589
{
9690
echo "apk-target-path=android/app/build/outputs/apk/prod/release"
9791
echo "test-apk-target-path=android/app/build/outputs/apk/androidTest/prod/release"
98-
echo "aab-target-path=android/app/build/outputs/bundle/prodRelease"
9992
echo "artifact_name=app-prod-release"
10093
} >> "$GITHUB_OUTPUT"
10194
else
@@ -110,7 +103,6 @@ jobs:
110103
path: |
111104
${{ steps.determine-target-paths.outputs.apk-target-path }}/${{ steps.determine-target-paths.outputs.artifact_name }}.apk
112105
${{ steps.determine-target-paths.outputs.test-apk-target-path }}/${{ steps.determine-target-paths.outputs.artifact_name }}-androidTest.apk
113-
${{ steps.determine-target-paths.outputs.aab-target-path }}/${{ steps.determine-target-paths.outputs.artifact_name }}.aab
114106
# Include Gradle properties in key to force rebuild when properties change
115107
# Keep the `hashFiles` call for Gradle config in-sync with these steps:
116108
# - "Cache Gradle dependencies"
@@ -241,7 +233,6 @@ jobs:
241233
path: |
242234
${{ steps.determine-target-paths.outputs.apk-target-path }}/${{ steps.determine-target-paths.outputs.artifact_name }}.apk
243235
${{ steps.determine-target-paths.outputs.test-apk-target-path }}/${{ steps.determine-target-paths.outputs.artifact_name }}-androidTest.apk
244-
${{ steps.determine-target-paths.outputs.aab-target-path }}/${{ steps.determine-target-paths.outputs.artifact_name }}.aab
245236
# Keep the `hashFiles` call for Gradle config in-sync with these steps:
246237
# - "Check and restore cached APKs if Fingerprint is found"
247238
# - "Cache Gradle dependencies"
@@ -264,13 +255,3 @@ jobs:
264255
path: ${{ steps.determine-target-paths.outputs.test-apk-target-path }}/${{ steps.determine-target-paths.outputs.artifact_name }}-androidTest.apk
265256
retention-days: 7
266257
if-no-files-found: error
267-
268-
- name: Upload Android AAB
269-
id: upload-aab
270-
uses: actions/upload-artifact@v4
271-
with:
272-
name: ${{ inputs.build_type }}-${{ inputs.metamask_environment }}-release.aab
273-
path: ${{ steps.determine-target-paths.outputs.aab-target-path }}/${{ steps.determine-target-paths.outputs.artifact_name }}.aab
274-
retention-days: 7
275-
if-no-files-found: warn
276-
continue-on-error: true

.github/workflows/run-e2e-workflow.yml

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,6 @@ jobs:
5656
outputs:
5757
apk-target-path: ${{ steps.determine-target-paths.outputs.apk-target-path }}
5858
test-apk-target-path: ${{ steps.determine-target-paths.outputs.test-apk-target-path }}
59-
aab-target-path: ${{ steps.determine-target-paths.outputs.aab-target-path }}
6059

6160
env:
6261
PREBUILT_IOS_APP_PATH: artifacts/MetaMask.app
@@ -131,14 +130,12 @@ jobs:
131130
{
132131
echo "apk-target-path=android/app/build/outputs/apk/flask/release"
133132
echo "test-apk-target-path=android/app/build/outputs/apk/androidTest/flask/release"
134-
echo "aab-target-path=android/app/build/outputs/bundle/flaskRelease"
135133
echo "artifact_name=app-flask-release"
136134
} >> "$GITHUB_OUTPUT"
137135
elif [[ "${{ inputs.build_type }}" == "main" ]]; then
138136
{
139137
echo "apk-target-path=android/app/build/outputs/apk/prod/release"
140138
echo "test-apk-target-path=android/app/build/outputs/apk/androidTest/prod/release"
141-
echo "aab-target-path=android/app/build/outputs/bundle/prodRelease"
142139
echo "artifact_name=app-prod-release"
143140
} >> "$GITHUB_OUTPUT"
144141
else
@@ -152,7 +149,6 @@ jobs:
152149
echo "🏗 Setting up Android artifacts from build job..."
153150
mkdir -p ${{ steps.determine-target-paths.outputs.apk-target-path }}
154151
mkdir -p ${{ steps.determine-target-paths.outputs.test-apk-target-path }}
155-
mkdir -p ${{ steps.determine-target-paths.outputs.aab-target-path }}
156152
157153
- name: Download Android build artifacts
158154
if: ${{ inputs.platform == 'android' }}

android/gradle.properties.github

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
# GitHub Actions-specific Gradle settings
22
# Optimized for E2E builds on GitHub Actions runners
33

4-
# JVM configuration - balanced settings to avoid OOM while maintaining performance
5-
# Using 16GB heap to leave room for parallel workers and native memory
6-
org.gradle.jvmargs=-Xmx16g -XX:MaxMetaspaceSize=1g -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:+UseStringDeduplication -XX:+OptimizeStringConcat
4+
# JVM configuration - tuned for 48GB runner to avoid OOM while maintaining performance
5+
# Heap: 12GB to leave room for Node.js/Metro and native memory
6+
# ExitOnOutOfMemoryError: fail-fast on OOM for CI
7+
org.gradle.jvmargs=-Xmx12g -Xms4g -XX:MaxMetaspaceSize=1g -XX:+UseG1GC -XX:G1HeapRegionSize=16m -XX:+UseStringDeduplication -XX:MaxGCPauseMillis=500 -XX:+ExitOnOutOfMemoryError -Dfile.encoding=UTF-8
78

89
# Enable performance optimizations but limit parallelism to prevent OOM
910
org.gradle.parallel=true
10-
org.gradle.configureondemand=true
1111
org.gradle.caching=true
12-
org.gradle.daemon=true
13-
org.gradle.workers.max=6
12+
org.gradle.daemon=false
13+
org.gradle.workers.max=2
1414
org.gradle.vfs.watch=false
1515

1616
# CI-specific optimizations - enabled for GitHub Actions
@@ -54,4 +54,4 @@ hermesEnabled=true
5454
android.disableResourceValidation=true
5555

5656
# Use legacy packaging to compress native libraries in the resulting APK.
57-
expo.useLegacyPackaging=false
57+
expo.useLegacyPackaging=false

scripts/build.sh

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -547,6 +547,8 @@ generateAndroidBinary() {
547547
local reactNativeArchitecturesArg=""
548548
# Define Test build type arg
549549
local testBuildTypeArg=""
550+
# Define Gradle debug flags
551+
local gradleDebugFlags=""
550552

551553
# Check if configuration is valid
552554
if [ "$configuration" != "Debug" ] && [ "$configuration" != "Release" ] ; then
@@ -572,14 +574,19 @@ generateAndroidBinary() {
572574
if [ "$METAMASK_ENVIRONMENT" = "e2e" ] ; then
573575
# Only build for x86_64 for E2E builds
574576
reactNativeArchitecturesArg="-PreactNativeArchitectures=x86_64"
577+
# Enable Gradle debugging flags for E2E builds to investigate Daemon disappearance issues
578+
gradleDebugFlags="--stacktrace --info"
579+
echo "📊 E2E build: Enabling Gradle debugging flags (--stacktrace --info)"
575580
fi
576581
fi
577582

578583
# Generate Android APKs
579584
echo "Generating Android binary for ($flavor) flavor with ($configuration) configuration"
580-
./gradlew $assembleApkTask $assembleTestApkTask $testBuildTypeArg $reactNativeArchitecturesArg
585+
./gradlew $assembleApkTask $assembleTestApkTask $testBuildTypeArg $reactNativeArchitecturesArg $gradleDebugFlags
581586

582-
if [ "$configuration" = "Release" ] ; then
587+
# Skip AAB bundle for E2E environments - AAB cannot be installed on emulators
588+
# and is only needed for Play Store distribution
589+
if [ "$configuration" = "Release" ] && [ "$METAMASK_ENVIRONMENT" != "e2e" ] ; then
583590
# Generate AAB bundle (not needed for E2E)
584591
bundleConfiguration="bundle${flavor}Release"
585592
echo "Generating AAB bundle for ($flavor) flavor with ($configuration) configuration"

0 commit comments

Comments
 (0)