Skip to content

test: mitigate e2e simulator hang / retry flakes#9057

Open
mikehardy wants to merge 6 commits into
mainfrom
continued-e2e-deflake
Open

test: mitigate e2e simulator hang / retry flakes#9057
mikehardy wants to merge 6 commits into
mainfrom
continued-e2e-deflake

Conversation

@mikehardy

Copy link
Copy Markdown
Collaborator

Summary

This PR is intended to hold a continued series of e2e flake fixes.

  1. Simulator hang / Jet retry recovery — Preserve the mocha-remote runner across transient Jet WS disconnects, kill and reboot the resolved iOS simulator before retries, and ensure debug Jet retries still wait for Metro before relaunching the app.

Test plan

  • iOS E2E debug CI passes on merge (or Jet attempt-2 retry recovers cleanly after a transient 1006/session desync)
  • iOS E2E release CI remains green

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a series of stability improvements for end-to-end testing, specifically targeting flaky simulator behavior and WebSocket connection issues. By enhancing the recovery logic for both the test runner and the underlying iOS simulator, the changes aim to reduce CI noise caused by transient infrastructure failures.

Highlights

  • Mocha Remote Server Resilience: Updated the mocha-remote-server to handle transient WebSocket disconnects by introducing a grace period and state management to preserve the test runner during brief network interruptions.
  • iOS Simulator Recovery: Implemented a more robust recovery mechanism for iOS E2E tests by rebooting the simulator when specific launch failures occur, rather than simply terminating the app.
  • E2E Test Stability: Expanded the list of retryable launch failures and added caching for Metro status to improve the reliability of the test suite execution.
Ignored Files
  • Ignored by pattern: .github/workflows/** (1)
    • .github/workflows/scripts/boot-simulator.sh
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a patch to mocha-remote-server to handle transient client disconnects with a reconnect grace timer, and updates the E2E test suite to reboot the iOS simulator upon retryable launch failures. Feedback highlights a critical race condition in the server patch where a restarted client process may hang waiting for a run command that is never sent. Additionally, the synchronous reboot of the iOS simulator blocks the Node.js event loop, potentially freezing the WebSocket server; it is recommended to refactor this to be asynchronous and properly awaited.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread .yarn/patches/mocha-remote-server-npm-1.13.2-619a29d2e3.patch
Comment thread tests/e2e/firebase.test.js
Comment thread tests/e2e/firebase.test.js
@codecov

codecov Bot commented Jun 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 70.58824% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.80%. Comparing base (74ddc9e) to head (063a53f).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9057      +/-   ##
============================================
- Coverage     60.86%   60.80%   -0.06%     
+ Complexity     1602     1601       -1     
============================================
  Files           455      455              
  Lines         33589    33602      +13     
  Branches       5438     5476      +38     
============================================
- Hits          20441    20428      -13     
- Misses        12016    12043      +27     
+ Partials       1132     1131       -1     
Flag Coverage Δ
android-native 48.55% <11.12%> (-<0.01%) ⬇️
e2e-ts-android 48.55% <11.12%> (-<0.01%) ⬇️
e2e-ts-ios 51.47% <16.67%> (-0.07%) ⬇️
e2e-ts-macos 25.65% <11.12%> (ø)
ios-native 51.47% <16.67%> (?)
jest 62.24% <80.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Cap Gradle workers at min(physical_cpus, 6) to limit parallel heap
pressure; 5GB daemon heap handles peak packageDebugAndroidTest load.
Scales with hardware without overwhelming low-core CI machines.
@mikehardy mikehardy force-pushed the continued-e2e-deflake branch from 8e498e5 to 12692b7 Compare June 21, 2026 13:52
…ilures

Run modular getSessionId probes before all other analytics tests; drop
namespace getSessionId coverage to avoid cross-test session interference.
Inline CI at Metro bundle time so Jet tests on device see the CI runner
flag instead of an undefined process.env lookup.
@mikehardy mikehardy force-pushed the continued-e2e-deflake branch from 897a66b to 98c2229 Compare June 21, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant