Skip to content

Commit aa6b1f1

Browse files
committed
docs(test): clarify e2e pre-flight requirements
1 parent 05f83e8 commit aa6b1f1

6 files changed

Lines changed: 57 additions & 35 deletions

File tree

okf-bundle/ci-workflows/ios.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ On GHA macOS runners, `simctl list` can show `Booted` before the simulator is te
1616

1717
**Pre-boot step** (`.github/workflows/scripts/boot-simulator.sh`), run via `nick-fields/retry` before Detox:
1818

19-
> **Not for local operators.** `boot-simulator.sh` is **CI-only** (this workflow) or invoked **internally** by `tests/e2e/firebase.test.js` on iOS Jet-level retry. Local e2e uses only [running-e2e.md](../testing/running-e2e.md) — Detox boots the simulator via `yarn tests:ios:test-cover`; do not run `boot-simulator.sh` as operator prep.
19+
> **Not for local operators.** `boot-simulator.sh` is **CI-only** (this workflow) or invoked **internally** by `tests/e2e/firebase.test.js` on iOS Jet-level retry. Local e2e uses only [running-e2e.md](../testing/running-e2e.md)[host-clear probes](../testing/running-e2e.md#host-clear-probes) require zero booted simulators; Detox boots `iPhone 17` via `yarn tests:ios:test-cover`. Do not run `boot-simulator.sh` as operator prep.
2020
2121
| Phase | What happens |
2222
|--------|----------------|

okf-bundle/packages/firestore/pipeline-coverage-work-queue.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ Ephemeral tracker; see [OKF policy](../../documentation-policy.md).
3838

3939
Gate prerequisites before any `:test-cover` ([host rule](../../testing/iteration-vocabulary.md#host-rule)):
4040

41-
1. [Pre-flight](../../testing/running-e2e.md#pre-flight-is-the-host-clear-to-start): host clear, [services ready](../../testing/running-e2e.md#2-services-ready), [harness matches validation tier](../../testing/running-e2e.md#3-harness-matches-validation-tier) ([narrowing gate](../../testing/running-e2e.md#harness-narrowing-gate-blocking) — required for **focused** and **area**; not [push harness](#harness)); [serial `:test-cover](../../testing/running-e2e.md#serialized-e2e-dispatch)`; [frozen tree](../../testing/iteration-vocabulary.md#frozen-tree) for `independent-review`.
41+
1. [Pre-flight](../../testing/running-e2e.md#pre-flight-is-the-host-clear-to-start): [host-clear probes](../../testing/running-e2e.md#host-clear-probes), [services ready](../../testing/running-e2e.md#2-services-ready), [harness matches validation tier](../../testing/running-e2e.md#3-harness-matches-validation-tier) ([narrowing gate](../../testing/running-e2e.md#harness-narrowing-gate-blocking) — required for **focused** and **area**; not [push harness](#harness)); [serial `:test-cover](../../testing/running-e2e.md#serialized-e2e-dispatch)`; [frozen tree](../../testing/iteration-vocabulary.md#frozen-tree) for `independent-review`.
4242
2. Guard probes: [SDK runtime verification](pipeline-sdk-support-audit.md#6-runtime-verification-authoritative) + [Phase J protocol](#phase-j-iteration-protocol-strict) below.
4343
3. Coverage deltas: full clean cycle; never trust stale `.ec`/profraw ([coverage stale data](../../testing/coverage-design.md#stale-coverage-data)).
4444

okf-bundle/packages/firestore/pipeline-implementation-workflow.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@ If review fails, return to `implementation`, then repeat `independent-review`.
103103

104104
iOS guard probes/bridge fixes use the same split; stricter serial gate: [work queue runtime guard protocol](pipeline-coverage-work-queue.md#phase-j-iteration-protocol-strict) + [running-e2e serial policy](../../testing/running-e2e.md#serialized-e2e-dispatch).
105105

106-
Guard probes: one function per commit; no batching. [Pre-flight](../../testing/running-e2e.md#pre-flight-is-the-host-clear-to-start) includes services ready + harness tier. Commands: [running-e2e.md](../../testing/running-e2e.md).
106+
Guard probes: one function per commit; no batching. [Pre-flight](../../testing/running-e2e.md#pre-flight-is-the-host-clear-to-start) (by reference). Commands: [running-e2e.md](../../testing/running-e2e.md).
107107

108108
**Live gate status:** [pipeline-coverage-work-queue](pipeline-coverage-work-queue.md) (ephemeral tracker — see [documentation policy](../../documentation-policy.md)).
109109

@@ -158,7 +158,7 @@ Skip only when continuing immediately after same-worktree `after-<export>` snaps
158158
**Retroactive baseline:** If implementation was already committed without baseline numbers, check out the parent commit, apply harness only if needed for e2e load, run full baseline, snapshot `before-<export>`, return to HEAD.
159159

160160
1. Revert leftover `.only`; area narrowing allowed for pipeline baseline.
161-
2. Start [running e2e pre-flight](../../testing/running-e2e.md#pre-flight-is-the-host-clear-to-start) (services + harness tier) and prerequisites: emulators, packager, rebuild native if needed.
161+
2. [Pre-flight](../../testing/running-e2e.md#pre-flight-is-the-host-clear-to-start) and prerequisites ([Rules §1–2](../../testing/running-e2e.md#rules)); rebuild native if needed.
162162
3. Full pipeline e2e + coverage on **macOS, iOS, Android**; no reuse variants.
163163
4. `bash scripts/snapshot-pipeline-coverage.sh before-<export-name>` — paste full stdout into the iteration report.
164164

okf-bundle/testing/iteration-vocabulary.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@ See also: [running e2e rule 7](running-e2e.md#rules) and [host rule](running-e2e
6464
On a shared dev host:
6565

6666
- One `:test-cover` at a time — never overlap focused-tier and area-tier runs.
67-
- [Pre-flight](running-e2e.md#pre-flight-is-the-host-clear-to-start) before every run: host clear, [services ready](running-e2e.md#2-services-ready), [harness matches `validation_tier`](running-e2e.md#3-harness-matches-validation-tier) ([narrowing gate](running-e2e.md#harness-narrowing-gate-blocking) for `focused` and `area`).
67+
- [Pre-flight](running-e2e.md#pre-flight-is-the-host-clear-to-start) before every run: [host-clear probes](running-e2e.md#host-clear-probes), [services ready](running-e2e.md#2-services-ready), [harness matches `validation_tier`](running-e2e.md#3-harness-matches-validation-tier) ([narrowing gate](running-e2e.md#harness-narrowing-gate-blocking) for `focused` and `area`).
6868
- Canonical commands only — [running e2e](running-e2e.md). Stalled runs → [stalled run detection](running-e2e.md#stalled-run-detection).
6969

7070
## Work-queue fields

okf-bundle/testing/running-e2e.md

Lines changed: 51 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ macOS: `yarn tests:macos:test-cover` → `jet --target=macos` (same `:8090`).
7676

7777
### Running one iteration
7878

79-
1. [Pre-flight](#pre-flight-is-the-host-clear-to-start); if busy/recovering, wait or clean.
79+
1. [Pre-flight](#pre-flight-is-the-host-clear-to-start); if [host-clear probes](#host-clear-probes) fail, [pre-flight recovery](#pre-flight-recovery) first.
8080
2. One foreground Shell command; set `block_until_ms` large enough (~15m macOS, ~45–60m iOS/Android). Do **not** background/poll.
8181
3. From repo root, tee canonical command:
8282

@@ -103,20 +103,56 @@ Markers: `✨ Tests Complete ✨`, Jest `N passing` / `N failing`, `[jet-coverag
103103

104104
### Pre-flight: is the host clear to start?
105105

106-
Run **all three** checks before every `:test-cover`.
106+
**Canonical owner** for host-clear probes, recovery after abort, and service checks. Other OKF docs link here by reference — do not duplicate commands or probes elsewhere.
107+
108+
Run **all three** steps before every `:test-cover`. After an [interrupted run](#interrupted-run-abort-killed-terminal-eaddrinuse-on-8090), run [pre-flight recovery](#pre-flight-recovery) and re-run the probes.
107109

108110
#### 1. Host clear
109111

110112
No in-flight test run on the target platform:
111113

112-
| Platform | Active e2e signal | Clear to start |
113-
|----------|-------------------|----------------|
114-
| **Android** | `adb -s emulator-5554 shell pidof com.invertase.testing.test` returns a PID | command prints nothing / empty |
115-
| **iOS** | `xcrun simctl spawn booted pgrep -x testing` returns a PID | no output |
116-
| **macOS** | `pgrep -x io.invertase.testing` returns a PID | no output |
114+
| Platform | Clear when |
115+
|----------|------------|
116+
| **Android** | [Host-clear probes](#host-clear-probes) pass (no instrumentation PID) |
117+
| **iOS** | [Host-clear probes](#host-clear-probes) pass — **zero booted simulators** and no stray Jet on `:8090`. Detox boots `iPhone 17` from `tests/.detoxrc.js`; do not pre-boot or leave simulators running. |
118+
| **macOS** | [Host-clear probes](#host-clear-probes) pass (no `io.invertase.testing` process) |
117119

118120
Also wait for any visible unfinished `yarn tests:*:test-cover`.
119121

122+
<a id="host-clear-probes"></a>
123+
124+
**Host-clear probes** — run the block for your platform; **exit 0 = clear** (chain with `&&`):
125+
126+
```bash
127+
# iOS — booted-device count must be 0
128+
test "$(xcrun simctl list devices booted | grep -c '(Booted)' || true)" -eq 0
129+
test -z "$(lsof -nP -iTCP:8090 -sTCP:LISTEN -t 2>/dev/null || true)"
130+
131+
# Android
132+
! adb -s emulator-5554 shell pidof com.invertase.testing.test >/dev/null 2>&1
133+
134+
# macOS
135+
! pgrep -x io.invertase.testing >/dev/null 2>&1
136+
```
137+
138+
<a id="pre-flight-recovery"></a>
139+
140+
**Pre-flight recovery** — when probes fail after abort, kill, or `EADDRINUSE` on `:8090`. Then re-run [host-clear probes](#host-clear-probes).
141+
142+
```bash
143+
# Android
144+
adb -s emulator-5554 shell am force-stop com.invertase.testing
145+
adb -s emulator-5554 shell am force-stop com.invertase.testing.test
146+
147+
# iOS — Detox re-boots iPhone 17 after shutdown booted
148+
lsof -nP -iTCP:8090 -sTCP:LISTEN -t | xargs kill 2>/dev/null || true
149+
pkill -f 'detox test --configuration ios' 2>/dev/null || true
150+
pkill -f 'jet.js --target=ios' 2>/dev/null || true
151+
xcrun simctl shutdown booted
152+
```
153+
154+
Do **not** use `boot-simulator.sh` or `simctl shutdown all` as routine prep ([what not to do](#what-not-to-do)).
155+
120156
#### 2. Services ready
121157

122158
Metro and emulators must be **running and responsive** — do not assume from a prior session or background start.
@@ -153,7 +189,7 @@ Completion = shell exit code + log markers — not open-ended log tailing.
153189
| **macOS** | `Jet client connected` | `✨ Tests Complete ✨`, Jest `N passing` |
154190
| **iOS/Android** | Detox launch done, Jet connected | Same |
155191

156-
**If stalled** — no new markers for **5 minutes**, or past tier budget (~15m macOS, ~45–60m iOS/Android) without `Tests Complete`: treat as [interrupted run](#interrupted-run-abort-killed-terminal-eaddrinuse-on-8090). Re-check [services ready](#2-services-ready), diagnose (`curl` Metro `/status`, platform active-signal commands above, `lsof -nP -iTCP:8090`), retry. Do not keep watching flat tee output.
192+
**If stalled** — no new markers for **5 minutes**, or past tier budget (~15m macOS, ~45–60m iOS/Android) without `Tests Complete`: treat as [interrupted run](#interrupted-run-abort-killed-terminal-eaddrinuse-on-8090). Run [pre-flight recovery](#pre-flight-recovery), confirm [host-clear probes](#host-clear-probes) and [services ready](#2-services-ready), retry. Do not keep watching flat tee output.
157193

158194
- macOS bundle/Metro hangs → [ci-workflows/other.md § bundle load hang](../ci-workflows/other.md#ci-failure-bundle-load-hang--could-not-connect-to-development-server)
159195
- iOS Metro at launch → [ci-workflows/ios.md § Metro unresponsive](../ci-workflows/ios.md)
@@ -184,7 +220,7 @@ Do not poll `pgrep` / `jet.js` / `:8090` for *completion* ([above](#how-a-platfo
184220

185221
For `implementation` work type ([focused tier](#e2e-validation-tiers-focused-area-full)):
186222

187-
1. [Pre-flight](#pre-flight-is-the-host-clear-to-start) — host clear, services ready, **harness narrowed** (step 3); if not, wait or clean per [interrupted run](#interrupted-run-abort-killed-terminal-eaddrinuse-on-8090).
223+
1. [Pre-flight](#pre-flight-is-the-host-clear-to-start)[host-clear probes](#host-clear-probes), services ready, **harness narrowed** (step 3); if probes fail, [pre-flight recovery](#pre-flight-recovery) first.
188224
2. Edit e2e/spec; add `.only` if needed; never commit narrowing.
189225
3. macOS first when TS-only: `yarn tests:macos:test-cover 2>&1 | tee /tmp/rnfb-e2e-macos.log` — wait for exit code ([stalled run](#stalled-run-detection) if markers stop).
190226
4. If macOS green and native touched: `yarn tests:<platform>:build && yarn tests:<platform>:test-cover 2>&1 | tee /tmp/rnfb-e2e-<platform>.log`; one platform at a time.
@@ -199,7 +235,7 @@ Never overlap runs that use `:test-cover`. See [host rule](iteration-vocabulary.
199235
|------|-------------|
200236
| **One e2e run at a time** | Wait for prior shell exit code + short log summary |
201237
| **No overlapping tiers** | Never run focused-tier and area-tier `:test-cover` concurrently on one host |
202-
| **Clean pre-flight every run** | [Pre-flight](#pre-flight-is-the-host-clear-to-start) — host, services, harness tier |
238+
| **Clean pre-flight every run** | [Pre-flight](#pre-flight-is-the-host-clear-to-start)[host-clear probes](#host-clear-probes), services, harness tier |
203239
| **iOS guard probe loop** | `implementation` (Jest + **focused**) → `independent-review` (**area**, frozen tree) → `commit`[work queue protocol](../packages/firestore/pipeline-coverage-work-queue.md#phase-j-iteration-protocol-strict) |
204240

205241
| Validation tier | E2e scope | Narrowing allowed | Typical work type |
@@ -212,21 +248,7 @@ Each run owns its blocking `:test-cover` and returns summaries only.
212248

213249
### Interrupted run (abort, killed terminal, EADDRINUSE on :8090)
214250

215-
Before next canonical command:
216-
217-
```bash
218-
# Android — stop app + instrumentation
219-
adb -s emulator-5554 shell am force-stop com.invertase.testing
220-
adb -s emulator-5554 shell am force-stop com.invertase.testing.test
221-
222-
# Stray Jet from an aborted run (note PID, kill only jet.js for this repo)
223-
lsof -nP -iTCP:8090 -sTCP:LISTEN
224-
225-
# Confirm clear
226-
adb -s emulator-5554 shell pidof com.invertase.testing.test || echo "instrumentation clear"
227-
```
228-
229-
Then rerun from repo root: `yarn tests:<platform>:build && yarn tests:<platform>:test-cover` (foreground; tee if logging).
251+
Run [pre-flight recovery](#pre-flight-recovery), confirm [host-clear probes](#host-clear-probes) pass, then rerun from repo root: `yarn tests:<platform>:build && yarn tests:<platform>:test-cover` (foreground; tee if logging).
230252

231253
### What not to do
232254

@@ -284,17 +306,17 @@ All tiers use [canonical commands](#rules), [host rule](iteration-vocabulary.md#
284306
**Universal rules:**
285307

286308
- E2e is **always serial** — one `:test-cover` at a time on the host.
287-
- Every run starts from **verified pre-flight** (host clear + services ready + harness matches tier); if not, wait or follow [interrupted-run cleanup](#interrupted-run-abort-killed-terminal-eaddrinuse-on-8090) — do not start another run.
309+
- Every run starts from **verified [pre-flight](#pre-flight-is-the-host-clear-to-start)**; if probes fail, [pre-flight recovery](#pre-flight-recovery) before another run.
288310
- Use **only** canonical commands from this doc.
289311
- Never overlap focused-tier and area-tier `:test-cover` on one host.
290312

291313
See also: [focused-tier loop](#focused-tier-iteration-loop), [dispatch](#serialized-e2e-dispatch), [pre-merge](#before-merge-pr-handoff).
292314

293315
## Environment
294316

295-
- **Devices** — Detox boots simulator/emulator (`TestingAVD`); macOS auto-starts app.
317+
- **Devices** — Detox boots simulator/emulator (`iPhone 17` on iOS, `TestingAVD` on Android); [host-clear probes](#host-clear-probes) require zero booted iOS simulators before `:test-cover`. macOS auto-starts app.
296318
- **adb empty**`adb kill-server && adb start-server && adb devices`
297-
- **Stale processes** — one Metro (`:8081`), one emulator set (`:8080`, `:9099`, `:9000`, `:4400`, …). Jet `:8090` is per-`:test-cover`, not background; listener after run usually means stray Jet. Check: `lsof -nP -iTCP -sTCP:LISTEN | rg ':8081|:8080|:9099|:4400'`. Kill Metro/emulators: `pkill -f "react-native start"`, `pkill -f "firebase emulators"`.
319+
- **Stale processes** — one Metro (`:8081`), one emulator set (`:8080`, `:9099`, `:9000`, `:4400`, …). Stray Jet on `:8090` after a run [pre-flight recovery](#pre-flight-recovery). Restart Metro/emulators: `pkill -f "react-native start"`, `pkill -f "firebase emulators"`, then [Rules §1–2](#rules).
298320

299321
## Diagnosing hangs
300322

@@ -315,7 +337,7 @@ See also: [focused-tier loop](#focused-tier-iteration-loop), [dispatch](#seriali
315337
Pre-merge applies once to the branch commit stream before merge/push intended for merge, not after every commit.
316338

317339
1. Revert all narrowing ([full tier](#e2e-validation-tiers-focused-area-full)): restore `tests/app.js` (`platformSupportedModules` + `require.context`), default `RNFBDebug` in `tests/globals.js`, remove all `.only`, remove native instrumentation.
318-
2. [Pre-flight](#pre-flight-is-the-host-clear-to-start)verified clean host before each platform run.
340+
2. [Pre-flight](#pre-flight-is-the-host-clear-to-start)[host-clear probes](#host-clear-probes) pass before each platform run.
319341
3. Rebuild if needed (`tests:<platform>:build`; `yarn lerna:prepare` for `lib/**`).
320342
4. Full unfocused suite with coverage on **iOS, Android, macOS** — one platform at a time, all green.
321343

okf-bundle/testing/validation-checklist.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ Native: `yarn lint:android`, `yarn lint:ios:check`. `lint:android` can flake; re
7171

7272
## E2e with coverage
7373

74-
[Pre-flight](running-e2e.md#pre-flight-is-the-host-clear-to-start) (host + services + harness tier) before every run. Match harness to work type — **focused**/**area** never use full app load ([running e2e § harness](running-e2e.md#3-harness-matches-validation-tier)).
74+
[Pre-flight](running-e2e.md#pre-flight-is-the-host-clear-to-start) (host-clear probes + services + harness tier) before every run. Match harness to work type — **focused**/**area** never use full app load ([running e2e § harness](running-e2e.md#3-harness-matches-validation-tier)).
7575

7676
Commands: [Running e2e tests](running-e2e.md). Post-process: [Coverage design](coverage-design.md) (iOS `tests:ios:test:process-coverage`, Android `tests:android:post-e2e-coverage`).
7777

0 commit comments

Comments
 (0)