|
| 1 | +--- |
| 2 | +type: Reference |
| 3 | +title: Coverage design |
| 4 | +description: Goals and implementation details for unit and e2e test coverage across platforms. |
| 5 | +tags: [testing, coverage, codecov, e2e, jest] |
| 6 | +timestamp: 2026-06-17T00:00:00Z |
| 7 | +--- |
| 8 | + |
| 9 | +# Goals |
| 10 | + |
| 11 | +Coverage exists to show which **TypeScript library sources** (`packages/*/lib/**`) and **native module sources** (Java, Objective-C, Swift under `packages/*/android/**` and `packages/*/ios/**`) are exercised by tests. |
| 12 | + |
| 13 | +| Layer | What it proves | Primary consumers | |
| 14 | +|-------|----------------|-------------------| |
| 15 | +| **Unit (Jest)** | Package logic in isolation with mocks | Fast feedback on `packages/*/lib/**` | |
| 16 | +| **E2e (Jet / Detox)** | Real app behaviour against Firebase emulators | Integration coverage for TS + native bridges | |
| 17 | + |
| 18 | +Codecov merges uploads from CI into a single project view. Small project-level percentage swings can be noise (non-deterministic indirect lines); **file-level** coverage on `packages/*/lib/modular/**` and native startup files is the meaningful signal. |
| 19 | + |
| 20 | +macOS e2e uses the **firebase-js-sdk** only — native RNFB coverage is not applicable there. |
| 21 | + |
| 22 | +# End-to-end overview |
| 23 | + |
| 24 | +```mermaid |
| 25 | +flowchart LR |
| 26 | + subgraph unit [Unit Jest] |
| 27 | + J1[jest --coverage] --> J2[coverage/lcov.info] |
| 28 | + end |
| 29 | + subgraph ts_e2e [E2e TypeScript] |
| 30 | + M[Metro + inline source maps] --> A[App bundle] |
| 31 | + A --> J[Jet --coverage] |
| 32 | + J --> N[NYC remap] |
| 33 | + N --> T2[coverage/lcov.info] |
| 34 | + end |
| 35 | + subgraph android_native [E2e Android native] |
| 36 | + D1[Detox e2e] --> EC[coverage.ec in app] |
| 37 | + EC --> P1[pull-native-coverage.js] |
| 38 | + P1 --> J1R[jacocoAndroidTestReport] |
| 39 | + J1R --> AX[jacoco XML] |
| 40 | + end |
| 41 | + subgraph ios_native [E2e iOS native] |
| 42 | + D2[Detox e2e] --> FL[RNFBTestingCoverage.flush] |
| 43 | + FL --> PR[coverage.profraw in Documents] |
| 44 | + PR --> P2[pull-native-coverage.js] |
| 45 | + P2 --> LLVM[process-ios-native-coverage.js] |
| 46 | + LLVM --> I2[coverage/ios-native.lcov.info] |
| 47 | + end |
| 48 | + J2 --> C[Codecov] |
| 49 | + T2 --> C |
| 50 | + AX --> C |
| 51 | + I2 --> C |
| 52 | +``` |
| 53 | + |
| 54 | +# Unit coverage (Jest) |
| 55 | + |
| 56 | +## Command |
| 57 | + |
| 58 | +```bash |
| 59 | +yarn tests:jest-coverage |
| 60 | +``` |
| 61 | + |
| 62 | +## Tooling |
| 63 | + |
| 64 | +- **Provider:** Jest with `coverageProvider: "babel"` (Istanbul via `babel-jest`), **not** NYC. |
| 65 | +- **Scope:** `packages/**/__tests__/**` only (see root `jest.config.js`). |
| 66 | +- **Output:** `coverage/lcov.info` at repo root (among other Istanbul artifacts). |
| 67 | + |
| 68 | +## Behaviour |
| 69 | + |
| 70 | +Jest instruments TypeScript/JavaScript directly under `packages/*/lib/**`. No source-map remapping is required because tests import library sources, not Metro bundles. |
| 71 | + |
| 72 | +# E2e TypeScript coverage (Jet + NYC) |
| 73 | + |
| 74 | +## Commands |
| 75 | + |
| 76 | +| Platform | Yarn script | Notes | |
| 77 | +|----------|-------------|-------| |
| 78 | +| macOS | `yarn tests:macos:test-cover` | Jet only | |
| 79 | +| iOS CI | `yarn tests:ios:test-cover` | Detox → Jet `--coverage` | |
| 80 | +| iOS local (reuse build) | `yarn tests:ios:test-cover-reuse` | Same; Jet self-wraps under NYC when `--coverage` is passed | |
| 81 | +| Android | `yarn tests:android:test-cover` | Detox → Jet `--coverage` | |
| 82 | + |
| 83 | +Android/iOS run Detox, which spawns Jet with `--coverage`. macOS runs Jet directly. |
| 84 | + |
| 85 | +## Tooling |
| 86 | + |
| 87 | +- **Metro** bundles `packages/*/dist/module/**` with inline source maps (`tests/.babelrc`: `useInlineSourceMaps: true`). |
| 88 | +- **NYC** (`tests/nyc.config.js`) collects coverage from instrumented bundles, remaps to `packages/*/lib/**` via source maps, and writes **`coverage/lcov.info`** (NYC `cwd: '..'`). |
| 89 | +- **Jet self-wrap:** When `--coverage` is passed, `tests/node_modules/jet/jet.js` re-invokes itself under `tests/node_modules/.bin/nyc` (checks `NYC_CONFIG` to avoid double-wrap). Detox/macOS do **not** need an extra `nyc` prefix on the yarn script — only Jet needs to run from the `tests/` directory so it can find NYC. |
| 90 | +- **Transfer:** Patched Jet / mocha-remote send coverage over the existing WebSocket (`coverage-data` event), replacing HTTP POST that failed on large (~4.5MB+) payloads. Patches live in `.yarn/patches/` (`jet`, `mocha-remote-client`, `mocha-remote-server`). |
| 91 | + |
| 92 | +## Key NYC settings |
| 93 | + |
| 94 | +```javascript |
| 95 | +include: ['packages/*/lib/**/*.{js,ts,tsx}', 'packages/*/dist/**/*.js'], |
| 96 | +sourceMap: true, |
| 97 | +'exclude-after-remap': true, |
| 98 | +instrument: false, |
| 99 | +reporter: ['lcov', 'html', 'text-summary'], |
| 100 | +``` |
| 101 | + |
| 102 | +## Verification |
| 103 | + |
| 104 | +After a Detox/macOS e2e run, expect log lines like `[jet-coverage] WS received N file(s)` and an NYC summary. `coverage/lcov.info` should contain `SF:packages/...` paths (not only `packages/*/dist/...`). |
| 105 | + |
| 106 | +# E2e Android native coverage (Jacoco) |
| 107 | + |
| 108 | +## Pipeline |
| 109 | + |
| 110 | +1. Gradle enables `testCoverageEnabled` on RNFB Android library modules (`tests/android/build.gradle`). |
| 111 | +2. Detox e2e runs; the instrumented app writes `coverage.ec`. |
| 112 | +3. When the Jet process exits successfully, `tests/scripts/pull-native-coverage.js` (called from `tests/e2e/firebase.test.js`) copies the `.ec` file to `tests/android/app/build/output/coverage/emulator_coverage.ec`. **The test fails if this pull fails** (same pattern as iOS). |
| 113 | +4. `yarn tests:android:test:jacoco-report` runs `jacocoAndroidTestReport`, producing XML at: |
| 114 | + |
| 115 | + `tests/android/app/build/reports/jacoco/jacocoAndroidTestReport/jacocoAndroidTestReport.xml` |
| 116 | + |
| 117 | +CI runs steps 3–4 in sequence inside the emulator job. |
| 118 | + |
| 119 | +## Jacoco configuration notes |
| 120 | + |
| 121 | +- Class directories must use the AGP 8.x path: `build/intermediates/javac/debug/compileDebugJavaWithJavac/classes` (not legacy `.../debug/classes`). |
| 122 | +- Source directories must include `src/reactnative/java` where modules place React Native entry code (e.g. `ReactNativeFirebaseAppInitProvider`). |
| 123 | +- Module list comes from `rootProject.ext.firebaseModulePaths` populated in `tests/android/build.gradle`. |
| 124 | +- `tests/android/app/jacoco.gradle` defines three report tasks sharing the same AGP 8 class paths and `firebaseModulePaths` source dirs: |
| 125 | + - **`jacocoAndroidTestReport`** — e2e only (`**/*.ec` from Detox pull) |
| 126 | + - **`jacocoUnitTestReport`** — unit tests only (`**/*.exec`; for when module/app unit tests are added) |
| 127 | + - **`jacocoTestReport`** — merged unit + e2e (`**/*.exec` and `**/*.ec`) |
| 128 | + CI uses `jacocoAndroidTestReport` after Detox. |
| 129 | + |
| 130 | +# E2e iOS native coverage (LLVM) |
| 131 | + |
| 132 | +## Pipeline |
| 133 | + |
| 134 | +1. **Build-time instrumentation:** `CLANG_ENABLE_CODE_COVERAGE=YES` plus explicit LLVM flags on the test app and Pod targets (`-fprofile-instr-generate`, `-fcoverage-mapping`, Swift `-profile-generate`) are set in **`tests/ios/Podfile` `post_install`** only (not in `project.pbxproj` — that file should stay opaque; run `pod install` after checkout). Detox builds with `-derivedDataPath ios/build`. **Do not** pass `-enableCodeCoverage` to `xcodebuild build` — that flag is test-only. |
| 135 | +2. **Runtime flush:** At app launch, `RNFBTestingConfigureCoverageProfilePath()` sets the profile output to `Documents/coverage.profraw`. After all Mocha tests complete, the Jet `after` hook in `tests/app.js` calls `NativeModules.RNFBTestingCoverage.flush()` (native module exported via `RCT_EXPORT_MODULE(RNFBTestingCoverage)`). **Do not use a custom URL scheme** — iOS shows an “Open in 'testing'?” dialog that blocks Detox. |
| 136 | +3. **Pull:** When the Jet process exits with code 0, `tests/scripts/pull-native-coverage.js` copies profraw from the simulator app container to `tests/ios/build/output/coverage/simulator_coverage.profraw`. **The Detox test fails if no profraw is found.** Pull happens on Jet `close` (not in `afterAll`) so it runs before Detox environment teardown. |
| 137 | +4. **Post-test export:** `yarn tests:ios:test:process-coverage` runs `tests/scripts/process-ios-native-coverage.js`, which: |
| 138 | + - exits **1** if no `.profraw` files are present (missing profraw after a successful e2e means flush or pull failed) |
| 139 | + - merges `.profraw` from `tests/ios/build/output/coverage/` (Detox) and optionally `Build/ProfileData/` (`xcodebuild test` only — not used by Detox today) |
| 140 | + - exports lcov with `xcrun llvm-cov export -format=lcov` against the main app binary (statically linked RNFB code), writing to a temp file (large reports exceed Node stdout buffer limits) |
| 141 | + - streams and rewrites `SF:` paths to repo-relative `packages/**` paths |
| 142 | + - writes **`coverage/ios-native.lcov.info`** |
| 143 | + - **deletes the processed `.profraw` files** so a missing file on the next run is a clear signal that e2e did not produce fresh native coverage |
| 144 | + |
| 145 | +## Objective-C and Swift |
| 146 | + |
| 147 | +Both languages are covered by the same LLVM pipeline. Swift files (e.g. under `packages/firestore/ios/**`) appear in the exported lcov alongside `.m` / `.mm` files. |
| 148 | + |
| 149 | +Most entries in the raw llvm-cov export are Pods/SDK/system code; only paths under `packages/` matter for Codecov. A healthy full e2e run typically reports on the order of **~50–60 `packages/*/ios/**` files** among ~2000 total source entries. |
| 150 | + |
| 151 | +## CocoaPods → SPM |
| 152 | + |
| 153 | +The Podfile `post_install` coverage flags are temporary. When native dependencies move to SPM, enable the same build settings on SPM targets; the post-test script remains unchanged. |
| 154 | + |
| 155 | +# Codecov uploads (CI) |
| 156 | + |
| 157 | +CI uses [codecov-action](https://github.com/codecov/codecov-action) v6 with `verbose: true`. It discovers coverage files under the repo, including: |
| 158 | + |
| 159 | +| Workflow | Artifacts | |
| 160 | +|----------|-----------| |
| 161 | +| `tests_jest.yml` | `coverage/lcov.info`, `coverage-final.json`, `clover.xml` | |
| 162 | +| `tests_e2e_android.yml` | `coverage/lcov.info` + Jacoco XML | |
| 163 | +| `tests_e2e_other.yml` | `coverage/lcov.info` (macOS TS) | |
| 164 | +| `tests_e2e_ios.yml` (debug) | `coverage/lcov.info` + `coverage/ios-native.lcov.info` | |
| 165 | + |
| 166 | +The iOS workflow runs `yarn tests:ios:test:process-coverage` after Detox (`if: always()`, `continue-on-error: true` for now). The process step exits 1 when profraw is missing. |
| 167 | + |
| 168 | +## File naming |
| 169 | + |
| 170 | +The repo standard for JavaScript lcov is `coverage/lcov.info`. iOS native uses **`coverage/ios-native.lcov.info`**. Codecov detects **lcov format by file content**, not only by the exact name `lcov.info`. |
| 171 | + |
| 172 | +Check the Codecov commit **Uploads** tab for **Processed** vs **Unusable** per upload — that is the authoritative signal, not small project percentage deltas. |
| 173 | + |
| 174 | +# Local iteration |
| 175 | + |
| 176 | +```bash |
| 177 | +# Full iOS (build once, then reuse) |
| 178 | +yarn tests:ios:build |
| 179 | +yarn tests:ios:test-cover-reuse |
| 180 | +yarn tests:ios:test:process-coverage |
| 181 | + |
| 182 | +# Or combined |
| 183 | +yarn tests:ios:test-cover-and-process # clean Detox run + process (no --reuse) |
| 184 | + |
| 185 | +# Android (after e2e) |
| 186 | +yarn tests:android:test-cover-reuse |
| 187 | +yarn tests:android:test:jacoco-report |
| 188 | + |
| 189 | +# Codecov CLI (optional) |
| 190 | +.codecov-venv/bin/codecovcli upload-process \ |
| 191 | + -t "$CODECOV_TOKEN" -r invertase/react-native-firebase \ |
| 192 | + --git-service github -C "$(git rev-parse HEAD)" -B "$(git branch --show-current)" \ |
| 193 | + -f coverage/ios-native.lcov.info -n local-ios-native --disable-search |
| 194 | +``` |
| 195 | + |
| 196 | +Metro must be running (`yarn tests:packager:jet`) for Detox e2e. |
| 197 | + |
| 198 | +# Critical invariants |
| 199 | + |
| 200 | +These must all be true for native iOS coverage to work. If any break, the e2e test should fail (not silently upload stale data). |
| 201 | + |
| 202 | +| Invariant | Where enforced | |
| 203 | +|-----------|----------------| |
| 204 | +| App built with LLVM profile flags | `Podfile` `post_install` (run `pod install`) | |
| 205 | +| Profile path set at launch | `AppDelegate` → `RNFBTestingConfigureCoverageProfilePath()` | |
| 206 | +| JS module name matches native export | `RCT_EXPORT_MODULE(RNFBTestingCoverage)` + `NativeModules.RNFBTestingCoverage` in `tests/app.js` | |
| 207 | +| Flush runs after Mocha tests | Jet `after` hook in `tests/app.js` | |
| 208 | +| Profraw pulled before Detox teardown | `pull-native-coverage.js` on Jet `close` in `firebase.test.js` | |
| 209 | +| Fresh profraw processed after e2e | `process-ios-native-coverage.js` (deletes profraw after export) | |
| 210 | + |
| 211 | +# Troubleshooting |
| 212 | + |
| 213 | +| Symptom | Likely cause | Fix | |
| 214 | +|---------|--------------|-----| |
| 215 | +| Simulator “Open in 'testing'?” dialog | Custom URL scheme handler | Use native module flush only; no `rnfb-testing://` | |
| 216 | +| No profraw after e2e; test still passes (old behaviour) | Pull in `afterAll` after `detox.cleanup()`, or wrong module name | Pull on Jet `close`; verify `RNFBTestingCoverage` export name | |
| 217 | +| Stale profraw uploaded | Re-processed old file without re-running e2e | Process step deletes profraw after export; missing file + exit 1 on next process | |
| 218 | +| `process-ios-native-coverage` succeeds but no `packages/` hits | Wrong binary / not instrumented | Rebuild with `yarn tests:ios:build`; check Podfile flags | |
| 219 | +| Empty Jacoco XML (~235 bytes) | AGP 8 class path or missing `src/reactnative/java` | See `jacocoAndroidTestReport` in `tests/android/app/jacoco.gradle` | |
| 220 | +| No `[jet-coverage] WS received` lines | Patches not applied | `yarn install` from repo root; check `.yarn/patches/` | |
| 221 | +| NYC summary missing / empty `lcov.info` | Jet not run from `tests/` cwd | Detox spawns `yarn jet` inside `tests/`; macOS uses `cd tests && npx jet` | |
| 222 | +| Codecov upload **Unusable** | Wrong `SF:` paths | Confirm path rewrite in `process-ios-native-coverage.js`; check Uploads tab message | |
| 223 | + |
| 224 | +# Future cleanups (non-blocking) |
| 225 | + |
| 226 | +- **CI:** drop `continue-on-error: true` on the iOS process-coverage step once stable in CI. |
| 227 | + |
| 228 | +# Citations |
| 229 | + |
| 230 | +[1] [Open Knowledge Format (OKF) specification](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md) |
| 231 | +[2] [Codecov CLI documentation](https://docs.codecov.com/docs/the-codecov-cli) |
0 commit comments