Skip to content

Commit 9299651

Browse files
committed
feat: add native iOS code coverage integration
Add coverage-ios package, configure RNRive + RiveRuntime pod instrumentation, and document the integration process.
1 parent 536ecdd commit 9299651

5 files changed

Lines changed: 404 additions & 1 deletion

File tree

NATIVE_IOS_COVERAGE_INTEGRATION.md

Lines changed: 354 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,354 @@
1+
# Integrating Native iOS Code Coverage
2+
3+
This guide integrates `@react-native-harness/coverage-ios` into the rive-nitro-react-native example app
4+
to collect native (Swift/ObjC) code coverage from `RNRive` and `RiveRuntime` pods.
5+
6+
## Prerequisites
7+
8+
- Xcode with `xcrun llvm-profdata` and `xcrun llvm-cov` available
9+
- iOS simulator booted (the harness test runner handles this)
10+
- The harness fork with coverage support: `mfazekas/react-native-harness` branch `feat/native-ios-coverage`
11+
12+
## Issues Found During Integration (round 1)
13+
14+
We attempted a full end-to-end integration. The native coverage **pipeline works** — profraw
15+
files are generated, merged, and lcov reports are produced — but several issues in the
16+
`@react-native-harness/coverage-ios` package prevented it from working out of the box.
17+
18+
All issues below have been addressed in the fork (`mfazekas/react-native-harness` branch
19+
`feat/native-ios-coverage`, commit `c709d70`). Remove any manual Podfile workarounds before
20+
retesting.
21+
22+
### Issue 1: Podspec fails to load — `Pod::Spec.new` must be the last expression -- FIXED
23+
24+
`require_relative 'scripts/harness_coverage_hook'` was after the `Pod::Spec.new` block.
25+
CocoaPods expects the last expression to return the spec object. Also `Pod::Installer` is
26+
undefined during podspec parsing (only `cocoapods-core` is loaded).
27+
28+
**Fix:** Moved `require_relative` before `Pod::Spec.new` with `if defined?(Pod::Installer)` guard.
29+
30+
### Issue 2: `install_modules_dependencies` not always available -- FIXED
31+
32+
Defined in React Native's `react_native_pods.rb`, not loaded during podspec parsing.
33+
34+
**Fix:** Guarded with `if defined?(install_modules_dependencies)`, falls back to `s.dependency "React-Core"`.
35+
36+
### Issue 3: `resolve-coverage-pods.mjs` returns empty — config schema mismatch -- FIXED
37+
38+
The script used `@react-native-harness/config`'s `getConfig()` which validates via Zod.
39+
The published npm version doesn't have `native.ios.pods` in its schema, so Zod stripped it.
40+
41+
**Fix:** Rewrote `resolve-coverage-pods.mjs` to read the config file directly (find + dynamic
42+
import). No longer depends on `@react-native-harness/config` at all — works with any published
43+
version.
44+
45+
### Issue 4: Swift class name mangling — `NSClassFromString` fails -- FIXED
46+
47+
The Swift class `HarnessCoverageHelper` got a mangled ObjC name like
48+
`_TtC15HarnessCoverage21HarnessCoverageHelper`.
49+
50+
**Fix:** Added `@objc(HarnessCoverageHelper)` annotation to the class.
51+
52+
### Issue 5: `+load` timing with debug dylibs — `NSClassFromString` returns nil -- FIXED
53+
54+
With Xcode 16+ debug dylibs (mergeable libraries), the Swift class may not be registered
55+
at `+load` time.
56+
57+
**Fix:** Deferred class lookup to `dispatch_async(dispatch_get_main_queue(), ...)`.
58+
59+
### Issue 6: `-force_load` required for HarnessCoverage static library -- FIXED
60+
61+
The `-ObjC` linker flag wasn't pulling HarnessCoverage object files into the final binary
62+
(Xcode 16+ debug dylib linking strategy).
63+
64+
**Fix:** The coverage hook now patches `Pods-*.xcconfig` files to add
65+
`-force_load "${PODS_CONFIGURATION_BUILD_DIR}/HarnessCoverage/libHarnessCoverage.a"`.
66+
67+
### Issue 7: `-fprofile-instr-generate` linker flag missing from app xcconfig -- FIXED
68+
69+
The hook only added `-fprofile-instr-generate` to pod targets, but the app target needs it
70+
too for `__llvm_profile_write_file` and `__llvm_profile_set_filename` symbols.
71+
72+
**Fix:** The coverage hook now also injects `-fprofile-instr-generate` into `Pods-*.xcconfig` files.
73+
74+
### Issue 8: End-to-end integration requires ALL harness packages from the fork -- NOT YET FIXED
75+
76+
The `collectNativeCoverage` integration spans multiple packages:
77+
- `@react-native-harness/config` — schema with `coverage.native.ios.pods`
78+
- `@react-native-harness/platforms``collectNativeCoverage` type on runner interface
79+
- `@react-native-harness/platform-apple` — implements `collectNativeCoverage`
80+
- `@react-native-harness/jest` — calls `collectNativeCoverage` in `disposeOnce()`
81+
82+
Only installing `@react-native-harness/coverage-ios` from the fork is not enough. All four
83+
packages above need to come from the fork for the harness to actually call the collector
84+
after tests complete. This resolves when the PR merges and packages are published.
85+
86+
**Workaround for testing:** The pod install side (issues 1-7) is now self-contained in
87+
`coverage-ios`. Profraw files will be generated and flushed by the app. The automatic
88+
collection after tests (merge + lcov export) won't happen until issue 8 is resolved, but
89+
you can collect manually — see Troubleshooting section.
90+
91+
## What Works (verified manually, round 1)
92+
93+
1. Coverage compiler flags (`-profile-generate`, `-fprofile-instr-generate`) are applied to
94+
RNRive and RiveRuntime pod targets
95+
2. `HARNESS_COVERAGE` compilation condition enables the coverage helper code
96+
3. `HarnessCoverageHelper.setup()` runs at app launch, sets profraw output path, starts
97+
flush timer
98+
4. `.profraw` files (7-13MB) are written to the app's Documents directory
99+
5. `xcrun llvm-profdata merge` merges profraw files successfully
100+
6. `xcrun llvm-cov export --format=lcov` produces valid lcov data (44K+ lines)
101+
7. Coverage report shows all 37 RNRive Swift/ObjC source files with line-level data
102+
8. The `coverage-collector.ts` in the fork handles Xcode 16+ `debug.dylib` binaries correctly
103+
104+
## Round 2 Results (2026-05-08)
105+
106+
Issues 1–7 fixes verified — all working with no manual Podfile workarounds:
107+
108+
- `pod install` automatically instruments RNRive + RiveRuntime, patches xcconfigs
109+
- Clean build succeeds (Xcode 16.4, ~260s)
110+
- Binary contains `HarnessCoverageHelper`, `HarnessCoverageSetup` symbols (in `.debug.dylib`)
111+
- Manual app launch produces profraw files immediately (verified via simulator logs:
112+
`[HarnessCoverage] +load called, HARNESS_COVERAGE is defined` and
113+
`[HarnessCoverage] Found HarnessCoverageHelper, calling setup`)
114+
- 105 harness tests pass (11 suites), JS coverage at 80.23%
115+
116+
**Remaining blocker — Issue 9: profraw files lost between app restarts -- FIXED**
117+
118+
The harness reinstalls the app for each test file, wiping the app's Documents directory.
119+
After a full test run (11 suites), no profraw files remain.
120+
121+
**Fix (commit `f15a893`):** Profraw files now write to `/tmp/harness-coverage/` instead of
122+
the app's Documents directory. On iOS simulators, `/tmp` maps to the simulator's filesystem
123+
(`~/Library/Developer/CoreSimulator/Devices/<UDID>/data/tmp/`) which persists across app
124+
reinstalls. The collector reads from this directory, and it's cleaned at test run start
125+
(to avoid stale data) and after successful collection.
126+
127+
Changes:
128+
- `HarnessCoverageHelper.swift` — writes to `/tmp/harness-coverage/harness-<PID>.profraw`
129+
- `coverage-collector.ts` — reads from `/tmp/harness-coverage/`, cleans after collection
130+
- `instance.ts` — calls `cleanProfrawDir()` at platform init when coverage is configured
131+
132+
## Round 3 Results (2026-05-08) — Native Coverage Working End-to-End
133+
134+
After issue 9 fix (`/tmp/harness-coverage/`), the full pipeline works:
135+
136+
1. `pod install` auto-instruments RNRive + RiveRuntime, patches xcconfigs — no Podfile workarounds
137+
2. Clean build succeeds (Xcode 16.4)
138+
3. 105 harness tests pass (11 suites), JS coverage at 80.23%
139+
4. **11 profraw files** generated in `/tmp/harness-coverage/` — one per test suite, all survive app restarts
140+
5. `llvm-profdata merge` + `llvm-cov report` produces real per-file native coverage
141+
142+
**Native iOS coverage: 26.91% line coverage across 37 RNRive source files**
143+
144+
Notable file coverage:
145+
- `HybridViewModel.swift` — 81% lines
146+
- `HybridViewModelNumberProperty.swift` — 80%
147+
- `RiveReactNativeView.swift` — 76%
148+
- `HybridViewModelTriggerProperty.swift` — 75%
149+
- `HybridViewModelStringProperty.swift` — 74%
150+
- `HybridViewModelInstance.swift` — 73%
151+
- `HybridRiveFile.swift` — 62%
152+
- `HybridRiveView.swift` — 52%
153+
154+
**Important:** The coverage-instrumented app must be pre-installed on the correct simulator
155+
(matching the harness config's `device` — e.g. `iPhone 16 Pro` iOS 18.6). The harness doesn't
156+
build the app itself; it starts whatever is already installed. Steps:
157+
```bash
158+
# Build with coverage
159+
DEVELOPER_DIR=/Applications/Xcode16.4.app/Contents/Developer xcodebuild build \
160+
-workspace ios/RiveExample.xcworkspace -scheme RiveExample \
161+
-sdk iphonesimulator -destination "platform=iOS Simulator,name=iPhone 16 Pro,OS=18.6" \
162+
-derivedDataPath ios/build
163+
164+
# Install on the correct simulator
165+
xcrun simctl install "iPhone 16 Pro" ios/build/Build/Products/Debug-iphonesimulator/RiveExample.app
166+
167+
# Run tests
168+
yarn test:harness:ios:coverage
169+
```
170+
171+
**Remaining gap (issue 8):** The automatic `collectNativeCoverage` call (merge + lcov export)
172+
after tests requires the fork's `platform-apple` and `jest` packages. Until those are published,
173+
collect manually:
174+
```bash
175+
xcrun llvm-profdata merge -sparse /tmp/harness-coverage/*.profraw -o coverage/native-ios.profdata
176+
xcrun llvm-cov export --format=lcov \
177+
--instr-profile=coverage/native-ios.profdata \
178+
ios/build/Build/Products/Debug-iphonesimulator/RiveExample.app/RiveExample.debug.dylib \
179+
> coverage/native-ios.lcov
180+
```
181+
182+
## Steps
183+
184+
### 1. Point harness dependencies to the coverage fork
185+
186+
In the **root** `package.json`, add a `pnpm.overrides` (or `resolutions` for yarn) block to
187+
redirect all harness packages to the fork. Alternatively, if you use npm/yarn workspaces,
188+
use `file:` or `git:` references.
189+
190+
The simplest approach for local testing: link the harness monorepo.
191+
192+
From the harness repo (`/Users/boga/Work/Margelo/react-native-harness`):
193+
194+
```bash
195+
# Build the coverage-ios package
196+
cd packages/coverage-ios
197+
pnpm build # or: pnpm exec tsc -p tsconfig.lib.json
198+
```
199+
200+
### 2. Add the coverage-ios dependency
201+
202+
In `example/package.json`, add to `devDependencies`:
203+
204+
```json
205+
"@react-native-harness/coverage-ios": "file:/Users/boga/Work/Margelo/react-native-harness/packages/coverage-ios"
206+
```
207+
208+
Then install:
209+
210+
```bash
211+
cd example
212+
pnpm install # or npm/yarn install
213+
```
214+
215+
### 3. Update the harness config
216+
217+
Edit `example/rn-harness.config.mjs` to add coverage configuration:
218+
219+
```js
220+
import { androidPlatform, androidEmulator } from '@react-native-harness/platform-android';
221+
import { applePlatform, appleSimulator } from '@react-native-harness/platform-apple';
222+
223+
const deviceModel = process.env.DEVICE_MODEL || 'iPhone 16 Pro';
224+
const iosVersion = process.env.IOS_VERSION || '18.6';
225+
226+
export default {
227+
entryPoint: './index.js',
228+
appRegistryComponentName: 'RiveExample',
229+
bridgeTimeout: 90000,
230+
maxAppRestarts: 3,
231+
forwardClientLogs: true,
232+
runners: [
233+
androidPlatform({
234+
name: 'android',
235+
device: androidEmulator(process.env.ANDROID_AVD || 'Medium_Phone_API_35'),
236+
bundleId: 'rive.example',
237+
}),
238+
applePlatform({
239+
name: 'ios',
240+
device: appleSimulator(deviceModel, iosVersion),
241+
bundleId: 'rive.example',
242+
}),
243+
],
244+
defaultRunner: 'ios',
245+
246+
// Native iOS code coverage
247+
coverage: {
248+
native: {
249+
ios: {
250+
pods: ['RNRive', 'RiveRuntime'],
251+
},
252+
},
253+
},
254+
};
255+
```
256+
257+
The `pods` array lists which CocoaPods targets get instrumented with
258+
`-profile-generate -profile-coverage-mapping` (Swift) and
259+
`-fprofile-instr-generate -fcoverage-mapping` (C/ObjC).
260+
261+
Start with just `['RNRive']` if you only care about your own code.
262+
Add `'RiveRuntime'` to also cover the upstream Rive SDK.
263+
264+
### 4. Run pod install
265+
266+
The `@react-native-harness/coverage-ios` podspec hooks into CocoaPods via
267+
`Pod::Installer.prepend`. Running pod install will:
268+
269+
- Read the `coverage.native.ios.pods` array from the harness config
270+
- Add coverage compiler flags to those pod targets
271+
- Enable the `HARNESS_COVERAGE` compilation condition on the `HarnessCoverage` pod
272+
- Add `-fprofile-instr-generate` linker flags
273+
274+
```bash
275+
cd example/ios
276+
pod install
277+
```
278+
279+
You should see output like:
280+
```
281+
[HarnessCoverage] Instrumenting pods for native coverage: RNRive, RiveRuntime
282+
[HarnessCoverage] -> RNRive
283+
[HarnessCoverage] -> RiveRuntime
284+
```
285+
286+
### 5. Build the app
287+
288+
A **clean build** is required since the compiler flags changed:
289+
290+
```bash
291+
cd example
292+
# Clean previous build artifacts
293+
xcodebuild clean -workspace ios/RiveExample.xcworkspace -scheme RiveExample
294+
295+
# Build (or let the harness do it via HARNESS_APP_PATH)
296+
xcodebuild build-for-testing \
297+
-workspace ios/RiveExample.xcworkspace \
298+
-scheme RiveExample \
299+
-sdk iphonesimulator \
300+
-destination "platform=iOS Simulator,name=$DEVICE_MODEL,OS=$IOS_VERSION" \
301+
-derivedDataPath ios/build
302+
```
303+
304+
Or if you normally let harness build via `HARNESS_APP_PATH`, just make sure
305+
the `.app` is rebuilt after the pod install.
306+
307+
### 6. Run harness tests with coverage
308+
309+
```bash
310+
cd example
311+
pnpm test:harness:ios:coverage
312+
```
313+
314+
When tests finish, the harness will:
315+
1. Send SIGTERM to the app (triggers `.profraw` flush)
316+
2. Wait briefly for filesystem sync
317+
3. Run `xcrun llvm-profdata merge` on the `.profraw` files
318+
4. Run `xcrun llvm-cov export --format=lcov` to produce an `.lcov` file
319+
5. Log: `Native coverage written to <path>`
320+
321+
The `.lcov` file lands in the project root (the `example/` directory).
322+
323+
### 7. View the coverage report
324+
325+
```bash
326+
# Quick summary
327+
lcov --summary coverage/native-ios.lcov
328+
329+
# Generate HTML report
330+
genhtml coverage/native-ios.lcov -o coverage/native-ios-html
331+
open coverage/native-ios-html/index.html
332+
```
333+
334+
## Troubleshooting
335+
336+
**No `.profraw` files generated:**
337+
- Verify `pod install` printed the `[HarnessCoverage] Instrumenting pods` message
338+
- Ensure the app was rebuilt from scratch after `pod install`
339+
- Check that the simulator app actually ran (not just built)
340+
- Check simulator logs: `xcrun simctl spawn booted log show --predicate 'message CONTAINS "HarnessCoverage"' --last 1m`
341+
342+
**`xcrun llvm-cov` fails:**
343+
- The `.profraw` file must match the exact binary that produced it
344+
- A stale build or incremental build can cause mismatches; do a clean build
345+
346+
**Empty coverage for a pod:**
347+
- The pod's source code must actually execute during the test run
348+
- Check that the pod name in the config matches the CocoaPods target name exactly
349+
(case-sensitive, visible in `Podfile.lock`)
350+
351+
**Xcode 16+ / debug dylibs:**
352+
- The app binary may be a thin stub; the real code is in `RiveExample.debug.dylib`
353+
- The `coverage-collector.ts` in the fork handles this correctly via `findAppExecutable()`
354+
- When checking symbols manually, use `nm` on the `.debug.dylib`, not the main binary

0 commit comments

Comments
 (0)