|
| 1 | +# Integrating Native iOS Code Coverage |
| 2 | + |
| 3 | +This guide integrates `@react-native-harness/coverage-ios` into the rive-nitro-react-native example app |
| 4 | +to collect native (Swift/ObjC) code coverage from `RNRive` and `RiveRuntime` pods. |
| 5 | + |
| 6 | +## Prerequisites |
| 7 | + |
| 8 | +- Xcode with `xcrun llvm-profdata` and `xcrun llvm-cov` available |
| 9 | +- iOS simulator booted (the harness test runner handles this) |
| 10 | +- The harness fork with coverage support: `mfazekas/react-native-harness` branch `feat/native-ios-coverage` |
| 11 | + |
| 12 | +## Issues Found During Integration (round 1) |
| 13 | + |
| 14 | +We attempted a full end-to-end integration. The native coverage **pipeline works** — profraw |
| 15 | +files are generated, merged, and lcov reports are produced — but several issues in the |
| 16 | +`@react-native-harness/coverage-ios` package prevented it from working out of the box. |
| 17 | + |
| 18 | +All issues below have been addressed in the fork (`mfazekas/react-native-harness` branch |
| 19 | +`feat/native-ios-coverage`, commit `c709d70`). Remove any manual Podfile workarounds before |
| 20 | +retesting. |
| 21 | + |
| 22 | +### Issue 1: Podspec fails to load — `Pod::Spec.new` must be the last expression -- FIXED |
| 23 | + |
| 24 | +`require_relative 'scripts/harness_coverage_hook'` was after the `Pod::Spec.new` block. |
| 25 | +CocoaPods expects the last expression to return the spec object. Also `Pod::Installer` is |
| 26 | +undefined during podspec parsing (only `cocoapods-core` is loaded). |
| 27 | + |
| 28 | +**Fix:** Moved `require_relative` before `Pod::Spec.new` with `if defined?(Pod::Installer)` guard. |
| 29 | + |
| 30 | +### Issue 2: `install_modules_dependencies` not always available -- FIXED |
| 31 | + |
| 32 | +Defined in React Native's `react_native_pods.rb`, not loaded during podspec parsing. |
| 33 | + |
| 34 | +**Fix:** Guarded with `if defined?(install_modules_dependencies)`, falls back to `s.dependency "React-Core"`. |
| 35 | + |
| 36 | +### Issue 3: `resolve-coverage-pods.mjs` returns empty — config schema mismatch -- FIXED |
| 37 | + |
| 38 | +The script used `@react-native-harness/config`'s `getConfig()` which validates via Zod. |
| 39 | +The published npm version doesn't have `native.ios.pods` in its schema, so Zod stripped it. |
| 40 | + |
| 41 | +**Fix:** Rewrote `resolve-coverage-pods.mjs` to read the config file directly (find + dynamic |
| 42 | +import). No longer depends on `@react-native-harness/config` at all — works with any published |
| 43 | +version. |
| 44 | + |
| 45 | +### Issue 4: Swift class name mangling — `NSClassFromString` fails -- FIXED |
| 46 | + |
| 47 | +The Swift class `HarnessCoverageHelper` got a mangled ObjC name like |
| 48 | +`_TtC15HarnessCoverage21HarnessCoverageHelper`. |
| 49 | + |
| 50 | +**Fix:** Added `@objc(HarnessCoverageHelper)` annotation to the class. |
| 51 | + |
| 52 | +### Issue 5: `+load` timing with debug dylibs — `NSClassFromString` returns nil -- FIXED |
| 53 | + |
| 54 | +With Xcode 16+ debug dylibs (mergeable libraries), the Swift class may not be registered |
| 55 | +at `+load` time. |
| 56 | + |
| 57 | +**Fix:** Deferred class lookup to `dispatch_async(dispatch_get_main_queue(), ...)`. |
| 58 | + |
| 59 | +### Issue 6: `-force_load` required for HarnessCoverage static library -- FIXED |
| 60 | + |
| 61 | +The `-ObjC` linker flag wasn't pulling HarnessCoverage object files into the final binary |
| 62 | +(Xcode 16+ debug dylib linking strategy). |
| 63 | + |
| 64 | +**Fix:** The coverage hook now patches `Pods-*.xcconfig` files to add |
| 65 | +`-force_load "${PODS_CONFIGURATION_BUILD_DIR}/HarnessCoverage/libHarnessCoverage.a"`. |
| 66 | + |
| 67 | +### Issue 7: `-fprofile-instr-generate` linker flag missing from app xcconfig -- FIXED |
| 68 | + |
| 69 | +The hook only added `-fprofile-instr-generate` to pod targets, but the app target needs it |
| 70 | +too for `__llvm_profile_write_file` and `__llvm_profile_set_filename` symbols. |
| 71 | + |
| 72 | +**Fix:** The coverage hook now also injects `-fprofile-instr-generate` into `Pods-*.xcconfig` files. |
| 73 | + |
| 74 | +### Issue 8: End-to-end integration requires ALL harness packages from the fork -- NOT YET FIXED |
| 75 | + |
| 76 | +The `collectNativeCoverage` integration spans multiple packages: |
| 77 | +- `@react-native-harness/config` — schema with `coverage.native.ios.pods` |
| 78 | +- `@react-native-harness/platforms` — `collectNativeCoverage` type on runner interface |
| 79 | +- `@react-native-harness/platform-apple` — implements `collectNativeCoverage` |
| 80 | +- `@react-native-harness/jest` — calls `collectNativeCoverage` in `disposeOnce()` |
| 81 | + |
| 82 | +Only installing `@react-native-harness/coverage-ios` from the fork is not enough. All four |
| 83 | +packages above need to come from the fork for the harness to actually call the collector |
| 84 | +after tests complete. This resolves when the PR merges and packages are published. |
| 85 | + |
| 86 | +**Workaround for testing:** The pod install side (issues 1-7) is now self-contained in |
| 87 | +`coverage-ios`. Profraw files will be generated and flushed by the app. The automatic |
| 88 | +collection after tests (merge + lcov export) won't happen until issue 8 is resolved, but |
| 89 | +you can collect manually — see Troubleshooting section. |
| 90 | + |
| 91 | +## What Works (verified manually, round 1) |
| 92 | + |
| 93 | +1. Coverage compiler flags (`-profile-generate`, `-fprofile-instr-generate`) are applied to |
| 94 | + RNRive and RiveRuntime pod targets |
| 95 | +2. `HARNESS_COVERAGE` compilation condition enables the coverage helper code |
| 96 | +3. `HarnessCoverageHelper.setup()` runs at app launch, sets profraw output path, starts |
| 97 | + flush timer |
| 98 | +4. `.profraw` files (7-13MB) are written to the app's Documents directory |
| 99 | +5. `xcrun llvm-profdata merge` merges profraw files successfully |
| 100 | +6. `xcrun llvm-cov export --format=lcov` produces valid lcov data (44K+ lines) |
| 101 | +7. Coverage report shows all 37 RNRive Swift/ObjC source files with line-level data |
| 102 | +8. The `coverage-collector.ts` in the fork handles Xcode 16+ `debug.dylib` binaries correctly |
| 103 | + |
| 104 | +## Round 2 Results (2026-05-08) |
| 105 | + |
| 106 | +Issues 1–7 fixes verified — all working with no manual Podfile workarounds: |
| 107 | + |
| 108 | +- `pod install` automatically instruments RNRive + RiveRuntime, patches xcconfigs |
| 109 | +- Clean build succeeds (Xcode 16.4, ~260s) |
| 110 | +- Binary contains `HarnessCoverageHelper`, `HarnessCoverageSetup` symbols (in `.debug.dylib`) |
| 111 | +- Manual app launch produces profraw files immediately (verified via simulator logs: |
| 112 | + `[HarnessCoverage] +load called, HARNESS_COVERAGE is defined` and |
| 113 | + `[HarnessCoverage] Found HarnessCoverageHelper, calling setup`) |
| 114 | +- 105 harness tests pass (11 suites), JS coverage at 80.23% |
| 115 | + |
| 116 | +**Remaining blocker — Issue 9: profraw files lost between app restarts -- FIXED** |
| 117 | + |
| 118 | +The harness reinstalls the app for each test file, wiping the app's Documents directory. |
| 119 | +After a full test run (11 suites), no profraw files remain. |
| 120 | + |
| 121 | +**Fix (commit `f15a893`):** Profraw files now write to `/tmp/harness-coverage/` instead of |
| 122 | +the app's Documents directory. On iOS simulators, `/tmp` maps to the simulator's filesystem |
| 123 | +(`~/Library/Developer/CoreSimulator/Devices/<UDID>/data/tmp/`) which persists across app |
| 124 | +reinstalls. The collector reads from this directory, and it's cleaned at test run start |
| 125 | +(to avoid stale data) and after successful collection. |
| 126 | + |
| 127 | +Changes: |
| 128 | +- `HarnessCoverageHelper.swift` — writes to `/tmp/harness-coverage/harness-<PID>.profraw` |
| 129 | +- `coverage-collector.ts` — reads from `/tmp/harness-coverage/`, cleans after collection |
| 130 | +- `instance.ts` — calls `cleanProfrawDir()` at platform init when coverage is configured |
| 131 | + |
| 132 | +## Round 3 Results (2026-05-08) — Native Coverage Working End-to-End |
| 133 | + |
| 134 | +After issue 9 fix (`/tmp/harness-coverage/`), the full pipeline works: |
| 135 | + |
| 136 | +1. `pod install` auto-instruments RNRive + RiveRuntime, patches xcconfigs — no Podfile workarounds |
| 137 | +2. Clean build succeeds (Xcode 16.4) |
| 138 | +3. 105 harness tests pass (11 suites), JS coverage at 80.23% |
| 139 | +4. **11 profraw files** generated in `/tmp/harness-coverage/` — one per test suite, all survive app restarts |
| 140 | +5. `llvm-profdata merge` + `llvm-cov report` produces real per-file native coverage |
| 141 | + |
| 142 | +**Native iOS coverage: 26.91% line coverage across 37 RNRive source files** |
| 143 | + |
| 144 | +Notable file coverage: |
| 145 | +- `HybridViewModel.swift` — 81% lines |
| 146 | +- `HybridViewModelNumberProperty.swift` — 80% |
| 147 | +- `RiveReactNativeView.swift` — 76% |
| 148 | +- `HybridViewModelTriggerProperty.swift` — 75% |
| 149 | +- `HybridViewModelStringProperty.swift` — 74% |
| 150 | +- `HybridViewModelInstance.swift` — 73% |
| 151 | +- `HybridRiveFile.swift` — 62% |
| 152 | +- `HybridRiveView.swift` — 52% |
| 153 | + |
| 154 | +**Important:** The coverage-instrumented app must be pre-installed on the correct simulator |
| 155 | +(matching the harness config's `device` — e.g. `iPhone 16 Pro` iOS 18.6). The harness doesn't |
| 156 | +build the app itself; it starts whatever is already installed. Steps: |
| 157 | +```bash |
| 158 | +# Build with coverage |
| 159 | +DEVELOPER_DIR=/Applications/Xcode16.4.app/Contents/Developer xcodebuild build \ |
| 160 | + -workspace ios/RiveExample.xcworkspace -scheme RiveExample \ |
| 161 | + -sdk iphonesimulator -destination "platform=iOS Simulator,name=iPhone 16 Pro,OS=18.6" \ |
| 162 | + -derivedDataPath ios/build |
| 163 | + |
| 164 | +# Install on the correct simulator |
| 165 | +xcrun simctl install "iPhone 16 Pro" ios/build/Build/Products/Debug-iphonesimulator/RiveExample.app |
| 166 | + |
| 167 | +# Run tests |
| 168 | +yarn test:harness:ios:coverage |
| 169 | +``` |
| 170 | + |
| 171 | +**Remaining gap (issue 8):** The automatic `collectNativeCoverage` call (merge + lcov export) |
| 172 | +after tests requires the fork's `platform-apple` and `jest` packages. Until those are published, |
| 173 | +collect manually: |
| 174 | +```bash |
| 175 | +xcrun llvm-profdata merge -sparse /tmp/harness-coverage/*.profraw -o coverage/native-ios.profdata |
| 176 | +xcrun llvm-cov export --format=lcov \ |
| 177 | + --instr-profile=coverage/native-ios.profdata \ |
| 178 | + ios/build/Build/Products/Debug-iphonesimulator/RiveExample.app/RiveExample.debug.dylib \ |
| 179 | + > coverage/native-ios.lcov |
| 180 | +``` |
| 181 | + |
| 182 | +## Steps |
| 183 | + |
| 184 | +### 1. Point harness dependencies to the coverage fork |
| 185 | + |
| 186 | +In the **root** `package.json`, add a `pnpm.overrides` (or `resolutions` for yarn) block to |
| 187 | +redirect all harness packages to the fork. Alternatively, if you use npm/yarn workspaces, |
| 188 | +use `file:` or `git:` references. |
| 189 | + |
| 190 | +The simplest approach for local testing: link the harness monorepo. |
| 191 | + |
| 192 | +From the harness repo (`/Users/boga/Work/Margelo/react-native-harness`): |
| 193 | + |
| 194 | +```bash |
| 195 | +# Build the coverage-ios package |
| 196 | +cd packages/coverage-ios |
| 197 | +pnpm build # or: pnpm exec tsc -p tsconfig.lib.json |
| 198 | +``` |
| 199 | + |
| 200 | +### 2. Add the coverage-ios dependency |
| 201 | + |
| 202 | +In `example/package.json`, add to `devDependencies`: |
| 203 | + |
| 204 | +```json |
| 205 | +"@react-native-harness/coverage-ios": "file:/Users/boga/Work/Margelo/react-native-harness/packages/coverage-ios" |
| 206 | +``` |
| 207 | + |
| 208 | +Then install: |
| 209 | + |
| 210 | +```bash |
| 211 | +cd example |
| 212 | +pnpm install # or npm/yarn install |
| 213 | +``` |
| 214 | + |
| 215 | +### 3. Update the harness config |
| 216 | + |
| 217 | +Edit `example/rn-harness.config.mjs` to add coverage configuration: |
| 218 | + |
| 219 | +```js |
| 220 | +import { androidPlatform, androidEmulator } from '@react-native-harness/platform-android'; |
| 221 | +import { applePlatform, appleSimulator } from '@react-native-harness/platform-apple'; |
| 222 | + |
| 223 | +const deviceModel = process.env.DEVICE_MODEL || 'iPhone 16 Pro'; |
| 224 | +const iosVersion = process.env.IOS_VERSION || '18.6'; |
| 225 | + |
| 226 | +export default { |
| 227 | + entryPoint: './index.js', |
| 228 | + appRegistryComponentName: 'RiveExample', |
| 229 | + bridgeTimeout: 90000, |
| 230 | + maxAppRestarts: 3, |
| 231 | + forwardClientLogs: true, |
| 232 | + runners: [ |
| 233 | + androidPlatform({ |
| 234 | + name: 'android', |
| 235 | + device: androidEmulator(process.env.ANDROID_AVD || 'Medium_Phone_API_35'), |
| 236 | + bundleId: 'rive.example', |
| 237 | + }), |
| 238 | + applePlatform({ |
| 239 | + name: 'ios', |
| 240 | + device: appleSimulator(deviceModel, iosVersion), |
| 241 | + bundleId: 'rive.example', |
| 242 | + }), |
| 243 | + ], |
| 244 | + defaultRunner: 'ios', |
| 245 | + |
| 246 | + // Native iOS code coverage |
| 247 | + coverage: { |
| 248 | + native: { |
| 249 | + ios: { |
| 250 | + pods: ['RNRive', 'RiveRuntime'], |
| 251 | + }, |
| 252 | + }, |
| 253 | + }, |
| 254 | +}; |
| 255 | +``` |
| 256 | + |
| 257 | +The `pods` array lists which CocoaPods targets get instrumented with |
| 258 | +`-profile-generate -profile-coverage-mapping` (Swift) and |
| 259 | +`-fprofile-instr-generate -fcoverage-mapping` (C/ObjC). |
| 260 | + |
| 261 | +Start with just `['RNRive']` if you only care about your own code. |
| 262 | +Add `'RiveRuntime'` to also cover the upstream Rive SDK. |
| 263 | + |
| 264 | +### 4. Run pod install |
| 265 | + |
| 266 | +The `@react-native-harness/coverage-ios` podspec hooks into CocoaPods via |
| 267 | +`Pod::Installer.prepend`. Running pod install will: |
| 268 | + |
| 269 | +- Read the `coverage.native.ios.pods` array from the harness config |
| 270 | +- Add coverage compiler flags to those pod targets |
| 271 | +- Enable the `HARNESS_COVERAGE` compilation condition on the `HarnessCoverage` pod |
| 272 | +- Add `-fprofile-instr-generate` linker flags |
| 273 | + |
| 274 | +```bash |
| 275 | +cd example/ios |
| 276 | +pod install |
| 277 | +``` |
| 278 | + |
| 279 | +You should see output like: |
| 280 | +``` |
| 281 | +[HarnessCoverage] Instrumenting pods for native coverage: RNRive, RiveRuntime |
| 282 | +[HarnessCoverage] -> RNRive |
| 283 | +[HarnessCoverage] -> RiveRuntime |
| 284 | +``` |
| 285 | + |
| 286 | +### 5. Build the app |
| 287 | + |
| 288 | +A **clean build** is required since the compiler flags changed: |
| 289 | + |
| 290 | +```bash |
| 291 | +cd example |
| 292 | +# Clean previous build artifacts |
| 293 | +xcodebuild clean -workspace ios/RiveExample.xcworkspace -scheme RiveExample |
| 294 | + |
| 295 | +# Build (or let the harness do it via HARNESS_APP_PATH) |
| 296 | +xcodebuild build-for-testing \ |
| 297 | + -workspace ios/RiveExample.xcworkspace \ |
| 298 | + -scheme RiveExample \ |
| 299 | + -sdk iphonesimulator \ |
| 300 | + -destination "platform=iOS Simulator,name=$DEVICE_MODEL,OS=$IOS_VERSION" \ |
| 301 | + -derivedDataPath ios/build |
| 302 | +``` |
| 303 | + |
| 304 | +Or if you normally let harness build via `HARNESS_APP_PATH`, just make sure |
| 305 | +the `.app` is rebuilt after the pod install. |
| 306 | + |
| 307 | +### 6. Run harness tests with coverage |
| 308 | + |
| 309 | +```bash |
| 310 | +cd example |
| 311 | +pnpm test:harness:ios:coverage |
| 312 | +``` |
| 313 | + |
| 314 | +When tests finish, the harness will: |
| 315 | +1. Send SIGTERM to the app (triggers `.profraw` flush) |
| 316 | +2. Wait briefly for filesystem sync |
| 317 | +3. Run `xcrun llvm-profdata merge` on the `.profraw` files |
| 318 | +4. Run `xcrun llvm-cov export --format=lcov` to produce an `.lcov` file |
| 319 | +5. Log: `Native coverage written to <path>` |
| 320 | + |
| 321 | +The `.lcov` file lands in the project root (the `example/` directory). |
| 322 | + |
| 323 | +### 7. View the coverage report |
| 324 | + |
| 325 | +```bash |
| 326 | +# Quick summary |
| 327 | +lcov --summary coverage/native-ios.lcov |
| 328 | + |
| 329 | +# Generate HTML report |
| 330 | +genhtml coverage/native-ios.lcov -o coverage/native-ios-html |
| 331 | +open coverage/native-ios-html/index.html |
| 332 | +``` |
| 333 | + |
| 334 | +## Troubleshooting |
| 335 | + |
| 336 | +**No `.profraw` files generated:** |
| 337 | +- Verify `pod install` printed the `[HarnessCoverage] Instrumenting pods` message |
| 338 | +- Ensure the app was rebuilt from scratch after `pod install` |
| 339 | +- Check that the simulator app actually ran (not just built) |
| 340 | +- Check simulator logs: `xcrun simctl spawn booted log show --predicate 'message CONTAINS "HarnessCoverage"' --last 1m` |
| 341 | + |
| 342 | +**`xcrun llvm-cov` fails:** |
| 343 | +- The `.profraw` file must match the exact binary that produced it |
| 344 | +- A stale build or incremental build can cause mismatches; do a clean build |
| 345 | + |
| 346 | +**Empty coverage for a pod:** |
| 347 | +- The pod's source code must actually execute during the test run |
| 348 | +- Check that the pod name in the config matches the CocoaPods target name exactly |
| 349 | + (case-sensitive, visible in `Podfile.lock`) |
| 350 | + |
| 351 | +**Xcode 16+ / debug dylibs:** |
| 352 | +- The app binary may be a thin stub; the real code is in `RiveExample.debug.dylib` |
| 353 | +- The `coverage-collector.ts` in the fork handles this correctly via `findAppExecutable()` |
| 354 | +- When checking symbols manually, use `nm` on the `.debug.dylib`, not the main binary |
0 commit comments