Skip to content

Vendor the SDK's SwiftPM-only deps as a pre-built static binary#69

Draft
peachbits wants to merge 2 commits into
masterfrom
zcash-spm-deps-binary
Draft

Vendor the SDK's SwiftPM-only deps as a pre-built static binary#69
peachbits wants to merge 2 commits into
masterfrom
zcash-spm-deps-binary

Conversation

@peachbits

@peachbits peachbits commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Summary

The modern ZcashLightClientKit SDK pulls grpc-swift (1.24+), SwiftNIO, SwiftProtobuf and SQLite.swift via SwiftPM only — grpc-swift's CocoaPods releases stopped at 1.8.0 — so those can no longer be CocoaPods dependencys. Consuming the SDK through React Native's spm_dependency would force the entire host app onto dynamic frameworks (and, on RN 0.85, an open-ended community-pod link cascade).

This vendors just those leaf dependencies as a pre-built static binary instead, so the host app stays on static frameworks and the ZcashLightClientKit source keeps compiling in-pod exactly as it does today.

What changed

  • scripts/buildVendoredDeps.ts (new): builds the SDK's SwiftPM dependency graph (excluding ZcashLightClientKit itself) in Release config for device (arm64) and simulator (arm64 + x86_64 — same arch baseline as libzcashlc.xcframework, so Intel Macs still build), merging raw per-target objects per arch and lipo-ing the simulator slices. Harvests the Swift .swiftmodules + C module.modulemaps the in-pod SDK source imports. Dependency versions are pinned to the SDK's own Package.resolved (-disableAutomaticPackageResolution hard-fails on drift). Driven by npm run update-sources; output is gitignored and shipped in the npm tarball (same model as libzcashlc.xcframework).
  • react-native-zcash.podspec: drops the gRPC-Swift / SQLite.swift pod dependencies; vendors the per-platform deps binary via SDK-conditional -force_load plus the module search paths. Bridge, SDK-source vendoring, checkpoints, and libzcashlc are unchanged.
  • Compiler-version guard: binary .swiftmodule files only load under the exact Swift compiler that produced them (the stable alternative — library-evolution .swiftinterface — is unavailable because swift-nio doesn't compile under evolution). The build stamps ios/vendored/swift-version.txt; the podspec fails fast at pod-install time with actionable instructions (npm run update-sources rebuilds locally, ~10–15 min) when the consuming Xcode doesn't match. This does mean the published tarball's prebuilt deps are coupled to the publishing Xcode — the guard turns that from a cryptic compile error into a one-command fix.
  • .gitignore: treats ios/vendored/ as generated.

Tradeoff: this deliberately gives up Xcode-agnosticism

Today's package is compiler-agnostic by construction: every Swift line (bridge + vendored SDK source + the gRPC/SQLite pods) compiles from source on the consumer's machine, and the only prebuilt binary (libzcashlc) sits behind a C interface, which is ABI-stable across all toolchains — any Xcode works. This PR introduces the package's first prebuilt Swift binary, and consuming prebuilt Swift means reading .swiftmodule files, a compiler-version-locked format (Swift's machine-code ABI is stable; its module format is not). The portable alternative (library-evolution .swiftinterface) is off the table by swift-nio upstream policy (apple/swift-nio#2470, #2897 — closed "not planned"), so the lock is permanent while NIO is in the SDK's dependency graph.

The rejected alternative that would preserve Xcode-agnosticism: vendor the grpc/NIO/SwiftProtobuf source into the pod the way the SDK source is vendored. Rejected because it means hand-maintaining a CocoaPods replica of SwiftPM's ~35-target build graph (including the CNIOBoringSSL C target and per-target module maps) and re-deriving it on every SDK bump. The prebuilt binary trades that permanent maintenance burden for: coupling to the publishing Xcode, mitigated by the stamp/fail-fast guard and a one-command local rebuild. If sustained mixed-Xcode pain ever exceeds that maintenance cost, the revisit levers are (a) source-vendoring the deps, or (b) per-compiler prebuilt variants as GitHub release assets (same download pattern as libzcashlc).

sqlite3 note (correcting an earlier claim)

An earlier revision of this PR claimed it deduplicated sqlite3 symbols. That was a misdiagnosis: SQLite.swift on Apple platforms has no C-shim target (it imports the system sqlite3 clang module), so the deps binary never contained sqlite3 definitions in the first place. The duplicate-sqlite3_* ld warnings visible in Edge builds come from libzcashlc vs react-native-piratechain's libpiratelc (both Rust cores embed sqlite3); they are pre-existing and unrelated to this change.

Validation

  • npm run update-sources runs clean end-to-end (SDK clone → libzcashlc → source vendoring → deps build for both platforms).
  • Artifacts: device .a = arm64; simulator .a = arm64 + x86_64 (lipo -info verified); swift-version.txt stamped; GRPC/NIO symbols present; 0 sqlite3 definitions; ZcashLightClientKit correctly absent (compiled from source in-pod).
  • Known size note: Release objects carry full DWARF, so the archives are larger than strictly necessary (~178M device / ~354M sim). Static-library content is dead-stripped at app link, so app size is unaffected; slimming the tarball (e.g. strip -S) is a possible follow-up.
  • GUI integration build (Edge app, static frameworks, simulator + device link) — running; results will be posted on this PR.

Pairs with the GUI's RN 0.85 upgrade: EdgeApp/edge-react-gui#6064

🤖 Generated with Claude Code

@peachbits

Copy link
Copy Markdown
Contributor Author

Validation passed. A full npm run update-sources ran clean end-to-end (clone SDK → libzcashlc → copySwift → buildVendoredDeps for device + simulator). Generated ios/vendored/:

  • libZcashDeps-ios-arm64.a (device, arm64) + libZcashDeps-ios-arm64-simulator.a (sim)
  • 24 Swift .swiftmodules + 13 C modules
  • Sanity: GRPC (21,870) + NIO (93,718) symbols present; ZcashLightClientKit = 0 (in-pod source, correctly excluded); sqlite3 defined = 0 (dedup works, host provides it)

This is the same artifact set the manual spike built — which builds + runs a ZEC wallet on the simulator.

@peachbits
peachbits force-pushed the zcash-spm-deps-binary branch from af4b872 to e5bcf7e Compare July 3, 2026 17:57
@peachbits

Copy link
Copy Markdown
Contributor Author

⚠️ Correction to my earlier validation comment: the claim that this PR removes duplicate sqlite3 symbols was wrong. SQLite.swift has no C-shim target on Apple platforms (it links the system sqlite3 module), so the deps binary never contained sqlite3 definitions — there was nothing to exclude. The 264 duplicate-sqlite3_* ld warnings in Edge builds are pre-existing and come from libzcashlc (zcash Rust core) vs libpiratelc (piratechain Rust core), both of which embed sqlite3. They are unrelated to this PR and are not fixed by it.

The branch has been amended (af4b872e5bcf7e) with: Release-config deps build, fat simulator slice (arm64+x86_64), versions pinned to the SDK's Package.resolved, a Swift-compiler-version stamp + pod-install fail-fast guard, and truthful sqlite comments.

@peachbits

Copy link
Copy Markdown
Contributor Author

GUI integration validation complete (Edge app, static frameworks, against the regenerated ios/vendored/ from this branch's npm run update-sources):

  • Simulator build: SUCCEEDED — 0 duplicate-symbol errors, 0 undefined symbols
  • Device build (arm64, first device link of this architecture): SUCCEEDED — exercises the OTHER_LDFLAGS[sdk=iphoneos*] force_load path and the Release device slice
  • Runtime smoke (simulator): app boots to login, PIN login works, wallet list renders the existing ZEC wallets with balances — i.e. the Release deps binary + in-pod SDK source load and run end-to-end
  • The 264 duplicate-sqlite3_* ld warnings appear identically on both platforms — consistent with the corrected diagnosis (pre-existing libzcashlc vs libpiratelc, unrelated to this PR)

One integration note for Edge: after npm ci, npm run prepare must run before building (repo convention — applies patch-package); the first tie-off attempt failed on an unpatched react-native-store-review, not on anything in this PR.

@peachbits
peachbits force-pushed the zcash-spm-deps-binary branch from e5bcf7e to 128a823 Compare July 3, 2026 18:46
@peachbits

Copy link
Copy Markdown
Contributor Author

🔎 Major correction + second commit (3b0ca95): the force_load mechanism never worked, and all prior GUI validation was accidentally linking against piratechain's grpc-1.8 pods.

What we found (while validating SDK 2.6.0-alpha.6 against develop in Release config): the podspec's pod_target_xcconfig OTHER_LDFLAGS -force_load flags never reached any link step — a static-framework pod's Libtool step ignores OTHER_LDFLAGS, and pod-target settings don't propagate to the app link. Every earlier green build resolved zcash's grpc/NIO references from react-native-piratechain's gRPC-Swift 1.8 CocoaPods instead: compiled against 1.27 modules, linked against 1.8 definitions. Debug's -Onone surface overlap masked it; Release optimization exposed it (DequeModule internals in generic specializations).

The fix: deps now ship as ios/vendored/libZcashDeps.xcframework via vendored_frameworks — a real link input CocoaPods places on the app link line. Isolation-validated: with piratechain's iOS pods removed from the host (no 1.8 fallback possible), a develop Release build links with zero undefined symbols, and the app binary audit shows the 1.27 GRPC (12k symbols)/NIOCore (18k)/DequeModule definitions, the in-pod SDK (15.7k symbols), and the alpha.6 voting FFI (119 refs).

Also in the commit: SDK bump to 2.6.0-alpha.6 (bridge needed zero changes — API surface is source-compatible; new Voting/PIR module + expanded FFI compile in), per-run DerivedData cleaning in buildVendoredDeps (stale-module poisoning across SDK bumps), and pod floor back to iOS 13.0 so the package serves develop (15.6) and the RN85 branch (16.4).

Caveats: (1) full runtime wallet validation of the isolated build is blocked on this workstation — SentinelOne EDR deletes the ad-hoc-signed Release simulator binary (from build dirs and even sim containers); needs an S1 exclusion or a CI/second machine. Boot-level launch did succeed before the agent intervened. (2) Shipping decision still open: with both this package (grpc 1.27 static) and piratechain (grpc 1.8 pods) in one app, lazy archive semantics mean symbol resolution is link-order-dependent — the clean paths are prelink+symbol-hiding for zcash, or upgrading piratechain onto the same vendored-deps architecture.

peachbits and others added 2 commits July 15, 2026 11:51
The modern ZcashLightClientKit SDK pulls grpc-swift (1.24+), SwiftNIO, SwiftProtobuf and SQLite.swift via SwiftPM only -- grpc-swift's CocoaPods releases stopped at 1.8.0, so it can no longer be a CocoaPods dependency. Consuming the SDK via spm_dependency would force the entire host app onto dynamic frameworks.

Instead, scripts/buildVendoredDeps.ts pre-builds those leaf dependencies in Release per platform (device arm64; simulator arm64+x86_64, matching the libzcashlc baseline), packages them as ios/vendored/libZcashDeps.xcframework, and harvests their Swift/C modules -- generated by 'npm run update-sources' and shipped in the npm tarball. Dependency versions are pinned to the SDK's Package.resolved (-disableAutomaticPackageResolution), and DerivedData is cleaned per run (stale precompiled modules of the libzcashlc header poison SDK bumps). The ZcashLightClientKit source keeps compiling in-pod exactly as before, and the host app stays on static frameworks.

The archive ships as an XCFramework consumed via vendored_frameworks because that is a real link input: CocoaPods places it on the app link line, where the linker pulls members on demand to satisfy the in-pod SDK source's references. (pod_target_xcconfig OTHER_LDFLAGS is NOT viable for this: a static-framework pod's Libtool step ignores OTHER_LDFLAGS, and pod-target settings never propagate to the app link -- flags there are silently dead.)

The binary .swiftmodule format only loads under the exact Swift compiler that produced it (library-evolution .swiftinterface is unavailable: swift-nio doesn't compile under evolution), so the build stamps ios/vendored/swift-version.txt and the podspec fails fast with rebuild instructions when the consuming Xcode doesn't match.

Note: sqlite3 is not part of this binary -- SQLite.swift on Apple platforms links the system sqlite3 module, so there is nothing to exclude. Edge's pre-existing duplicate-sqlite3 ld warnings come from libzcashlc vs libpiratelc (both Rust cores embed sqlite3) and are unrelated to this change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First hardfork-era prerelease: new Voting/PIR module, expanded libzcashlc FFI, updated checkpoints. Pin changes only -- the bridge needs no changes (the SDK's bridge-facing API surface is source-compatible with 2.5.2) and the dependency graph pins are identical, so the vendored deps binary is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@peachbits

Copy link
Copy Markdown
Contributor Author

History rewrite: re-split into two clean commits (tip d237b00).

The previous split leaked mechanism into the version bump: commit 2 carried the xcframework/vendored_frameworks link-input change, the force_load removal, the DerivedData cleaning, and an iOS floor that commit 1 had raised to 16.0 and commit 2 reverted (net zero, and 16.0 would have broken develop's 15.6). All of that is how we vendor, not which SDK — it now lives in commit 1.

  • 36b4907 Vendor the SDK's SwiftPM-only deps as a pre-built static binary — the complete mechanism, done right from the start (XCFramework as a real app-link input; iOS floor stays 13.0; Release/fat-sim/pinning/stamp; per-run DerivedData clean), on stable SDK 2.5.2.
  • d237b00 Bump SDK to 2.6.0-alpha.6 — pins only (ZCASH_SWIFT_SDK_VERSION, xcframework SHA-256, SDK repo hash). Bridge unchanged; dep graph identical.

Tip tree vs the previous tip differs only by two stale header comments in buildVendoredDeps.ts that still described the dead force_load design — spotted during the split and corrected. All validation posted earlier (3-config link + runtime matrix) was run on a tree identical to this tip modulo those comments.

@peachbits

Copy link
Copy Markdown
Contributor Author

📌 Open item found during review discussion: react-native-piratechain coordination is required before this ships.

piratechain still declares the gRPC-Swift ~> 1.8 + SQLite.swift pod dependencies. In an app with this PR, both stacks are present and the link only stays green via static-archive pull semantics: zcash's -force_loaded vendored deps define every grpc/NIO symbol first, so piratechain's code (compiled against 1.8 modules) resolves against the vendored 1.27 objects at runtime, and the 1.8 pod's archives mostly never get pulled. Works today by ABI/link-order luck; one symbol skew away from duplicate-symbol errors or untested mixed-version runtime behavior.

Resolution options (one copy of the deps must exist, exactly one force_load):

  1. Shared deps package (cleanest): extract ios/vendored/ into a small shared pod both react-native-zcash and react-native-piratechain depend on.
  2. Provide-if-absent: each podspec ships the artifacts but only emits force_load/search-paths when the sibling isn't installed.
  3. Near-term stopgap: pin the empirically-working combination and document the fragility.

Until then, the current state (pirate on 1.8 pods) is the tested configuration.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant