-
Notifications
You must be signed in to change notification settings - Fork 91
Feat/dart precompiled binary #1307
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
dbee4b2
ab987ef
7aa1337
47b89ba
403a268
1e57740
a717894
754974f
2f8d685
00e7fdd
01ce73a
48ab20d
7b709d0
317d395
f0fc5a7
d3ad65f
a1199e1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| name: Precompile Payjoin Dart Binaries | ||
|
|
||
| on: | ||
| workflow_call: | ||
| secrets: | ||
| PRECOMPILED_PRIVATE_KEY: | ||
| required: true | ||
| workflow_dispatch: | ||
| push: | ||
| branches: | ||
| - master | ||
|
|
||
| permissions: | ||
| contents: write | ||
|
|
||
| env: | ||
| CRATE_DIR: "native" | ||
| CRATE_PACKAGE: "payjoin-ffi-wrapper" | ||
| WORK_DIR: "payjoin-ffi/dart" | ||
|
|
||
| jobs: | ||
| precompile_macos_ios: | ||
| runs-on: macos-latest | ||
| env: | ||
| PRIVATE_KEY: ${{ secrets.PRECOMPILED_PRIVATE_KEY }} | ||
| GH_TOKEN: ${{ github.token }} | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions-rs/toolchain@v1 | ||
| with: | ||
| profile: minimal | ||
| toolchain: 1.85.1 | ||
| override: true | ||
| - uses: dart-lang/setup-dart@v1 | ||
| - uses: subosito/flutter-action@v2 | ||
| with: | ||
| channel: "stable" | ||
| - name: Pub get | ||
| working-directory: ${{ env.WORK_DIR }} | ||
| run: dart pub get | ||
| - name: Precompile (macOS + iOS) | ||
| working-directory: ${{ env.WORK_DIR }} | ||
| run: | | ||
| set -euo pipefail | ||
| dart run bin/build_tool.dart precompile-binaries \ | ||
| -v \ | ||
| --os=macos \ | ||
| --manifest-dir="${CRATE_DIR}" \ | ||
| --crate-package="${CRATE_PACKAGE}" \ | ||
| --repository="${GITHUB_REPOSITORY}" | ||
|
|
||
| precompile_android: | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| PRIVATE_KEY: ${{ secrets.PRECOMPILED_PRIVATE_KEY }} | ||
| GH_TOKEN: ${{ github.token }} | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
| ANDROID_NDK_VERSION: "26.3.11579264" | ||
| ANDROID_MIN_SDK: "23" | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions-rs/toolchain@v1 | ||
| with: | ||
| profile: minimal | ||
| toolchain: 1.85.1 | ||
| override: true | ||
| - uses: dart-lang/setup-dart@v1 | ||
| - name: Set up Android SDK | ||
| uses: android-actions/setup-android@v3 | ||
| - name: Install NDK | ||
| run: | | ||
| set -euo pipefail | ||
| sdkmanager --install "ndk;${ANDROID_NDK_VERSION}" | ||
| - name: Install cargo-ndk | ||
| run: cargo install cargo-ndk --locked --version 3.5.4 | ||
| - name: Pub get | ||
| working-directory: ${{ env.WORK_DIR }} | ||
| run: dart pub get | ||
| - name: Precompile (Android) | ||
| working-directory: ${{ env.WORK_DIR }} | ||
| env: | ||
| ANDROID_SDK_ROOT: ${{ env.ANDROID_SDK_ROOT }} | ||
| run: | | ||
| set -euo pipefail | ||
| ANDROID_SDK_ROOT="${ANDROID_SDK_ROOT:-$ANDROID_HOME}" | ||
| dart run bin/build_tool.dart precompile-binaries \ | ||
| -v \ | ||
| --os=android \ | ||
| --manifest-dir="${CRATE_DIR}" \ | ||
| --crate-package="${CRATE_PACKAGE}" \ | ||
| --repository="${GITHUB_REPOSITORY}" \ | ||
| --android-sdk-location="${ANDROID_SDK_ROOT}" \ | ||
| --android-ndk-version="${ANDROID_NDK_VERSION}" \ | ||
| --android-min-sdk-version="${ANDROID_MIN_SDK}" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,6 +3,7 @@ target | |
| !example.config.toml | ||
| *.sqlite | ||
| Cargo.lock | ||
|
|
||
| .vscode | ||
| mutants.out* | ||
| *.ikm | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,6 +2,48 @@ | |
|
|
||
| Welcome to the Dart language bindings for the [Payjoin Dev Kit](https://payjoindevkit.org/)! | ||
|
|
||
| ## Using the bindings in your app | ||
|
|
||
| Declare the package as a dependency just like any other Dart package. When developing against the repo directly, point at the local path and let `flutter pub get` (or `dart pub get`) run the build hook: | ||
|
|
||
| ```yaml | ||
| dependencies: | ||
| payjoin: | ||
| path: ../rust-payjoin/payjoin-ffi/dart | ||
| ``` | ||
|
Comment on lines
+10
to
+13
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shouldn't the SOP be to get this from |
||
|
|
||
| The `hook/build.dart` script drives `native_toolchain_rust` (plus the precompiled-binaries helper) so that `flutter pub get` downloads the verified binaries when available or builds the native crate locally on demand. | ||
|
|
||
| If you prefer to inspect or regenerate `payjoin.dart` manually, run the binder script from the `payjoin-ffi/dart` directory: | ||
|
|
||
| ```bash | ||
| bash ./scripts/generate_bindings.sh | ||
| ``` | ||
|
|
||
| This produces `lib/payjoin.dart` and the native artifacts under `target/`. These files are not tracked in the repository, so you should regenerate them locally whenever the Rust API changes. | ||
|
|
||
| ## Precompiled binaries | ||
|
|
||
| This package supports downloading signed precompiled binaries or building locally via Dart's Native Assets hook. | ||
| If precompiled binaries are attempted but unavailable or verification fails, it falls back to building from source. | ||
|
|
||
| ### pubspec.yaml configuration | ||
|
|
||
| In your app's `pubspec.yaml`, add the `payjoin` section at the top level (next to `dependencies`), like: | ||
|
|
||
| ```yaml | ||
| payjoin: | ||
| precompiled_binaries: | ||
| mode: auto # auto | always | never | ||
| ``` | ||
|
|
||
| `mode` controls when the precompiled path is used: | ||
| - `auto` prefers local builds if Rust toolchain is detected, otherwise uses precompiled binaries | ||
| - `always` requires precompiled binaries and skips local builds | ||
| - `never` always builds from source via the build hook | ||
|
|
||
| If your tooling must rely on the signed GitHub releases, set `mode: always` and configure `artifact_host`/`public_key` to point at the published assets so `PrecompiledBuilder` can download the `precompiled_<crateHash>` bundles (macOS/iOS + Android builds are published via `.github/workflows/payjoin-dart-precompile-binaries.yml`). | ||
|
|
||
| ## Running Tests | ||
|
|
||
| Follow these steps to clone the repository and run the tests. | ||
|
|
@@ -16,3 +58,5 @@ bash ./scripts/generate_bindings.sh | |
| # Run all tests | ||
| dart test | ||
| ``` | ||
|
|
||
| Maintainers: see `docs/precompiled_binaries.md` for CI details, manual release steps, and configuration. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this file exists as of commit 1e57740 where this is introduced
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Oops, yes, adding it now. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| import 'package:payjoin/src/precompiled/cli/cli.dart'; | ||
|
|
||
| Future<void> main(List<String> args) async { | ||
| await runCli(args); | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| # Precompiled binaries (maintainers) | ||
|
|
||
| This document describes how precompiled binaries are built, signed, and published for the Dart package. | ||
|
|
||
| ## Overview | ||
|
|
||
| - CI builds and uploads precompiled binaries via a GitHub Actions workflow. | ||
| - Artifacts are tagged by the crate hash and uploaded to a GitHub release. | ||
| - Each binary is signed with an Ed25519 key; the public key is embedded in `pubspec.yaml`. | ||
| - The build hook downloads verified binaries when appropriate and falls back to local builds. | ||
|
|
||
| ## Mode behavior | ||
|
|
||
| The `mode` configuration in app `pubspec.yaml` controls fallback behavior: | ||
|
|
||
| - `auto`: prefers local builds if `rustup` is detected; otherwise downloads precompiled binaries. | ||
| - `always`: requires precompiled binaries and skips local builds. | ||
| - `never`: always builds locally via the standard build hook. | ||
|
|
||
| ## CI workflow | ||
|
|
||
| The workflow runs on manual dispatch or on a workflow call. It invokes: | ||
|
|
||
| ``` | ||
| dart run bin/build_tool.dart precompile-binaries ... | ||
| ``` | ||
|
|
||
| It builds macOS/iOS and Android targets. | ||
|
|
||
| ## Release expectations | ||
|
|
||
| - The workflow creates/releases a GitHub release named `precompiled_<crateHash>`. | ||
| - If the release already exists, the workflow uploads missing assets without rebuilding. | ||
| - If `gh release view precompiled_<crateHash>` fails locally, rerun `dart run bin/build_tool.dart precompile-binaries ...`. | ||
|
|
||
| ## How the download works | ||
|
|
||
| - The crate hash is computed from the Rust crate sources plus the plugin's `precompiled_binaries` config. | ||
| - The release tag is `precompiled_<crateHash>`. | ||
| - Assets are named `<targetTriple>_<libraryFileName>` with a matching `.sig` file. | ||
| - The hook downloads the signature and binary, verifies it, then places it in the build output. | ||
| - If any step fails, the hook builds locally via the standard build hook. | ||
|
|
||
| ## Manual release (local) | ||
|
|
||
| Required environment variables: | ||
|
|
||
| - `PRIVATE_KEY` (Ed25519 private key, hex-encoded, 64 bytes) | ||
| - `GH_TOKEN` or `GITHUB_TOKEN` (GitHub token with release upload permissions) | ||
|
|
||
| Example: | ||
|
|
||
| ``` | ||
| dart run bin/build_tool.dart precompile-binaries \ | ||
| --manifest-dir="native" \ | ||
| --crate-package="payjoin-ffi-wrapper" \ | ||
| --repository="owner/repo" \ | ||
| --os=macos | ||
| ``` | ||
|
|
||
| ## Troubleshooting & ops tips | ||
|
|
||
| - If `gh release view precompiled_<crateHash>` shows a release without expected assets, rerun the build locally. | ||
| - A stale crate hash (because sources or `precompiled_binaries` config changed) will point to a release that either doesn't exist yet or lacks current binaries; re-run `dart run bin/build_tool.dart hash --manifest-dir=native` to confirm the hash and rebuild with the same inputs. | ||
| - Use `gh release view precompiled_<crateHash> --json assets --jq '.assets[].name'` to inspect uploaded assets. | ||
| - Set `PAYJOIN_DART_PRECOMPILED_VERBOSE=1` to see download and verification details when debugging consumer builds. | ||
|
|
||
| ## Configuration knobs | ||
|
|
||
| - `rust-toolchain.toml` controls the Rust channel and target list. | ||
| - `pubspec.yaml` under `payjoin.precompiled_binaries` must include: | ||
| - `artifact_host` (owner/repo) | ||
| - `public_key` (Ed25519 public key, hex-encoded, 32 bytes) | ||
|
|
||
| ## Environment, keys, and secrets | ||
|
|
||
| - `PRIVATE_KEY`: 64-byte hex string (Ed25519 private key). Keep it out of source control. | ||
| - `PUBLIC_KEY`: Add the matching 32-byte hex public key to `pubspec.yaml`. | ||
| - `GH_TOKEN` / `GITHUB_TOKEN`: release upload permissions. | ||
| - `PAYJOIN_DART_PRECOMPILED_VERBOSE=1`: optional; shows download and verification details. | ||
|
|
||
| Generate a keypair with `dart run bin/build_tool.dart gen-key` and copy the printed `PRIVATE_KEY`/`PUBLIC_KEY` values. Rotate the pair if you ever suspect the signing key was exposed, and update every release’s config accordingly. | ||
|
|
||
| ## Security reminder | ||
|
|
||
| - Treat the `PRIVATE_KEY` used for signing as highly sensitive; do not commit it to version control and rotate it immediately if you suspect compromise. | ||
| - Update the public key in `pubspec.yaml` if the private key is rotated so consumers can still verify downloads. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,11 +1,29 @@ | ||
| import 'package:hooks/hooks.dart'; | ||
| import 'package:native_toolchain_rust/native_toolchain_rust.dart'; | ||
| import 'package:native_toolchain_rust/native_toolchain_rust.dart' as ntr; | ||
| import 'package:payjoin/src/precompiled/precompiled_builder.dart'; | ||
|
|
||
| void main(List<String> args) async { | ||
| await build(args, (input, output) async { | ||
| await RustBuilder( | ||
| if (!input.config.linkingEnabled) { | ||
| return; | ||
| } | ||
| final builder = PrecompiledBuilder( | ||
| assetName: 'uniffi:payjoin_ffi', | ||
| features: ['_test-utils'], | ||
| ).run(input: input, output: output); | ||
| buildModeName: ntr.BuildMode.release.name, | ||
| fallback: (input, output, assetRouting, logger) async { | ||
| final rustBuilder = ntr.RustBuilder( | ||
| assetName: 'uniffi:payjoin_ffi', | ||
| buildMode: ntr.BuildMode.release, | ||
| features: ['_test-utils'], | ||
| ); | ||
| await rustBuilder.run( | ||
| input: input, | ||
| output: output, | ||
| assetRouting: assetRouting, | ||
| logger: logger, | ||
| ); | ||
| }, | ||
| ); | ||
| await builder.run(input: input, output: output); | ||
| }); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
? why include this commit