Feat: Android support

[jaoleal]

This pr contain 3 commits:

1. Build instructions for android building, concise and aiming to be executed on the nix environment introduced in the second commit.
2. Nix expressions on flake.nix gathering dependencies for building and testing the android output.
3. CI runner that evaluates the nix expressions.

[sedited]

This looks nice, can you rebase this?

[jaoleal]

Changelog:

• Removed some PoC code
• Included android native building in ci (whitout nix).
• A fix regarding building on macos was included to.
• Rebased on top of Chore: remove bindgen and replace with hand-rolled FFI bindings #163 and included a fix regarding pointer sizes.

[jaoleal]

I reduced a lot the code and i think this is the concise version to support android

[alexanderwiederin]

I don't think android is a supported target for libbitcoinkernel. Doesn't this require upstream work on bitcoin core first?

[alexanderwiederin]

From what I understand the ubuntu boost packages don't ship with the cmake config that core needs when cross-compiling.

Am I right that you got it working on nix locally? If so, can we stick with with only supporting the nix build for android?

[jaoleal]

Am I right that you got it working on nix locally? If so, can we stick with with only supporting the nix build for android?

Yes, i worked all this on nix but Im not sure about leaving android building exclusively for nix, this would make it a direct dependency of the project and its not something really special that only nix can do. Besides the dependency problem, nix can drift versions because of nixpkgs releases and that can raise problems only by itself.

Therefore, i still would like for CI to keep the runner as ubuntu. Its easier for the general developer audience to replicate the build on their machine if they need

Edit: see the comment below

[jaoleal]

Correction, I changed my mind... The idea is that the build.rs file is already bloated because of the android support additions, adding support for ubuntu bloated it even more... It looks like leaving to nix to cross-compile to android is actually a good choice, the changes stay minimal on build.rs side and it have broader support running on linux and macos machines... Ill make sure to document that somewhere and state how to make the android build.

[jaoleal]

Okay the android build being exclusive for nix made things easier for this PR, the changes on build.rs are minimal... I included some inline doc comments so the reviewer can follow why each line were added.

I added a section on the readme explaining the android build but its not that explanatory in a level that one can learn how to build this lib to android, do you guys think thats needed ?

I can extend the docs to include instructions for one to reproduce it on a downstream build.rs, just tell me where to put it.
AFAIK is trivial for those using Kotlin, as mandacaru is using, to pull an already compiled shared object during build time. Thats what ill do on floresta-nix for mandacaru, ideally the CI would also offer the compiled objects too for those who might need it.

[alexanderwiederin]

Great to see this working.

If we land the hand-written bindings PR (#163) first, the changes in build.rs should become simpler. I would suggest we wait for it.

[sedited]

@jaoleal I'm interested in getting this change through. Can you rebase it and address Alex' comments?

[jaoleal]

@jaoleal I'm interested in getting this change through. Can you rebase it and address Alex' comments?

Yes, sorry for taking long... I was waiting for the bindgen pr to be merged so we had sure what that would imply here without rebasing on a pr

[jaoleal]

Addressed @alexanderwiederin suggestions and fixes

[alexanderwiederin]

Getting there! Could you update the PR description? I don't think it's a POC anymore. The PR also consists of four and not three commits.

[jaoleal]

Getting there!

Gosh, I entered in automatic mode for this one... Sorry

[jaoleal]

Changelog:

• I reworded some commits into better and descriptive messages.
• I squashed the last commit that introduce test preparation into the commit that introduced the nix build instructions, so the nix changes are final. The commit message should also explain that testing is restricted to select capable hosts.
• I removed a variable and the newly introduced methods and inline their logic. I think this helps readers to understand whats necessary only for the android scope. Also, the less diff the better ?

[jaoleal]

Just a thing, since the android outputs can only be built with nix, I still didnt found a way to integrate it on floresta upstream so what im planning is to offer the compiled binaries on floresta-nix with the current patches presented here.

I know that this is more a floresta problem but I think other projects, those depending on rust-bitcoinkernel as floresta does, may lead into the same problem and maybe a good solution would be if this project offered its own pre-compiled Android artifacts, what you guys think about it ?

[Davidson-Souza]

Since yesterday I'm experimenting with this PR and integrating Floresta with Maven, so Android devs can consume Floresta as any other library. We already had some scaffolding in floresta-ffi but couldn't build bitcoinkernel due to the things fixed here.

I've opened getfloresta/floresta-ffi#11 fixing it, I can now build and run floresta with kernel enabled. Writing the main pain points here for reviewers and future reference. All my experiments did not use Nix. I've coded this on Arch Linux and Ci runs on Ubuntu.

• NDK couldn't find boost's headers in the host system. I had to symlink them to my ndk folder. This is kinda ugly, currently looking for a better approach
• export Boost_DIR and CMAKE_PREFIX_PATH, which is fine
• I also had to vendor libc++_shared.so because it kept loading what Android calls the system STL. This is a dummy STL with only new/delete. When kernel tries to use some C++ name, I would get a linker error. I'm currently digging the docs to figure a better way to do that.
• Something during the build is using a static version of android's libc, but using the symbol at runtime without explicitly mentioning it to the linker, so it segfaults on runtime. I still didn't really get what's going on, but it seems not related to kernel or C++.

a minor nit I've found when debugging this is that sometimes build.rs just ignores cmake's output, so it can silently fail and get a cryptic error about not finding the shared object. You can fix this with something like:

    let install_output = Command::new("cmake")
            .arg("--install")
            .arg(&build_dir)
            .arg("--config")
            .arg(build_config)
            .output()
            .unwrap();

        if !install_output.status.success() {
            let stdout = String::from_utf8_lossy(&install_output.stdout);
            let stderr = String::from_utf8_lossy(&install_output.stderr);
            panic!(
                "cmake --install failed with status {}.\nstdout:\n{}\nstderr:\n{}",
                install_output.status, stdout, stderr
            );
    }

[sedited]

Thanks for the report @Davidson-Souza! I hope we can get rid of boost soonish, there is some work happening towards that on the Bitcoin Core side.

[alexanderwiederin]

Thanks @Davidson-Souza. I've opened a PR (still in draft) for the nit in build.rs.

This looks ready to merge from my side.

@jaoleal @Davidson-Souza, given your context on the Android integration, I'll make sure to loop you in on any future issues or PRs touching this build path.

[jaoleal]

Hey folks, ill address the suggestions.

Just a thing about the last change, in order to make armv7 floresta target to work i had to patch it with

  // The pre-compiled libcompiler_builtins for armv7-linux-androideabi
        // ships ARM EABI helper symbols tagged with @@LIBC_N (e.g.
        // __aeabi_memcpy@@LIBC_N).  When lld links a shared library or
        // executable it errors because the LIBC_N version node is not
        // defined in any version script.  --exclude-libs,ALL marks every
        // symbol pulled from static archives as local, which suppresses
        // the version-node error.
        if ndk_triple == "arm-linux-androideabi" {
            println!("cargo:rustc-link-arg=-Wl,--exclude-libs,ALL");
        }

[jaoleal]

Rebased

[jaoleal]

Okay, CHANGELOG:

• Addressed suggestions about naming and extra commenting.
• Now that davidson achieved to build the targets whitout nix, i included manual build structions that made the Android Section on README pretty verbose, i think its nice to have but tell me if you guys think its too much.
• Renamed the outputs with a libbitcoinkernel prefix to be more explicit about what they are.

[jaoleal]

Consuming the nix outputs look like this https://github.com/getfloresta/floresta-nix/pull/29/changes#diff-c765aab0783c96c0d736cf34b8b34bf38166bbfa04478db4391ce45e16d6b030R40-R42

[alexanderwiederin]

Nice - thanks for doing all this!

I had two smaller comments that build on my previous ones - they are mostly nits so feel free to ignore.

Regarding the manual setup and prebuilt LIBBITCOINKERNEL_LIB_DIR: Can we focus this PR only on supporting nix? The next PR could be focused on the manual setup instructions and include a CI job so we can continuously make sure it works manually. The third PR could focus on the the LIBBITCOINKERNEL_LIB_DIR proposal.

I appreciate your patience here, I know this has taken a while. I think breaking these three pieces down has a lot of value!

[alexanderwiederin]

Missing the closing "`" for result/include

[alexanderwiederin]

- name: Build${{ matrix.package != 'libbitcoinkernel-android-x86_64' && ' and Test' || '' }}

[jaoleal]

I think breaking these three pieces down has a lot of value!

I agree with you, do you mind if i just cherry pick them out and open a pr so we can work on them later ?