Commit b5dab26
authored
Migrate to dylibs (#2202)
When I first submitted the patch to compile libsql for mobiles I didn't
know what I was doing. A static library is the easiest way to make
things work but takes too much disk space as all the necessary code is
packaged. Migrating to dylibs is better for mobiles as a lot of the
binary size is reduced.
I've managed to get iOS working and the binary went from 35mbs to 9mbs.
However, on Android I'm getting a compilation error (even on the main
branch without my changes), maybe @levydsa can give some pointers on
what's wrong with the build pipeline right now?
Once this PR is merged I can update op-sqlite to use this new smaller
version (OP-Engineering/op-sqlite#365)
When I try to compile for Android (via `make android`) I get the
following error
```
make android
cargo ndk --target aarch64-linux-android --platform 31 build --release --color=always
Building arm64-v8a (aarch64-linux-android)
Compiling libsql-ffi v0.9.29 (/Users/osp/Developer/libsql/libsql-ffi)
Compiling sql-experimental v0.0.0 (/Users/osp/Developer/libsql/bindings/c)
error: failed to run custom build command for `libsql-ffi v0.9.29 (/Users/osp/Developer/libsql/libsql-ffi)`
Caused by:
process didn't exit successfully: `/Users/osp/Developer/libsql/target/release/build/libsql-ffi-da53f4f133127afb/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-changed=bundled/src/sqlite3.c
cargo:rerun-if-changed=bundled/SQLite3MultipleCiphers
cargo:lib_dir=/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out
CMAKE_TOOLCHAIN_FILE_aarch64-linux-android = None
CMAKE_TOOLCHAIN_FILE_aarch64_linux_android = None
TARGET_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_GENERATOR_aarch64-linux-android = None
CMAKE_GENERATOR_aarch64_linux_android = None
TARGET_CMAKE_GENERATOR = None
CMAKE_GENERATOR = None
CMAKE_PREFIX_PATH_aarch64-linux-android = None
CMAKE_PREFIX_PATH_aarch64_linux_android = None
TARGET_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_aarch64-linux-android = None
CMAKE_aarch64_linux_android = None
TARGET_CMAKE = None
CMAKE = None
running: cd "/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc" "-DSQLITE3MC_STATIC=ON" "-DCODEC_TYPE=AES256" "-DSQLITE3MC_BUILD_SHELL=OFF" "-DSQLITE_SHELL_IS_UTF8=OFF" "-DSQLITE_USER_AUTHENTICATION=OFF" "-DSQLITE_SECURE_DELETE=OFF" "-DSQLITE_ENABLE_COLUMN_METADATA=ON" "-DSQLITE_USE_URI=ON" "-DCMAKE_POSITION_INDEPENDENT_CODE=ON" "-DCMAKE_ANDROID_NDK=/Users/osp/Library/Android/sdk/ndk/26.1.10909125" "-DLIBSQL_ENCRYPTION=1" "-DANDROID_PLATFORM=31" "-DCMAKE_SYSTEM_NAME=Android" "-DCMAKE_SYSTEM_PROCESSOR=aarch64" "-DCMAKE_INSTALL_PREFIX=/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out" "-DCMAKE_C_FLAGS= -DANDROID -ffunction-sections -fdata-sections -fPIC --target=aarch64-linux-android --target=aarch64-linux-android31" "-DCMAKE_C_COMPILER=/Users/osp/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang" "-DCMAKE_CXX_FLAGS= -DANDROID -ffunction-sections -fdata-sections -fPIC --target=aarch64-linux-android --target=aarch64-linux-android31" "-DCMAKE_CXX_COMPILER=/Users/osp/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++" "-DCMAKE_ASM_FLAGS= -DANDROID -ffunction-sections -fdata-sections -fPIC --target=aarch64-linux-android --target=aarch64-linux-android31" "-DCMAKE_ASM_COMPILER=/Users/osp/Library/Android/sdk/ndk/26.1.10909125/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang" "-DCMAKE_BUILD_TYPE=Release"
-- Configuring done (0.0s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/build
running: cd "/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/build" && "cmake" "--build" "/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/build" "--target" "sqlite3mc_static" "--config" "Release" "--parallel" "11"
[ 50%] Building C object CMakeFiles/sqlite3mc_static.dir/src/sqlite3mc.c.o
--- stderr
[libsql-ffi/build.rs:462:9] format!("{BUNDLED_DIR}/SQLite3MultipleCiphers") = "bundled/SQLite3MultipleCiphers"
Will build sqlite3mc_static as STATIC
In file included from /Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sqlite3mc.c:132:
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sqlite3.c:212044:1: warning: non-void function does not return a value in all control paths [-Wreturn-type]
}
^
In file included from /Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sqlite3mc.c:217:
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/cipher_wxaes256.c:168:23: warning: passing 'char *' to parameter of type 'unsigned char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign]
sha256(userPad, 32, digest);
^~~~~~
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sha2.c:337:76: note: passing argument to parameter 'digest' here
void sha256(const unsigned char *message, unsigned int len, unsigned char *digest)
^
In file included from /Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sqlite3mc.c:217:
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/cipher_wxaes256.c:171:12: warning: passing 'char *' to parameter of type 'const unsigned char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign]
sha256(digest, KEYLENGTH_AES256, digest);
^~~~~~
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sha2.c:337:34: note: passing argument to parameter 'message' here
void sha256(const unsigned char *message, unsigned int len, unsigned char *digest)
^
In file included from /Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sqlite3mc.c:217:
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/cipher_wxaes256.c:171:38: warning: passing 'char *' to parameter of type 'unsigned char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign]
sha256(digest, KEYLENGTH_AES256, digest);
^~~~~~
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sha2.c:337:76: note: passing argument to parameter 'digest' here
void sha256(const unsigned char *message, unsigned int len, unsigned char *digest)
^
In file included from /Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/sqlite3mc.c:212:
In file included from /Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/rijndael.c:85:
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:513:14: error: always_inline function 'vaeseq_u8' requires target feature 'aes', but would be inlined into function 'aesGenKeyEncryptInternal' that is compiled without support for 'aes'
v8 = vaeseq_u8(v8, vdupq_n_u8(0));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:576:26: error: always_inline function 'vaesimcq_u8' requires target feature 'aes', but would be inlined into function 'aesGenKeyDecrypt' that is compiled without support for 'aes'
keySchedule[j] = vaesimcq_u8(tempKeySchedule[j]);
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:623:29: error: always_inline function 'vaeseq_u8' requires target feature 'aes', but would be inlined into function 'aesEncryptCBC' that is compiled without support for 'aes'
feedback = vaesmcq_u8(vaeseq_u8(feedback, key[j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:623:18: error: always_inline function 'vaesmcq_u8' requires target feature 'aes', but would be inlined into function 'aesEncryptCBC' that is compiled without support for 'aes'
feedback = vaesmcq_u8(vaeseq_u8(feedback, key[j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:625:16: error: always_inline function 'vaeseq_u8' requires target feature 'aes', but would be inlined into function 'aesEncryptCBC' that is compiled without support for 'aes'
feedback = vaeseq_u8(feedback, key[numberOfRounds-1]);
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:647:29: error: always_inline function 'vaeseq_u8' requires target feature 'aes', but would be inlined into function 'aesEncryptCBC' that is compiled without support for 'aes'
feedback = vaesmcq_u8(vaeseq_u8(feedback, key[j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:647:18: error: always_inline function 'vaesmcq_u8' requires target feature 'aes', but would be inlined into function 'aesEncryptCBC' that is compiled without support for 'aes'
feedback = vaesmcq_u8(vaeseq_u8(feedback, key[j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:649:16: error: always_inline function 'vaeseq_u8' requires target feature 'aes', but would be inlined into function 'aesEncryptCBC' that is compiled without support for 'aes'
feedback = vaeseq_u8(feedback, key[numberOfRounds-1]);
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:699:26: error: always_inline function 'vaesdq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesimcq_u8(vaesdq_u8(data, key[numberOfRounds-j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:699:14: error: always_inline function 'vaesimcq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesimcq_u8(vaesdq_u8(data, key[numberOfRounds-j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:701:12: error: always_inline function 'vaesdq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesdq_u8(data, key[1]);
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:716:26: error: always_inline function 'vaesdq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesimcq_u8(vaesdq_u8(data, key[numberOfRounds-j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:716:14: error: always_inline function 'vaesimcq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesimcq_u8(vaesdq_u8(data, key[numberOfRounds-j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:718:12: error: always_inline function 'vaesdq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesdq_u8(data, key[1]);
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:744:26: error: always_inline function 'vaesdq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesimcq_u8(vaesdq_u8(data, key[numberOfRounds-j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:744:14: error: always_inline function 'vaesimcq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesimcq_u8(vaesdq_u8(data, key[numberOfRounds-j]));
^
/Users/osp/Developer/libsql/target/aarch64-linux-android/release/build/libsql-ffi-4d1cee77a8dff0d3/out/sqlite3mc/src/aes_hardware.c:746:12: error: always_inline function 'vaesdq_u8' requires target feature 'aes', but would be inlined into function 'aesDecryptCBC' that is compiled without support for 'aes'
data = vaesdq_u8(data, key[1]);
^
4 warnings and 17 errors generated.
make[4]: *** [CMakeFiles/sqlite3mc_static.dir/src/sqlite3mc.c.o] Error 1
make[3]: *** [CMakeFiles/sqlite3mc_static.dir/all] Error 2
make[2]: *** [CMakeFiles/sqlite3mc_static.dir/rule] Error 2
make[1]: *** [sqlite3mc_static] Error 2
thread 'main' (533981) panicked at /Users/osp/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/cmake-0.1.54/src/lib.rs:1119:5:
command did not execute successfully, got: exit status: 2
build script failed, must exit now
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
note: If the build failed due to a missing target, you can run this command:
note:
note: rustup target install aarch64-linux-android
make: *** [aarch64-linux-android] Error 101
```File tree
5 files changed
+153
-32
lines changed- bindings/c
- templates/libsql_experimental.xcframework
- ios-arm64-simulator/libsql_experimental.framework
- ios-arm64/libsql_experimental.framework
- libsql-ffi
5 files changed
+153
-32
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
7 | 10 | | |
8 | | - | |
9 | | - | |
10 | | - | |
11 | | - | |
12 | 11 | | |
13 | 12 | | |
14 | 13 | | |
| |||
21 | 20 | | |
22 | 21 | | |
23 | 22 | | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
24 | 74 | | |
25 | 75 | | |
26 | 76 | | |
27 | 77 | | |
28 | 78 | | |
29 | 79 | | |
30 | 80 | | |
| 81 | + | |
31 | 82 | | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
36 | 88 | | |
37 | 89 | | |
38 | 90 | | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
| |||
Lines changed: 43 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
Lines changed: 24 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
Lines changed: 24 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
504 | 504 | | |
505 | 505 | | |
506 | 506 | | |
507 | | - | |
| 507 | + | |
508 | 508 | | |
509 | 509 | | |
510 | 510 | | |
| |||
0 commit comments