feat(linux): vendor static deps for portable shared libraries#2
feat(linux): vendor static deps for portable shared libraries#2jamesarich wants to merge 339 commits into
Conversation
…ibre#3249) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Bart Louwers <bart@emeel.net>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…-actions group (maplibre#3290) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Michał Gwóźdź <114614618+michalgwo@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Add MLN_LINUX_STATIC_DEPS CMake option that builds zlib, libpng, libjpeg-turbo, libwebp, libuv, and bzip2 from source with -fPIC, then amalgamates them into the mbgl-core static archive. This eliminates runtime dependencies on specific system library versions (ICU, libpng, zlib, etc.) that cause UnsatisfiedLinkError crashes when the shared library is loaded on a different distro or version than it was built on. System curl, X11, OpenGL, libc, and libstdc++ remain dynamic (stable ABI, universally available). Fixes: maplibre/maplibre-compose#627 Changes: - Add vendored_deps.cmake with FetchContent/ExternalProject builds for zlib 1.3.1, libpng 1.6.44, libjpeg-turbo 3.1.0, libwebp 1.5.0, libuv 1.50.0, and bzip2 1.0.8 - Add vendored amalgamation path in linux.cmake (symbols hidden via -Wl,--exclude-libs,ALL to prevent collisions) - Gate freetype PNG/BZip2 system linking behind MLN_LINUX_STATIC_DEPS - Force builtin ICU when vendoring (MLN_USE_BUILTIN_ICU=TRUE) - Require armerge when MLN_LINUX_STATIC_DEPS is enabled - Add INTERFACE_MAPLIBRE_* license metadata for all vendored deps Also fixes (in the non-vendored path): - Use ${ARMERGE} variable instead of bare 'armerge' command - Fix ICU check: ICUUC_FOUND without ${} expansion - Fix typo: 'requestd' -> 'requested' Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Hi @sargunv - we (Meshtastic) are interested in potentially leveraging maplibre in our KMP projects. I'd like to help unblock Desktop JVM maps parity - let me know how we can assist! |
|
Closing — this should target maplibre/maplibre-native instead. Re-opening there. |
Hey, I'd love some help on this front. Are you on the OSMUS slack workspace? The main issue for the desktop JVM parity is the native core integration -- I'm somewhat amateur at modern C++ and very amateur at C++ <> JVM integration and OpenGL/Vulkan/Metal, just figuring it all out as I go. If you have skills/experience there, that's would be extremely helpful. Some immediate issues, off the top of my head:
(It's been a few months since I worked on desktop stuff though, so my memory might be wrong on some things) But the long term goal is to make the MapLibre Compose <> native core integration stable and complete enough that it becomes the path for Android and iOS targets too, rather than the platform-specific SDKs we integrate with today. |
Nope, but you can find me on the Meshtastic Discord @olm3c
I'll take a poke around, thanks for the response! |
Add SKIA_DIVERGENCE.md capturing findings from a focused parallel audit of the Skia backend against the OpenGL backend, covering clipping, mesh geometry, and the 50-100x performance gap at high pitch + high zoom. Each entry is concrete (file:line) so it can be acted on individually. Highest-leverage targets identified: - Per-frame MeshVertex rebuild in Drawable::draw (maplibre#10) — single biggest perf win, no correctness implications - Polyline-level near-plane clipping in the line bucket (#2+maplibre#7+maplibre#9) — fixes the slashes correctness bug at high pitch - Consolidated UBO updates (maplibre#16) - Lift image->makeShader into texture upload path (maplibre#15) - Reuse per-tile clip paths per frame (maplibre#14) Also restore the seattle-slashes _skia-repro test (deleted by the earlier line-clip revert). Headless pitch=85 currently reproduces as solid water rather than the live-path slashes, which is its own mismatch worth tracking. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
At extreme pitch, line-bucket triangles that span the near plane have one vertex with clip-space w near zero. The perspective divide stretches it across the canvas — the documented "slashes" artifact (SKIA_DIVERGENCE.md items #2, maplibre#7, maplibre#9). The CPU near-plane clipper that fills already use (`clipProjectedTriangles`) is attribute-unaware: it linearly interpolates the entire MeshVertex, which corrupts packed line attrs (normal, blur, progress). This is the line-shader-local mitigation: in lineMeshSpecification, lineGradientMeshSpecification, linePatternMeshSpecification, and lineSDFMeshSpecification, clamp `inv_w` to 0 and push the projected ndc to (-1e10, -1e10) when projected.w < 1e-4. Triangles with a behind-near vertex are then trivially clipped by the GPU rasterizer. Tradeoff: triangles partially behind near are entirely dropped instead of clipped at the near plane. Visually that's close to GL's behavior at the edge cases that triggered the bug — and a huge improvement over the canvas-spanning smears. Render-test diff for line-pattern/pitch (a known failure on Skia for unrelated sub-pixel AA) shows the smears gone: expected and actual now match shape-for-shape; only sub-pixel placement differs. Render-test counts unchanged: 1265 passed, 38 failed, 3 errored. The architecturally clean fix — polyline-level clipping in the line bucket, shared with GL — is still pending (maplibre#28 stays open). This is a Skia-local hack, the option (b) the divergence doc described. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Description
Add a new
MLN_LINUX_STATIC_DEPSCMake option that vendors zlib, libpng, libjpeg-turbo, libwebp, libuv, and bzip2 as static libraries built from source. When enabled, these are amalgamated intombgl-corevia armerge, producing a self-contained shared library with no runtime dependency on distro-specific library versions.Problem
Published Linux shared libraries (e.g. JNI
.sofor maplibre-compose) dynamically link against system libraries at build time. When users run on a different distro or version, symbol mismatches causeUnsatisfiedLinkErrorcrashes — particularly ICU (version-mangled C++ symbols), libpng, and zlib. This is the blocker described in maplibre/maplibre-compose#627.Approach
add_subdirectory())-Wl,--exclude-libs,ALLhides vendored symbols from the final.soexportsAll vendored deps inherit
CMAKE_POSITION_INDEPENDENT_CODE=ONfrom the root CMakeLists.txt. The existing system-deps path (MLN_LINUX_STATIC_DEPS=OFF, the default) is completely unchanged.Files changed
CMakeLists.txt— Addedoption(MLN_LINUX_STATIC_DEPS)declarationplatform/linux/cmake/vendored_deps.cmake— New: downloads, builds, and configures all vendored deps with license metadataplatform/linux/linux.cmake— Conditional paths for vendored vs system deps, armerge enforcement, amalgamationvendor/freetype.cmake— Disable PNG/BZip2 when vendoring (gated behindCMAKE_SYSTEM_NAME STREQUAL "Linux" AND MLN_LINUX_STATIC_DEPS)CHANGELOG.md— Added entryTest plan
Linux (Ubuntu 25.04, GCC 15.2, CMake 4.2.3):
MLN_LINUX_STATIC_DEPS=ON: full build succeeds,.sohas no dynamic deps on vendored libs (lddclean), no leaked vendored symbols (nm -Dclean)MLN_LINUX_STATIC_DEPS=OFF: CMake configure succeeds, system-deps path unchanged.soloads and renders map tiles in a Compose Desktop appCross-platform:
MLN_LINUX_STATIC_DEPS)vendor/freetype.cmakechange additionally gated behindCMAKE_SYSTEM_NAME STREQUAL "Linux"Requirements
When
MLN_LINUX_STATIC_DEPS=ON:cargo install armerge) — clearFATAL_ERRORif missingBreaking changes
None. The option defaults to
OFF, preserving existing behavior.