Skip to content

feat(linux): vendor static deps for portable shared libraries#2

Closed
jamesarich wants to merge 339 commits into
sargunv:mainfrom
jamesarich:fix/linux-static-deps-amalgamation
Closed

feat(linux): vendor static deps for portable shared libraries#2
jamesarich wants to merge 339 commits into
sargunv:mainfrom
jamesarich:fix/linux-static-deps-amalgamation

Conversation

@jamesarich
Copy link
Copy Markdown

Description

Add a new MLN_LINUX_STATIC_DEPS CMake option that vendors zlib, libpng, libjpeg-turbo, libwebp, libuv, and bzip2 as static libraries built from source. When enabled, these are amalgamated into mbgl-core via armerge, producing a self-contained shared library with no runtime dependency on distro-specific library versions.

Problem

Published Linux shared libraries (e.g. JNI .so for maplibre-compose) dynamically link against system libraries at build time. When users run on a different distro or version, symbol mismatches cause UnsatisfiedLinkError crashes — particularly ICU (version-mangled C++ symbols), libpng, and zlib. This is the blocker described in maplibre/maplibre-compose#627.

Approach

  • FetchContent for zlib 1.3.1, libpng 1.6.44, libwebp 1.5.0, libuv 1.50.0
  • FetchContent_Populate + manual build for bzip2 1.0.8 (no CMakeLists.txt)
  • ExternalProject for libjpeg-turbo 3.1.0 (blocks add_subdirectory())
  • System curl kept dynamic (stable ABI, complex dependency tree)
  • Builtin ICU forced on when vendoring (maplibre-native already has a vendored ICU)
  • -Wl,--exclude-libs,ALL hides vendored symbols from the final .so exports
  • FreeType PNG/BZip2 features disabled when vendoring to prevent system lib leakage

All vendored deps inherit CMAKE_POSITION_INDEPENDENT_CODE=ON from the root CMakeLists.txt. The existing system-deps path (MLN_LINUX_STATIC_DEPS=OFF, the default) is completely unchanged.

Files changed

  • CMakeLists.txt — Added option(MLN_LINUX_STATIC_DEPS) declaration
  • platform/linux/cmake/vendored_deps.cmakeNew: downloads, builds, and configures all vendored deps with license metadata
  • platform/linux/linux.cmake — Conditional paths for vendored vs system deps, armerge enforcement, amalgamation
  • vendor/freetype.cmake — Disable PNG/BZip2 when vendoring (gated behind CMAKE_SYSTEM_NAME STREQUAL "Linux" AND MLN_LINUX_STATIC_DEPS)
  • CHANGELOG.md — Added entry

Test plan

Linux (Ubuntu 25.04, GCC 15.2, CMake 4.2.3):

  • MLN_LINUX_STATIC_DEPS=ON: full build succeeds, .so has no dynamic deps on vendored libs (ldd clean), no leaked vendored symbols (nm -D clean)
  • MLN_LINUX_STATIC_DEPS=OFF: CMake configure succeeds, system-deps path unchanged
  • Built .so loads and renders map tiles in a Compose Desktop app

Cross-platform:

  • No changes to macOS/Windows/iOS/Android build paths (all gated behind MLN_LINUX_STATIC_DEPS)
  • vendor/freetype.cmake change additionally gated behind CMAKE_SYSTEM_NAME STREQUAL "Linux"

Requirements

When MLN_LINUX_STATIC_DEPS=ON:

  • armerge must be installed (cargo install armerge) — clear FATAL_ERROR if missing
  • Clang compiler recommended (consistent with existing Linux presets)

Breaking changes

None. The option defaults to OFF, preserving existing behavior.

dependabot Bot and others added 30 commits February 27, 2025 01:52
…ibre#3249)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Bart Louwers <bart@emeel.net>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
…-actions group (maplibre#3290)

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
adrian-cojocaru and others added 17 commits August 19, 2025 18:57
Co-authored-by: Michał Gwóźdź <114614618+michalgwo@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Add MLN_LINUX_STATIC_DEPS CMake option that builds zlib, libpng,
libjpeg-turbo, libwebp, libuv, and bzip2 from source with -fPIC,
then amalgamates them into the mbgl-core static archive.

This eliminates runtime dependencies on specific system library
versions (ICU, libpng, zlib, etc.) that cause UnsatisfiedLinkError
crashes when the shared library is loaded on a different distro or
version than it was built on. System curl, X11, OpenGL, libc, and
libstdc++ remain dynamic (stable ABI, universally available).

Fixes: maplibre/maplibre-compose#627

Changes:
- Add vendored_deps.cmake with FetchContent/ExternalProject builds
  for zlib 1.3.1, libpng 1.6.44, libjpeg-turbo 3.1.0, libwebp 1.5.0,
  libuv 1.50.0, and bzip2 1.0.8
- Add vendored amalgamation path in linux.cmake (symbols hidden via
  -Wl,--exclude-libs,ALL to prevent collisions)
- Gate freetype PNG/BZip2 system linking behind MLN_LINUX_STATIC_DEPS
- Force builtin ICU when vendoring (MLN_USE_BUILTIN_ICU=TRUE)
- Require armerge when MLN_LINUX_STATIC_DEPS is enabled
- Add INTERFACE_MAPLIBRE_* license metadata for all vendored deps

Also fixes (in the non-vendored path):
- Use ${ARMERGE} variable instead of bare 'armerge' command
- Fix ICU check: ICUUC_FOUND without ${} expansion
- Fix typo: 'requestd' -> 'requested'

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jamesarich
Copy link
Copy Markdown
Author

jamesarich commented Apr 16, 2026

Hi @sargunv - we (Meshtastic) are interested in potentially leveraging maplibre in our KMP projects. I'd like to help unblock Desktop JVM maps parity - let me know how we can assist!

@jamesarich
Copy link
Copy Markdown
Author

Closing — this should target maplibre/maplibre-native instead. Re-opening there.

@jamesarich jamesarich closed this Apr 16, 2026
@jamesarich jamesarich deleted the fix/linux-static-deps-amalgamation branch April 16, 2026 15:07
@sargunv
Copy link
Copy Markdown
Owner

sargunv commented Apr 16, 2026

I'd like to help unblock Desktop JVM maps parity - let me know how we can assist!

Hey, I'd love some help on this front. Are you on the OSMUS slack workspace?

The main issue for the desktop JVM parity is the native core integration -- I'm somewhat amateur at modern C++ and very amateur at C++ <> JVM integration and OpenGL/Vulkan/Metal, just figuring it all out as I go. If you have skills/experience there, that's would be extremely helpful.

Some immediate issues, off the top of my head:

  • I think some of the ways I'm doing memory management across the JVM / C++ boundary are unsound; would love a review or improvements there
  • Linux support right now is broken with a segfault (and my main workstation these days is Linux). I haven't tracked down this particular bug yet
  • Related to your recent PRs: would be nice to build MapLibre Compose on MapLibre Native's prebuilt binaries, so the dev ex is a bit easier
  • Most of the missing parity work is in style support. Implementing all the layer/source/image/etc adapters I think is mostly mechanical work and not directly blocked by any of the foundational stuff above (I expect recent frontier LLMs can probably do this part well enough)

(It's been a few months since I worked on desktop stuff though, so my memory might be wrong on some things)

But the long term goal is to make the MapLibre Compose <> native core integration stable and complete enough that it becomes the path for Android and iOS targets too, rather than the platform-specific SDKs we integrate with today.

@jamesarich
Copy link
Copy Markdown
Author

Are you on the OSMUS slack workspace

Nope, but you can find me on the Meshtastic Discord @olm3c

Most of the missing parity work is in style support. Implementing all the layer/source/image/etc adapters I think is mostly mechanical work and not directly blocked by any of the foundational stuff above (I expect recent frontier LLMs can probably do this part well enough)

I'll take a poke around, thanks for the response!

sargunv added a commit that referenced this pull request May 4, 2026
Add SKIA_DIVERGENCE.md capturing findings from a focused parallel
audit of the Skia backend against the OpenGL backend, covering
clipping, mesh geometry, and the 50-100x performance gap at high
pitch + high zoom. Each entry is concrete (file:line) so it can be
acted on individually.

Highest-leverage targets identified:
- Per-frame MeshVertex rebuild in Drawable::draw (maplibre#10) — single
  biggest perf win, no correctness implications
- Polyline-level near-plane clipping in the line bucket (#2+maplibre#7+maplibre#9)
  — fixes the slashes correctness bug at high pitch
- Consolidated UBO updates (maplibre#16)
- Lift image->makeShader into texture upload path (maplibre#15)
- Reuse per-tile clip paths per frame (maplibre#14)

Also restore the seattle-slashes _skia-repro test (deleted by the
earlier line-clip revert). Headless pitch=85 currently reproduces as
solid water rather than the live-path slashes, which is its own
mismatch worth tracking.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sargunv added a commit that referenced this pull request May 4, 2026
At extreme pitch, line-bucket triangles that span the near plane have one
vertex with clip-space w near zero. The perspective divide stretches it
across the canvas — the documented "slashes" artifact (SKIA_DIVERGENCE.md
items #2, maplibre#7, maplibre#9). The CPU near-plane clipper that fills already use
(`clipProjectedTriangles`) is attribute-unaware: it linearly interpolates
the entire MeshVertex, which corrupts packed line attrs (normal, blur,
progress).

This is the line-shader-local mitigation: in lineMeshSpecification,
lineGradientMeshSpecification, linePatternMeshSpecification, and
lineSDFMeshSpecification, clamp `inv_w` to 0 and push the projected ndc to
(-1e10, -1e10) when projected.w < 1e-4. Triangles with a behind-near vertex
are then trivially clipped by the GPU rasterizer.

Tradeoff: triangles partially behind near are entirely dropped instead of
clipped at the near plane. Visually that's close to GL's behavior at the
edge cases that triggered the bug — and a huge improvement over the
canvas-spanning smears. Render-test diff for line-pattern/pitch (a known
failure on Skia for unrelated sub-pixel AA) shows the smears gone:
expected and actual now match shape-for-shape; only sub-pixel placement
differs.

Render-test counts unchanged: 1265 passed, 38 failed, 3 errored.

The architecturally clean fix — polyline-level clipping in the line
bucket, shared with GL — is still pending (maplibre#28 stays open). This is a
Skia-local hack, the option (b) the divergence doc described.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.