Skip to content

build(release): vendor openssl in gravity_node so -p builds stay self-contained#717

Merged
nekomoto911 merged 7 commits into
mainfrom
build/vendor-openssl-gravity-node
May 15, 2026
Merged

build(release): vendor openssl in gravity_node so -p builds stay self-contained#717
nekomoto911 merged 7 commits into
mainfrom
build/vendor-openssl-gravity-node

Conversation

@keanji-x

@keanji-x keanji-x commented May 15, 2026

Copy link
Copy Markdown
Contributor

Summary

  • cargo build -p gravity_node (e.g. via make gravity_node) was producing a binary with dynamic libssl.so.3 + libcrypto.so.3 deps, which fail to run on hosts whose libssl ABI differs from the build host (Debian 11 vs 12, libssl1.1 vs libssl3, minor-version skew).
  • Root cause: openssl-sys is pulled transitively via aptos-vault-client → native-tls, and resolver-v2 only sees the vendored feature when gravity_cli (which declares openssl = { vendored }) is part of the same cargo resolve. Scoping with -p gravity_node excludes it, so openssl-sys falls back to dynamic system linkage.
  • The CI release workflow happens to build both binaries in one invocation (cargo build --bin gravity_node --bin gravity_cli) so the official artifact is fine, but make gravity_node and any one-off -p gravity_node build are brittle.
  • Fix: declare openssl = { version = "0.10", features = ["vendored"] } directly on bin/gravity_node/Cargo.toml, mirroring bin/gravity_cli/Cargo.toml. openssl-sys now always builds OpenSSL from source and statically links it regardless of cargo invocation.

Verification

Build invocation Before After
cargo build --bin gravity_node --bin gravity_cli (CI release) no libssl dep no libssl dep
cargo build --bin gravity_node (no -p) no libssl dep no libssl dep
cargo build -p gravity_node (make gravity_node) libssl.so.3 dynamic no libssl dep ✅

Reproduced from-scratch in rust:1.93-bookworm with apt-get remove libssl-dev libcurl4-openssl-dev first (no ssl.h in /usr/include). cargo build --profile quick-release -p gravity_node finished in 3m39s, and the resulting binary runs cleanly on debian:bookworm-slim (no libssl present) — confirms static OpenSSL 3.4.1 baked in (strings, nm, objdump -p all agree).

End-to-end node startup verified inside both debian:bookworm-slim and ubuntu:24.04 with no openssl packages installed: full reth init, RocksDB open, P2P networking up, JWT secret generated, consensus engine started, RPC HTTP/auth/IPC servers listening, plus successful outgoing HTTPS to telemetry.aptoslabs.com and sepolia.drpc.org (rustls + the vendored OpenSSL inside the binary).

After this PR, gravity_node only DT_NEEDEDs libc.so.6 / ld-linux-x86-64.so.2 / libstdc++.so.6 / libgcc_s.so.1 / libm.so.6 — all of which ship in any glibc-based base image (Debian 12 / Ubuntu 22.04+).

Follow-on commits in this PR

build(docker): drop libssl-dev / libssl3 / libudev{,1} / libzstd{,1}, add perl for vendored OpenSSL

Since gravity_node no longer DT_NEEDEDs libssl3/libudev1/libzstd1, the explicit apt installs of those packages in the runtime stage of docker/gravity_node/Dockerfile{,.host-binary} are cleaned up. Builder stage drops libssl-dev (replaced by vendored OpenSSL self-compile) and adds perl (needed by OpenSSL's Configure script).

Honesty caveatlibssl3 / libudev1 / libzstd1 are still in the runtime image as transitive dependencies of ca-certificates → openssl, dpkg / libsystemd0 / libapt-pkg6.0, and util-linux. They're structural, not removable without dropping ca-certificates (which we need for rustls cert-chain validation against external HTTPS endpoints — telemetry, drpc, GCP secret manager, etc.). This commit is about not declaring deps we don't actually consume — image content is essentially unchanged for the runtime stage.

The builder-stage libssl-dev → perl swap is a real cleanup (~few MB of dev headers gone).

build(docker): drop curl from runtime image

curl was only present for operator-side ad-hoc RPC poking (per the README); nothing in the runtime path uses it. Removing it does shrink the image — libcurl4 / libgnutls30 / libkrb5-3 / libssh2-1 / libldap-2.5-0 all come along with curl and now go away.

build(docker): switch runtime base from debian:12-slim to ubuntu:24.04

Both Dockerfile leaves switch their runtime stage from debian:12-slim to ubuntu:24.04 — longer support window (LTS to 2029 + ESM to 2034 vs Debian 12 to 2026-06 + ELTS to 2031), more glibc headroom (2.39 vs 2.36), comparable image size. Builder stage stays on rust:1.93-slim-bookworm (no upstream Ubuntu-based rust:1.93 image).

build(docker): consolidate Dockerfile.host-binary into Dockerfile as runtime-host-binary target

Restructured the main Dockerfile around two leaf targets sharing a runtime-base stage:

  • runtime (default) — COPY --from=builder .... Runs cargo build inside the container. Behaviour unchanged for plain docker build .; backwards compatible.
  • runtime-host-binaryCOPY ${HOST_BINARY} ... from the host build context. BuildKit's stage pruning means the builder stage is genuinely skipped — no cargo runs, ~12s build vs 20-30 min from-source. If the host binary is missing the build fails fast at the COPY step.

Dockerfile.host-binary is deleted; the 4 in-repo callers — regression/pfn_chain_stress/run.sh, the matching docker-compose, and the two regression READMEs — are updated to point at the unified Dockerfile with --target runtime-host-binary.

ci(release): build + push gravity_node docker image to ghcr.io

After the native cargo build that produces the GitHub-Release binaries, the workflow now also builds a docker image via --target runtime-host-binary, then (on tag pushes only) pushes it to ghcr.io/<owner-lowercased>/gravity_node:<tag>. Stage pruning means no recompile after the native cargo step — the binary the runner just produced is the same one that ends up in the image. PR dry-runs build the image but don't push it.

  • Workflow-level permissions: packages: write added so the default GITHUB_TOKEN can push to GHCR.
  • Owner namespace is lowercased to satisfy GHCR's naming rules (works for any org case).
  • Runner deps doc-comment updated: libssl-dev removed (no longer needed thanks to the vendored OpenSSL change), docker added.

Test plan

All verified locally:

  • cargo build -p gravity_node produces a binary with no libssl / libcrypto in ldd
  • cargo build --bin gravity_node --bin gravity_cli (release CI command) still works, both binaries free of libssl deps
  • cargo build -p gravity_node --features randomness_disabled resolves and builds
  • Built gravity_node from-scratch in a rust:1.93-bookworm container with libssl-dev removed and zero ssl.h in /usr/include — succeeded
  • docker build -f docker/gravity_node/Dockerfile . (default runtime target, full multi-stage cargo build inside the container) — succeeded, 203MB image, boots cleanly with vendored-OpenSSL outbound HTTPS
  • docker build --target runtime-host-binary --build-arg HOST_BINARY=... -f docker/gravity_node/Dockerfile . — succeeded in ~12s, 205MB image, boots cleanly, BuildKit confirms builder stage was pruned (no cargo build ran)
  • Booted gravity_node node --dev end-to-end inside both debian:bookworm-slim and ubuntu:24.04 runtime images with no openssl packages installed
  • Confirmed curl is not referenced by entrypoint.sh, docker-compose*.yaml, or any runtime path in this repo
  • Verified tini lives at /usr/bin/tini and nologin at /usr/sbin/nologin on ubuntu:24.04

Will be exercised by CI on this PR (release.yml dry-run runs on PRs that touch the file, including this one):

  • Self-hosted runner has docker + BuildKit available
  • --target runtime-host-binary docker build step works on the runner's docker version
  • (Tag-push only, not exercised by PR dry-run) GHCR login with GITHUB_TOKEN succeeds and docker push to ghcr.io/<owner>/gravity_node:<tag> works

Genuinely external — reviewer needs to confirm:

  • No internal operator scripts or runbooks outside this repo rely on curl being inside the container
  • No internal infra (kubernetes manifests, deploy scripts, base-image scanners) outside this repo assumes the container is Debian-derived
  • No external script references the deleted docker/gravity_node/Dockerfile.host-binary (the 4 in-repo references are updated in this PR)
  • ghcr.io/<owner>/gravity_node is the right image name (the workflow uses ${GITHUB_REPOSITORY_OWNER} lowercased + gravity_node; if you want a different namespace, this needs adjusting)

🤖 Generated with Claude Code

keanji-x and others added 7 commits May 15, 2026 13:00
…-contained

`cargo build -p gravity_node` (e.g. via `make gravity_node`) scopes the
feature resolve to gravity_node only — without gravity_cli in the same
invocation, openssl-sys defaults to dynamic system linkage and the
resulting binary depends on libssl.so.3 / libcrypto.so.3, breaking
portability across hosts with mismatched libssl ABI (Debian 11 vs 12,
minor-version skew, etc).

Add openssl = { vendored } as a direct dep on gravity_node, mirroring
gravity_cli, so any cargo invocation produces a self-contained static
binary. Verified by building in a debian:bookworm-slim container with
libssl-dev removed (no ssl.h in /usr/include) — build succeeded and
the resulting binary runs cleanly on debian:bookworm-slim with no
libssl present.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… add perl for vendored OpenSSL

gravity_node no longer DT_NEEDEDs libssl.so.3 / libcrypto.so.3 / libudev /
libzstd at runtime, so the runtime stages of both Dockerfiles can drop the
explicit `libssl3 libudev1 libzstd1` apt installs. They will still come
along transitively via curl / ca-certificates / dpkg / libsystemd0, so
the image content is essentially unchanged — this is a "no longer
declaring deps we don't actually consume" cleanup, not a size win.

The builder stage of `Dockerfile` drops `libssl-dev` (vendored OpenSSL
compiles its own copy from source) and adds `perl`, which is what the
OpenSSL `Configure` script needs. Verified by building the binary in a
`rust:1.93-bookworm` container with libssl-dev explicitly removed before
the cargo build — produces a static-OpenSSL binary identical to the
host build.

End-to-end verified by booting gravity_node (built without libssl-dev)
from the trimmed `Dockerfile.host-binary` image: full reth + consensus
engine startup + outgoing HTTPS to telemetry.aptoslabs.com all work,
using the vendored OpenSSL inside the binary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
curl is not used by entrypoint.sh, docker-compose healthchecks, or any
runtime path — it was only present as an operator-debug convenience
documented in README.md. Removing it saves the libcurl4 / libgnutls30 /
libkrb5-3 / libssh2-1 / libldap-2.5-0 transitive set: ~5MB off the
final image (335MB → 330MB on the host-binary variant).

Operators who curl into the container for ad-hoc RPC pokes can either
`docker exec` from a curl-equipped host or install on demand. The README
debug example still applies — just run it from the host pointing at the
container's host-networked port.

Verified end-to-end: built the host-binary image without curl, booted
gravity_node node1 dev mode, and watched full reth + consensus engine
startup plus successful outgoing HTTPS to telemetry.aptoslabs.com and
sepolia.drpc.org from inside the container.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Longer support window (Ubuntu 24.04 LTS until 2029, +ESM to 2034 vs
Debian 12 standard support to 2026-06), newer glibc (2.39 vs 2.36 →
~4 minor versions of headroom for future binary symbol upgrades), and
fewer surprises for users on Ubuntu infra.

Net image size: 330MB → 327MB (host-binary variant). Ubuntu's
ca-certificates + jq + tini installs end up slightly leaner than
Debian's despite the base image being ~3MB larger; the runtime stage
trims the difference back.

ca-certificates / jq / tini all present in Ubuntu repos under the same
names. /usr/bin/tini and /usr/sbin/nologin paths match. Builder stage
left on rust:1.93-slim-bookworm (Debian) — no upstream Ubuntu-based
official rust image at the pinned 1.93 version, and the resulting
binary works on either runtime base.

Verified end-to-end: built the host-binary image on ubuntu:24.04,
booted gravity_node node1 dev mode, watched full reth + P2P +
consensus engine + RPC servers start, plus outgoing HTTPS to
telemetry.aptoslabs.com and sepolia.drpc.org succeed via the
vendored OpenSSL.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…runtime-host-binary target

The release pipeline already produces a native gravity_node binary on
the runner before any docker build. Compiling a second time inside
docker/gravity_node/Dockerfile's builder stage just to copy the result
into a runtime image is pure waste — 20-30 min on the slow path, plus
all the in-container cargo-git SSL flakiness regression/pfn_chain_stress
already had to work around with a separate Dockerfile.host-binary.

Restructure the main Dockerfile around two leaf targets sharing a
runtime-base stage:

  - `runtime` (default): COPY from the in-container `builder` stage.
    Same behaviour as before for plain `docker build .` — backwards
    compatible.
  - `runtime-host-binary`: COPY from a host build-context path supplied
    via --build-arg HOST_BINARY. BuildKit's stage pruning means the
    `builder` stage is genuinely skipped here, not just cached.

Usage in the release pipeline (no double compile):

    cargo build --profile quick-release --bin gravity_node --bin gravity_cli
    DOCKER_BUILDKIT=1 docker build \
        --target runtime-host-binary \
        --build-arg HOST_BINARY=target/quick-release/gravity_node \
        -t gravity_node:$TAG \
        -f docker/gravity_node/Dockerfile .

Delete Dockerfile.host-binary; update the 4 in-repo callers
(regression/pfn_chain_stress/run.sh, the corresponding docker-compose,
and the two regression READMEs) to use `--target runtime-host-binary`
against the unified Dockerfile.

Verified end-to-end:
  - `docker build --target runtime-host-binary ...` → 205MB image,
    node boots cleanly, HTTPS to telemetry/drpc works.
  - `docker build ...` (default `runtime` target, full cargo build
    inside the container) → 203MB image, same end-to-end behaviour.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the native cargo build that produces the GitHub-Release binaries,
add a step that wraps the just-built target/quick-release/gravity_node
into a docker image using docker/gravity_node/Dockerfile's
`runtime-host-binary` target.

BuildKit stage pruning means the `builder` stage is skipped entirely
— no recompile after the native cargo step. Build runs on both PR
dry-runs (validation only, no push) and tag pushes (followed by
docker login + push to ghcr.io/<owner>/gravity_node:<tag>).

Image is pushed to ghcr.io using the workflow's GITHUB_TOKEN (now with
`packages: write` permission added at the workflow level). Owner name
is lowercased for GHCR namespace rules.

Runner deps doc-comment updated: libssl-dev removed from required
build deps (the vendored OpenSSL change earlier in this PR makes it
no longer needed), and docker added.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…uild

The repo's .dockerignore excludes `target/` from the docker build
context (standard Rust pattern — `target/` is too big to ship into
every docker build). The new docker image step was passing
`HOST_BINARY=target/quick-release/gravity_node` directly, which made
the COPY fail at runtime:

  ERROR: "/target/quick-release/gravity_node" not found: not found

Fix: copy the cargo-built binary into docker/gravity_node/bin/ first
(a path that IS in the docker context — used by regression/pfn_chain_stress
for the same reason) and point HOST_BINARY at the staged copy. Clean
up the staged file after the build so the runner's working dir stays
tidy across jobs.

Verified locally: re-ran the exact step verbatim, docker build
succeeded in <1s (BuildKit reused all cached layers), image is 203MB,
boots cleanly with vendored-OpenSSL outbound HTTPS.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@nekomoto911 nekomoto911 merged commit 359bc9e into main May 15, 2026
10 checks passed
@nekomoto911 nekomoto911 deleted the build/vendor-openssl-gravity-node branch May 15, 2026 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants