From d36133a064a2ebba9562cd7485071e944aa174ec Mon Sep 17 00:00:00 2001 From: Sean McBride Date: Sat, 13 Jun 2026 17:27:12 -0400 Subject: [PATCH 1/2] fix: repair Docker dev-environment build and clarify native setup The documented setup was failing. The native path assumes a Debian host with LLVM 13, but aWsm's llvm-alt bindings call LLVM C-API functions (LLVMBuildCall, LLVMBuildLoad) removed in LLVM 15+, so the build fails to link against newer system LLVM (e.g. LLVM 21 on Ubuntu 24.04). The Docker route exists for exactly this reason but had bit-rotted in several places. Dockerfile.x86_64: - Install `hey` from focal's universe repo instead of downloading the prebuilt binary, whose hey-release S3/GCS URLs now return 403. - Add `git config --system --add safe.directory '*'` so the root-run `make install` doesn't abort with "dubious ownership" on the host-owned, bind-mounted /sledge. - Set RUSTUP_HOME/CARGO_HOME to the dev user's install so the root-run build can resolve the default Rust toolchain instead of failing with "no default is configured". Makefile: - Make the wasm_apps symlink rule idempotent (ln -srfn). The symlink is committed to the repo, so it already exists on a fresh clone and the old `ln -sr` failed with "File exists", breaking `make install`. README.md: - Recommend the Docker route and add an explicit caveat to the native section documenting the LLVM-13/Debian requirement and why Ubuntu 24.04 (and other non-LLVM-13 hosts) cannot use it. Verified end-to-end via `./devenv.sh setup`: aWsm, the runtime, and all benchmark apps build, and the runtime serves fibonacci, gocr, and resize_image requests with correct responses. Co-Authored-By: Claude Opus 4.8 (1M context) --- Dockerfile.x86_64 | 19 +++++++++++++------ Makefile | 7 +++++-- README.md | 28 ++++++++++++++++++---------- 3 files changed, 36 insertions(+), 18 deletions(-) diff --git a/Dockerfile.x86_64 b/Dockerfile.x86_64 index fb9163f92..7e97be7bf 100644 --- a/Dockerfile.x86_64 +++ b/Dockerfile.x86_64 @@ -5,7 +5,6 @@ ENV LLVM_VERSION=13 ENV WASI_SDK_VERSION=12 ARG DEBIAN_FRONTEND=noninteractive -ARG HEY_URL=https://hey-release.s3.us-east-2.amazonaws.com/hey_linux_amd64 ARG WASI_SDK_URL=https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-$WASI_SDK_VERSION/wasi-sdk_$WASI_SDK_VERSION.0_amd64.deb ARG SHFMT_URL=https://github.com/mvdan/sh/releases/download/v3.2.4/shfmt_v3.2.4_linux_amd64 ARG SHELLCHECK_URL=https://github.com/koalaman/shellcheck/releases/download/stable/shellcheck-stable.linux.x86_64.tar.xz @@ -32,6 +31,11 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ libtool \ pkg-config +# The repo is bind-mounted from the host, so its files are owned by the host user +# while the build runs as root (docker exec -u 0). Modern git rejects this as +# "dubious ownership", so whitelist any directory for all users in the image. +RUN git config --system --add safe.directory '*' + # Needed to install from http endpoints via curl or wget RUN apt-get update && apt-get install -y --no-install-recommends \ curl \ @@ -43,12 +47,15 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ wget # Test Script Stuff +# hey is a load generator used by the test workloads (focal ships it in universe; +# the previously-used hey-release S3/GCS binaries now return 403) RUN apt-get update && apt-get install -y --no-install-recommends \ bc \ fonts-dejavu \ fonts-cascadia-code \ fonts-roboto \ gnuplot \ + hey \ httpie \ imagemagick \ jq \ @@ -57,11 +64,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ pango1.0-tools \ wamerican -# Hey is a load generator we have to recklessly download from the 'net, as it is only published to brew -# Binaries are only provided for AMD64 though, so ARM will have to build from source -# See https://github.com/rakyll/hey -RUN wget $HEY_URL -O hey && chmod +x hey && mv hey /usr/bin/hey - # shfmt is a formatter for shell scripts RUN wget $SHFMT_URL -O shfmt && chmod +x shfmt && mv shfmt /usr/local/bin/shfmt RUN wget $SHELLCHECK_URL -O shellcheck && chmod +x shellcheck && mv shellcheck /usr/local/bin/shellcheck @@ -110,6 +112,11 @@ RUN cd sledge && ./fix_root.sh # Rustup does not cleanly support system installs, so install as non-root user RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain stable --component rustfmt --target wasm32-wasi -y ENV PATH=/home/dev/.cargo/bin:$PATH +# `make install` runs as root (docker exec -u 0), but Rust is installed for the +# dev user. Point rustup/cargo at dev's install so root resolves the default +# toolchain instead of failing with "no default is configured". +ENV RUSTUP_HOME=/home/dev/.rustup +ENV CARGO_HOME=/home/dev/.cargo RUN cargo install --debug cargo-audit cargo-watch rsign2 # We need to set the locale for pango-view diff --git a/Makefile b/Makefile index 929f95a86..9b782cb7a 100644 --- a/Makefile +++ b/Makefile @@ -50,9 +50,12 @@ applications: applications.clean: make -C applications clean -# Instead of having two copies of wasm_apps, just link to the awsm repo's copy +# Instead of having two copies of wasm_apps, just link to the awsm repo's copy. +# -f -n make this idempotent: the symlink is committed to the repo, so it already +# exists on a fresh clone and on rebuilds; without these flags ln fails with +# "File exists" and breaks `make install`. wasm_apps: - ln -sr awsm/applications/wasm_apps/ applications/ + ln -srfn awsm/applications/wasm_apps/ applications/wasm_apps # Tests .PHONY: test diff --git a/README.md b/README.md index ee6e99f93..b8ec4579f 100644 --- a/README.md +++ b/README.md @@ -4,18 +4,11 @@ ## Setting up a development environment -### Native on Debian Host +SLEdge's `aWsm` compiler is built against a **specific LLVM version (LLVM 13)** via the `llvm-alt` Rust bindings. Those bindings use LLVM C-API functions (e.g. `LLVMBuildCall`, `LLVMBuildLoad`) that were **removed in LLVM 15+**, so the build will fail to link against newer LLVM toolchains. -```sh -git clone https://github.com/gwsystems/sledge-serverless-framework.git -cd sledge-serverless-framework -./install_deb.sh -source ~/.bashrc -make install -make test -``` +**For this reason, the Docker environment below is the recommended way to build SLEdge.** It pins the exact Debian + LLVM 13 + WASI SDK toolchain that the compiler needs, and works on any host with Docker (macOS, Windows/WSL2, or a Linux distro other than the one targeted by `install_deb.sh`). -### Docker +### Docker (recommended) **Note: These steps require Docker. Make sure you've got it installed!** @@ -75,6 +68,21 @@ If you are finished working with the SLEdge runtime and wish to remove it, run t And then simply delete this repository. +### Native build (Debian + LLVM 13 only) + +**Caveat:** `install_deb.sh` only works on a host whose package repositories provide **LLVM 13** and `libtinfo5` — in practice a Debian release (or older Ubuntu such as 20.04/22.04) that `apt.llvm.org` still serves LLVM 13 for. It will **not** work on Ubuntu 24.04 (noble) or other distros where LLVM 13 is unavailable: `libtinfo5` has no install candidate, `apt.llvm.org/llvm.sh 13` provides no packages, and the `aWsm` build will fail to link against the newer system LLVM (`undefined symbol: LLVMBuildCall`). Use the Docker route above on those systems. + +On a supported host: + +```sh +git clone https://github.com/gwsystems/sledge-serverless-framework.git +cd sledge-serverless-framework +./install_deb.sh +source ~/.bashrc +make install +make test +``` + ## Running your first serverless function An SLEdge serverless function consists of a shared library (\*.so) and a JSON configuration file that determines how the runtime should execute the serverless function. As an example, here is the configuration file for our sample fibonacci function: From 8eccb03541f0e7edf4da5c3751f759cb6c6f97b8 Mon Sep 17 00:00:00 2001 From: Sean McBride Date: Sat, 20 Jun 2026 10:32:49 -0400 Subject: [PATCH 2/2] fix: support Ubuntu noble (24.04) for the Docker and native dev builds The dev toolchain is pinned to LLVM 13 (aWsm builds against an LLVM-13 binding fork), but apt.llvm.org never published llvm-toolchain-noble-13 and noble dropped libtinfo5, so a plain focal->noble bump failed to build. - install_llvm.sh: on noble, pin the focal apt.llvm.org repo for LLVM 13 and install the focal-era libtinfo5/libffi7 the LLVM 13 packages need; skip lldb-13 (needs the absent libpython3.8). Other releases keep the upstream llvm.sh path. - Dockerfile.x86_64: base on ubuntu:noble; drop libtinfo5 from apt (handled by install_llvm.sh); reuse noble's stock ubuntu user at UID/GID 1000 instead of creating a colliding dev user. - install_deb.sh: drop libtinfo5 from apt (handled by install_llvm.sh). - README: document the validated native build on Ubuntu incl. 24.04 noble, with caveats (update-alternatives repoints default clang; focal-pinned packages). Verified end-to-end on Ubuntu 24.04.4 (noble), x86_64, both in Docker and natively: aWsm, libsledge, the runtime, and all nine sample *.wasm.so modules build and sledgert runs. Co-Authored-By: Claude Opus 4.8 (1M context) --- Dockerfile.x86_64 | 25 ++++++++++---------- README.md | 16 +++++++++---- install_deb.sh | 4 +++- install_llvm.sh | 60 ++++++++++++++++++++++++++++++++++++++--------- 4 files changed, 77 insertions(+), 28 deletions(-) diff --git a/Dockerfile.x86_64 b/Dockerfile.x86_64 index 7e97be7bf..4e47d003d 100644 --- a/Dockerfile.x86_64 +++ b/Dockerfile.x86_64 @@ -1,5 +1,5 @@ -# using ubuntu 20 docker image -FROM ubuntu:focal +# using ubuntu 24.04 (noble) docker image +FROM ubuntu:noble ENV LLVM_VERSION=13 ENV WASI_SDK_VERSION=12 @@ -27,7 +27,6 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ binutils-dev \ cmake \ git \ - libtinfo5 \ libtool \ pkg-config @@ -47,7 +46,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \ wget # Test Script Stuff -# hey is a load generator used by the test workloads (focal ships it in universe; +# hey is a load generator used by the test workloads (noble ships it in universe; # the previously-used hey-release S3/GCS binaries now return 403) RUN apt-get update && apt-get install -y --no-install-recommends \ bc \ @@ -86,12 +85,14 @@ RUN ./sledge/install_llvm.sh $LLVM_VERSION RUN curl -sS -L -O $WASI_SDK_URL && dpkg -i wasi-sdk_$WASI_SDK_VERSION.0_amd64.deb && rm -f wasi-sdk_$WASI_SDK_VERSION.0_amd64.deb ENV WASI_SDK_PATH=/opt/wasi-sdk -# Create non-root user and add to sudoers -ARG USERNAME=dev +# Non-root user and sudoers. +# Ubuntu noble (24.04) already ships a default non-root login user "ubuntu" at +# UID/GID 1000 (real shell + /home/ubuntu, in the sudo group) intended to be the +# account you work as, so reuse it rather than creating a second user at 1000 +# (which would collide on UID 1000 and fail). +ARG USERNAME=ubuntu ARG USER_UID=1000 ARG USER_GID=$USER_UID -RUN groupadd --gid $USER_GID $USERNAME -RUN useradd --uid $USER_UID --gid $USER_GID -m $USERNAME RUN apt-get update && apt-get install -y sudo RUN echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME RUN chmod 0440 /etc/sudoers.d/$USERNAME @@ -111,12 +112,12 @@ RUN cd sledge && ./fix_root.sh # Rust # Rustup does not cleanly support system installs, so install as non-root user RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- --default-toolchain stable --component rustfmt --target wasm32-wasi -y -ENV PATH=/home/dev/.cargo/bin:$PATH +ENV PATH=/home/ubuntu/.cargo/bin:$PATH # `make install` runs as root (docker exec -u 0), but Rust is installed for the -# dev user. Point rustup/cargo at dev's install so root resolves the default +# ubuntu user. Point rustup/cargo at ubuntu's install so root resolves the default # toolchain instead of failing with "no default is configured". -ENV RUSTUP_HOME=/home/dev/.rustup -ENV CARGO_HOME=/home/dev/.cargo +ENV RUSTUP_HOME=/home/ubuntu/.rustup +ENV CARGO_HOME=/home/ubuntu/.cargo RUN cargo install --debug cargo-audit cargo-watch rsign2 # We need to set the locale for pango-view diff --git a/README.md b/README.md index b8ec4579f..8c0c8865d 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ SLEdge's `aWsm` compiler is built against a **specific LLVM version (LLVM 13)** via the `llvm-alt` Rust bindings. Those bindings use LLVM C-API functions (e.g. `LLVMBuildCall`, `LLVMBuildLoad`) that were **removed in LLVM 15+**, so the build will fail to link against newer LLVM toolchains. -**For this reason, the Docker environment below is the recommended way to build SLEdge.** It pins the exact Debian + LLVM 13 + WASI SDK toolchain that the compiler needs, and works on any host with Docker (macOS, Windows/WSL2, or a Linux distro other than the one targeted by `install_deb.sh`). +**For this reason, the Docker environment below is the recommended way to build SLEdge.** It pins the exact Ubuntu + LLVM 13 + WASI SDK toolchain that the compiler needs, and works on any host with Docker (macOS, Windows/WSL2, or any Linux distribution). To build directly on a Debian or Ubuntu host instead — including Ubuntu 24.04 (noble) — see [Native build](#native-build-linux-debian-or-ubuntu-including-2404-noble) below. ### Docker (recommended) @@ -68,11 +68,11 @@ If you are finished working with the SLEdge runtime and wish to remove it, run t And then simply delete this repository. -### Native build (Debian + LLVM 13 only) +### Native build (Linux: Debian or Ubuntu, including 24.04 noble) -**Caveat:** `install_deb.sh` only works on a host whose package repositories provide **LLVM 13** and `libtinfo5` — in practice a Debian release (or older Ubuntu such as 20.04/22.04) that `apt.llvm.org` still serves LLVM 13 for. It will **not** work on Ubuntu 24.04 (noble) or other distros where LLVM 13 is unavailable: `libtinfo5` has no install candidate, `apt.llvm.org/llvm.sh 13` provides no packages, and the `aWsm` build will fail to link against the newer system LLVM (`undefined symbol: LLVMBuildCall`). Use the Docker route above on those systems. +`install_deb.sh` installs the full toolchain directly on the host — the apt dependencies, **LLVM 13**, the WASI SDK, and a Rust toolchain — after which `make install` builds the runtime and the sample functions. -On a supported host: +On **Ubuntu 24.04 (noble)** neither the distro nor `apt.llvm.org/noble` provides LLVM 13, so `install_llvm.sh` automatically pins the **focal** `apt.llvm.org` repository for LLVM 13 and installs the two focal-era runtime libraries that noble dropped (`libtinfo5`, `libffi7`). This path is validated on Ubuntu 24.04.4 (noble), x86_64: `aWsm`, `libsledge`, the runtime, and all nine sample `*.wasm.so` modules compile, and the resulting `sledgert` binary runs. ```sh git clone https://github.com/gwsystems/sledge-serverless-framework.git @@ -83,6 +83,14 @@ make install make test ``` +**Caveats:** + +- `install_deb.sh` requires `sudo` and uses `update-alternatives` to point the default `clang`, `wasm-ld`, and `llvm-config` at version 13. On a host that already runs a newer LLVM as its default (e.g. noble's system clang), this changes that default system-wide. +- On noble the toolchain is pinned to focal-era LLVM 13 packages, which are on an older support track; the `apt.llvm.org/focal` and `archive.ubuntu.com` `.deb` URLs it relies on may eventually move or disappear. +- The sample functions are C, compiled with the WASI SDK's clang, so a Rust `wasm32-wasi`/`wasm32-wasip1` target is not required to build them. + +On a distribution where neither the native repositories nor the focal fallback provide LLVM 13, use the Docker route above instead. + ## Running your first serverless function An SLEdge serverless function consists of a shared library (\*.so) and a JSON configuration file that determines how the runtime should execute the serverless function. As an example, here is the configuration file for our sample fibonacci function: diff --git a/install_deb.sh b/install_deb.sh index b83e1e921..f9c23dad2 100755 --- a/install_deb.sh +++ b/install_deb.sh @@ -45,7 +45,6 @@ sudo apt-get update && sudo apt-get install -y --no-install-recommends \ jq \ less \ libssl-dev \ - libtinfo5 \ libtool \ libz3-4 \ lsb-release \ @@ -64,6 +63,9 @@ sudo apt-get update && sudo apt-get install -y --no-install-recommends \ wget $SHFMT_URL -O shfmt && chmod +x shfmt && sudo mv shfmt /usr/local/bin/shfmt +# Installs LLVM $LLVM_VERSION. On Ubuntu noble this pins the focal apt.llvm.org +# repo and pulls the focal-era libtinfo5/libffi7 the LLVM 13 packages need (noble +# dropped libtinfo5, which is why it is no longer in the apt list above). sudo ./install_llvm.sh $LLVM_VERSION curl -sS -L -O $WASI_SDK_URL && sudo dpkg -i wasi-sdk_$WASI_SDK_VERSION.0_amd64.deb && rm -f wasi-sdk_$WASI_SDK_VERSION.0_amd64.deb diff --git a/install_llvm.sh b/install_llvm.sh index 515385738..2af2e5677 100755 --- a/install_llvm.sh +++ b/install_llvm.sh @@ -1,12 +1,48 @@ #!/bin/bash -# Installs LLVM tooling, delegating the to the LLVM script as much as possible +# Installs LLVM tooling for the SLEdge x86_64 dev image. +# +# On most releases this just delegates to the upstream apt.llvm.org installer. +# On Ubuntu noble (24.04) that doesn't work: apt.llvm.org never published an +# llvm-toolchain-noble-13 repo (only 18/19+), and SLEdge is pinned to LLVM 13 +# (awsm builds against an LLVM-13 binding fork). So on noble we pin the *focal* +# apt.llvm.org repo instead, plus the two focal-era runtime libs noble dropped +# (libtinfo5, libffi7) that the focal LLVM packages depend on. +set -e LLVM_VERSION=$1 echo "Installing LLVM $LLVM_VERSION" -# Script Installs clang, lldb, lld, and clangd -curl --proto '=https' --tlsv1.2 -sSf https://apt.llvm.org/llvm.sh | bash -s -- "$LLVM_VERSION" +CODENAME="$(. /etc/os-release && echo "$VERSION_CODENAME")" + +if [ "$CODENAME" = "noble" ]; then + echo "Detected Ubuntu noble: pinning the focal apt.llvm.org repo for LLVM $LLVM_VERSION" + + # focal-era runtime libs no longer shipped by noble, required by focal LLVM debs + cd /tmp + curl -fsSL -o libtinfo5.deb http://archive.ubuntu.com/ubuntu/pool/universe/n/ncurses/libtinfo5_6.2-0ubuntu2.1_amd64.deb + curl -fsSL -o libffi7.deb http://archive.ubuntu.com/ubuntu/pool/main/libf/libffi/libffi7_3.3-4_amd64.deb + dpkg -i libtinfo5.deb libffi7.deb + rm -f libtinfo5.deb libffi7.deb + + wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key | gpg --dearmor -o /usr/share/keyrings/llvm.gpg + echo "deb [signed-by=/usr/share/keyrings/llvm.gpg] http://apt.llvm.org/focal/ llvm-toolchain-focal-$LLVM_VERSION main" \ + > /etc/apt/sources.list.d/llvm.list + apt-get update + + # Note: lldb-$LLVM_VERSION is intentionally omitted. The focal lldb package + # depends on libpython3.8, which noble no longer provides, and the debugger is + # not needed to build SLEdge. + apt-get install -y --no-install-recommends \ + "clang-$LLVM_VERSION" \ + "lld-$LLVM_VERSION" \ + "clangd-$LLVM_VERSION" \ + "llvm-$LLVM_VERSION" \ + "llvm-$LLVM_VERSION-dev" +else + # Upstream installer: installs clang, lldb, lld, and clangd + curl --proto '=https' --tlsv1.2 -sSf https://apt.llvm.org/llvm.sh | bash -s -- "$LLVM_VERSION" +fi # Installing "libc++-xx-dev" automagically installs "libc++1-xx", "libunwind-xx" and "libunwind-xx-dev" apt-get install -y --no-install-recommends \ @@ -16,14 +52,16 @@ apt-get install -y --no-install-recommends \ "clang-tidy-$LLVM_VERSION" \ "clang-format-$LLVM_VERSION" -update-alternatives --remove-all wasm-ld -update-alternatives --remove-all llvm-config -update-alternatives --remove-all llvm-objdump -update-alternatives --remove-all llvm-dis -update-alternatives --remove-all clang-format -update-alternatives --remove-all clang -update-alternatives --remove-all clang++ -update-alternatives --remove-all clang-tidy +# --remove-all clears any prior registration; harmless if none exists (so do not +# let "no alternatives" abort the script under set -e). +update-alternatives --remove-all wasm-ld 2> /dev/null || true +update-alternatives --remove-all llvm-config 2> /dev/null || true +update-alternatives --remove-all llvm-objdump 2> /dev/null || true +update-alternatives --remove-all llvm-dis 2> /dev/null || true +update-alternatives --remove-all clang-format 2> /dev/null || true +update-alternatives --remove-all clang 2> /dev/null || true +update-alternatives --remove-all clang++ 2> /dev/null || true +update-alternatives --remove-all clang-tidy 2> /dev/null || true update-alternatives --install /usr/bin/wasm-ld wasm-ld "/usr/bin/wasm-ld-$LLVM_VERSION" 100 update-alternatives --install /usr/bin/llvm-config llvm-config "/usr/bin/llvm-config-$LLVM_VERSION" 100