Skip to content

Commit a8e60cd

Browse files
author
Roy Lin
committed
docs(k8s): one-click RuntimeClass node installer + README
deploy/scripts/install-runtimeclass.sh provisions a node to run RuntimeClass=a3s-box pods: installs the a3s-box CLI + helpers + libkrun and the containerd runtime-v2 shim (containerd-shim-a3s-box-v2), registers the io.containerd.a3s-box.v2 runtime via an /etc/containerd/conf.d drop-in, restarts containerd, and warms the one-time per-node boot cache so the first pod boots fast. Download (default) and --from-dir (air-gapped) modes; idempotent. README: document the deploy flow — create the RuntimeClass, run the installer on each node, label the node (a3s-box.io/runtime=true), run a pod. Validated end-to-end on a 5-node cluster: installer ran on every worker, each then started an a3s-box pod and served `kubectl exec` on the first try. Note: v2.6.0's release tarball ships the CLI/libkrun but not the containerd shim (it is a separate cargo project release.yml does not build); the shim is published as the release asset containerd-shim-a3s-box-v2-linux-x86_64. Follow-up: have release.yml build + attach it so future releases are self-contained.
1 parent adb8116 commit a8e60cd

2 files changed

Lines changed: 232 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -347,6 +347,79 @@ helm install a3s-box deploy/helm/a3s-box/ -n a3s-box-system --create-namespace
347347

348348
Windows CRI is intentionally unsupported.
349349

350+
## Deploy as a Kubernetes RuntimeClass
351+
352+
Run selected pods as a3s-box MicroVMs by setting `runtimeClassName: a3s-box`. Each
353+
pod's containers become libkrun MicroVMs under containerd, and `kubectl exec` works
354+
against them. This is opt-in per node — a node must have the runtime installed **and**
355+
carry the label `a3s-box.io/runtime=true` before a3s-box pods schedule there.
356+
357+
**1. Create the RuntimeClass (once per cluster):**
358+
359+
```bash
360+
kubectl apply -f - <<'EOF'
361+
apiVersion: node.k8s.io/v1
362+
kind: RuntimeClass
363+
metadata:
364+
name: a3s-box
365+
handler: a3s-box
366+
scheduling:
367+
nodeSelector:
368+
a3s-box.io/runtime: "true" # only labeled nodes run a3s-box pods
369+
EOF
370+
```
371+
372+
**2. Provision each node** — run the installer as root on every node that should
373+
host a3s-box workloads. It installs the a3s-box CLI + helpers, libkrun, and the
374+
containerd runtime-v2 shim (`containerd-shim-a3s-box-v2`), registers the
375+
`io.containerd.a3s-box.v2` runtime via an `/etc/containerd/conf.d` drop-in, restarts
376+
containerd, and warms the one-time per-node boot cache. Requires containerd ≥ 2.0
377+
and `/dev/kvm`.
378+
379+
```bash
380+
# one-liner (downloads the pinned release artifacts from GitHub):
381+
curl -fsSL https://raw.githubusercontent.com/AI45Lab/Box/main/deploy/scripts/install-runtimeclass.sh | sudo bash
382+
383+
# or from a checkout:
384+
sudo deploy/scripts/install-runtimeclass.sh # default version
385+
sudo deploy/scripts/install-runtimeclass.sh --version v2.6.0 # pin a version
386+
```
387+
388+
Then label the node from a machine with `kubectl`:
389+
390+
```bash
391+
kubectl label node <node-name> a3s-box.io/runtime=true
392+
```
393+
394+
Notes:
395+
- **Control-plane nodes** carry a `NoSchedule` taint and are normally excluded —
396+
leave them unlabeled unless you intentionally run workloads there.
397+
- The installer warms up with `busybox:latest` so the *first* pod boots fast (the
398+
first box on a fresh node builds a one-time cache that can exceed the shim's boot
399+
window). Use `--warmup-image <ref>` to point at a mirror, or `--no-warmup` to skip.
400+
- **Air-gapped:** pre-stage the release tarball
401+
(`a3s-box-<ver>-linux-<arch>.tar.gz`) and `containerd-shim-a3s-box-v2-linux-<arch>`
402+
in a directory and pass `--from-dir <dir>` (no network needed).
403+
404+
**3. Run a pod:**
405+
406+
```bash
407+
kubectl apply -f - <<'EOF'
408+
apiVersion: v1
409+
kind: Pod
410+
metadata:
411+
name: hello-a3s-box
412+
spec:
413+
runtimeClassName: a3s-box
414+
containers:
415+
- name: app
416+
image: busybox:latest
417+
command: ["sleep", "3600"]
418+
EOF
419+
420+
kubectl exec hello-a3s-box -- sh -c 'echo "hello from $(hostname)"; uname -m'
421+
```
422+
350423
## Architecture
351424

352425
```text

‎deploy/scripts/install-runtimeclass.sh‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
#!/usr/bin/env bash
2+
#
3+
# install-runtimeclass.sh — provision a Kubernetes node to run RuntimeClass=a3s-box pods.
4+
#
5+
# Run as root ON each node that should host a3s-box MicroVM workloads. It installs:
6+
# * the a3s-box CLI + helpers (a3s-box, a3s-box-cri, a3s-box-guest-init, a3s-box-shim)
7+
# * libkrun / libkrunfw (the MicroVM VMM, into the system lib dir + ldconfig)
8+
# * the containerd runtime-v2 shim (containerd-shim-a3s-box-v2)
9+
# and registers the `io.containerd.a3s-box.v2` runtime with containerd, then restarts it.
10+
#
11+
# After running this on a node, label it from a control-plane so the RuntimeClass
12+
# nodeSelector (a3s-box.io/runtime=true) lets a3s-box pods schedule there:
13+
#
14+
# kubectl label node <node-name> a3s-box.io/runtime=true
15+
#
16+
# Usage:
17+
# install-runtimeclass.sh [--version vX.Y.Z] [--repo OWNER/REPO] [--from-dir DIR]
18+
#
19+
# --version release tag to install (default: v2.6.0)
20+
# --repo GitHub repo to download artifacts from (default: AI45Lab/Box)
21+
# --from-dir install from a local directory instead of downloading; the dir must
22+
# contain a3s-box-<version>-linux-<arch>.tar.gz and a containerd shim
23+
# binary (containerd-shim-a3s-box-v2[-linux-<arch>]).
24+
#
25+
# Idempotent: safe to re-run (re-installs binaries, rewrites the containerd drop-in).
26+
set -euo pipefail
27+
28+
VERSION="v2.6.0"
29+
REPO="AI45Lab/Box"
30+
FROM_DIR=""
31+
WARMUP_IMAGE="busybox:latest" # first box on a fresh node builds a one-time cache
32+
# (~40s+); booting one here primes it so the first
33+
# real pod doesn't exceed the shim's boot poll. Best
34+
# effort — skipped silently if the image can't pull.
35+
36+
while [ $# -gt 0 ]; do
37+
case "$1" in
38+
--version) VERSION="$2"; shift 2 ;;
39+
--repo) REPO="$2"; shift 2 ;;
40+
--from-dir) FROM_DIR="$2"; shift 2 ;;
41+
--warmup-image) WARMUP_IMAGE="$2"; shift 2 ;;
42+
--no-warmup) WARMUP_IMAGE=""; shift ;;
43+
-h|--help) sed -n '2,33p' "$0"; exit 0 ;;
44+
*) echo "unknown arg: $1" >&2; exit 2 ;;
45+
esac
46+
done
47+
48+
log() { printf '\033[1;34m==>\033[0m %s\n' "$*"; }
49+
die() { printf '\033[1;31mERROR:\033[0m %s\n' "$*" >&2; exit 1; }
50+
51+
# ── preflight ───────────────────────────────────────────────────────────────
52+
[ "$(id -u)" = 0 ] || die "must run as root"
53+
command -v containerd >/dev/null || die "containerd not found on this node"
54+
[ -e /dev/kvm ] || die "/dev/kvm missing — this node has no KVM virtualization; a3s-box cannot run here"
55+
56+
case "$(uname -m)" in
57+
x86_64) ARCH=x86_64 ;;
58+
aarch64|arm64) ARCH=arm64 ;;
59+
*) die "unsupported architecture: $(uname -m)" ;;
60+
esac
61+
62+
TARBALL="a3s-box-${VERSION}-linux-${ARCH}.tar.gz"
63+
SHIM_ASSET="containerd-shim-a3s-box-v2-linux-${ARCH}"
64+
65+
work="$(mktemp -d)"
66+
trap 'rm -rf "$work"' EXIT
67+
68+
# ── obtain artifacts ────────────────────────────────────────────────────────
69+
if [ -n "$FROM_DIR" ]; then
70+
log "Installing from local dir: $FROM_DIR"
71+
[ -f "$FROM_DIR/$TARBALL" ] || die "missing $TARBALL in $FROM_DIR"
72+
cp "$FROM_DIR/$TARBALL" "$work/$TARBALL"
73+
if [ -f "$FROM_DIR/$SHIM_ASSET" ]; then cp "$FROM_DIR/$SHIM_ASSET" "$work/shim"
74+
elif [ -f "$FROM_DIR/containerd-shim-a3s-box-v2" ]; then cp "$FROM_DIR/containerd-shim-a3s-box-v2" "$work/shim"
75+
else die "missing containerd-shim-a3s-box-v2 in $FROM_DIR"; fi
76+
else
77+
base="https://github.com/${REPO}/releases/download/${VERSION}"
78+
log "Downloading $TARBALL"
79+
curl -fsSL "$base/$TARBALL" -o "$work/$TARBALL" || die "download failed: $base/$TARBALL"
80+
log "Downloading $SHIM_ASSET"
81+
curl -fsSL "$base/$SHIM_ASSET" -o "$work/shim" || die "download failed: $base/$SHIM_ASSET"
82+
fi
83+
84+
tar xzf "$work/$TARBALL" -C "$work"
85+
src="$work/a3s-box-${VERSION}-linux-${ARCH}"
86+
[ -d "$src" ] || die "unexpected tarball layout (no $src)"
87+
88+
# ── install binaries + libkrun ──────────────────────────────────────────────
89+
log "Installing a3s-box binaries to /usr/local/bin"
90+
install -m0755 "$src/a3s-box" "$src/a3s-box-cri" "$src/a3s-box-guest-init" "$src/a3s-box-shim" /usr/local/bin/
91+
92+
log "Installing libkrun to /usr/lib + ldconfig"
93+
cp -a "$src"/lib/libkrun* /usr/lib/
94+
ldconfig
95+
96+
log "Installing containerd-shim-a3s-box-v2 (/usr/local/bin + /opt/containerd/bin)"
97+
install -m0755 "$work/shim" /usr/local/bin/containerd-shim-a3s-box-v2
98+
install -d /opt/containerd/bin
99+
install -m0755 "$work/shim" /opt/containerd/bin/containerd-shim-a3s-box-v2
100+
101+
install -d /var/lib/a3s-box # shared A3S_HOME for the shim's a3s-box invocations
102+
103+
# ── register the runtime with containerd ────────────────────────────────────
104+
cfg=/etc/containerd/config.toml
105+
runtime_block() {
106+
cat <<'TOML'
107+
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.a3s-box]
108+
runtime_type = 'io.containerd.a3s-box.v2'
109+
[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.a3s-box.options]
110+
TOML
111+
}
112+
113+
if grep -qE "^imports[[:space:]]*=.*conf\.d" "$cfg" 2>/dev/null; then
114+
# containerd merges /etc/containerd/conf.d/*.toml — register via a drop-in so we
115+
# never touch the main config (clean + idempotent).
116+
install -d /etc/containerd/conf.d
117+
{ echo "version = 3"; echo; runtime_block; } > /etc/containerd/conf.d/a3s-box.toml
118+
log "Registered runtime via /etc/containerd/conf.d/a3s-box.toml"
119+
elif grep -q "runtimes.a3s-box\]" "$cfg" 2>/dev/null; then
120+
log "Runtime already present in $cfg — leaving as-is"
121+
else
122+
# No conf.d imports: append the runtime table to the main config.
123+
{ echo; runtime_block; } >> "$cfg"
124+
log "Registered runtime in $cfg"
125+
fi
126+
127+
log "Restarting containerd"
128+
systemctl restart containerd
129+
sleep 2
130+
131+
# ── verify ──────────────────────────────────────────────────────────────────
132+
log "Verification"
133+
systemctl is-active --quiet containerd || die "containerd is not active after restart"
134+
"$src/a3s-box" --version >/dev/null 2>&1 || /usr/local/bin/a3s-box --version >/dev/null 2>&1 || die "a3s-box CLI not runnable"
135+
echo " a3s-box: $(/usr/local/bin/a3s-box --version 2>/dev/null)"
136+
echo " libkrun: $(ldconfig -p | awk '/libkrun\.so/{print $1; exit}')"
137+
echo " shim: $(command -v containerd-shim-a3s-box-v2)"
138+
echo " /dev/kvm: present"
139+
echo " containerd: active"
140+
141+
# ── warm up (prime the one-time per-node boot cache) ────────────────────────
142+
if [ -n "$WARMUP_IMAGE" ]; then
143+
log "Warming up with $WARMUP_IMAGE (primes first-boot cache; --no-warmup to skip)"
144+
if A3S_HOME=/var/lib/a3s-box timeout 240 /usr/local/bin/a3s-box run \
145+
--name a3sbox-warmup "$WARMUP_IMAGE" -- true >/dev/null 2>&1; then
146+
echo " warm-up OK — first pod will boot fast"
147+
else
148+
echo " warm-up skipped (could not pull $WARMUP_IMAGE) — first pod may cold-start slowly"
149+
fi
150+
A3S_HOME=/var/lib/a3s-box /usr/local/bin/a3s-box rm -f a3sbox-warmup >/dev/null 2>&1 || true
151+
fi
152+
153+
printf '\n\033[1;32mDone.\033[0m a3s-box runtime installed on %s.\n' "$(hostname)"
154+
cat <<EOF
155+
Final step — from a control-plane node, label this node so a3s-box pods can schedule:
156+
157+
kubectl label node $(hostname) a3s-box.io/runtime=true
158+
159+
EOF

0 commit comments

Comments
 (0)