Skip to content
This repository was archived by the owner on Apr 6, 2026. It is now read-only.

Commit 31c6e9b

Browse files
drbhdanieldk
drbh
and
danieldk
authored
feat: built root and user docker image variants (#139)
This PR improves the docker built in various ways 1. split dockerfile into root and non root builds - root for dev/ when the user has control over container launch - non root that expect uid 1000 to lock down permissions in CI/non rootless container envs 2. bump workflow to publish both variants of containers 3. prefer the `--mount` flag over the older `--volume` when launching the container for more granular permission control` --------- Co-authored-by: Daniël de Kok <me@danieldk.eu>
1 parent 5e48ab9 commit 31c6e9b

8 files changed

Lines changed: 350 additions & 26 deletions

File tree

‎.github/workflows/docker-build-push.yaml‎

Lines changed: 55 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ on:
77
paths:
88
# Only run on changes to the Dockerfile or workflow file
99
- "Dockerfile"
10+
- "dockerfiles/**"
1011
- ".github/workflows/docker-build-push.yaml"
1112
workflow_dispatch: # Allow manual triggering
1213

@@ -15,8 +16,8 @@ env:
1516
IMAGE_NAME: ${{ github.repository_owner }}/kernel-builder
1617

1718
jobs:
18-
build-and-push:
19-
name: Build and Push Docker Image
19+
build-and-push-user:
20+
name: Build and Push User Docker Image
2021
runs-on: ubuntu-latest
2122
permissions:
2223
contents: read
@@ -43,15 +44,16 @@ jobs:
4344
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
4445
tags: |
4546
type=raw,value=latest,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
46-
type=sha,format=short
47-
type=ref,event=branch
48-
type=semver,pattern={{version}}
49-
type=semver,pattern={{major}}.{{minor}}
47+
type=sha,prefix=user-,format=short
48+
type=ref,prefix=user-,event=branch
49+
type=semver,prefix=user-,pattern={{version}}
50+
type=semver,prefix=user-,pattern={{major}}.{{minor}}
5051
5152
- name: Build and push Docker image
5253
uses: docker/build-push-action@v5
5354
with:
5455
context: .
56+
file: ./dockerfiles/Dockerfile.user
5557
push: true
5658
tags: ${{ steps.meta.outputs.tags }}
5759
labels: ${{ steps.meta.outputs.labels }}
@@ -60,3 +62,50 @@ jobs:
6062
CORES=8
6163
cache-from: type=gha
6264
cache-to: type=gha,mode=max
65+
66+
build-and-push-root:
67+
name: Build and Push Root Docker Image
68+
runs-on: ubuntu-latest
69+
permissions:
70+
contents: read
71+
packages: write
72+
73+
steps:
74+
- name: Checkout repository
75+
uses: actions/checkout@v4
76+
77+
- name: Set up Docker Buildx
78+
uses: docker/setup-buildx-action@v3
79+
80+
- name: Log in to the Container registry
81+
uses: docker/login-action@v3
82+
with:
83+
registry: ${{ env.REGISTRY }}
84+
username: ${{ github.actor }}
85+
password: ${{ secrets.GITHUB_TOKEN }}
86+
87+
- name: Extract metadata (tags, labels) for Docker
88+
id: meta-root
89+
uses: docker/metadata-action@v5
90+
with:
91+
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
92+
tags: |
93+
type=raw,value=root,enable=${{ github.ref == format('refs/heads/{0}', 'main') }}
94+
type=sha,format=short
95+
type=ref,event=branch
96+
type=semver,pattern={{version}}
97+
type=semver,pattern={{major}}.{{minor}}
98+
99+
- name: Build and push Docker image
100+
uses: docker/build-push-action@v5
101+
with:
102+
context: .
103+
file: ./dockerfiles/Dockerfile
104+
push: true
105+
tags: ${{ steps.meta-root.outputs.tags }}
106+
labels: ${{ steps.meta-root.outputs.labels }}
107+
build-args: |
108+
MAX_JOBS=8
109+
CORES=8
110+
cache-from: type=gha,scope=root
111+
cache-to: type=gha,mode=max,scope=root

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
1-
.obsidian
1+
.obsidian
2+
target

‎README.md‎

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -24,20 +24,21 @@ is updated for the final release.
2424

2525
## 🚀 Quick Start
2626

27-
We provide a Docker image with which you can build a kernel:
27+
We provide Docker containers for building kernels. For a quick build:
2828

2929
```bash
30-
# navigate to the activation directory
31-
cd examples/activation
30+
# Using the prebuilt container
31+
docker run --mount type=bind,source=$(pwd),target=/kernelcode ghcr.io/huggingface/kernel-builder:{SHA}
32+
```
33+
34+
or build the container locally:
3235

33-
# then run the following command to build the kernel
34-
docker run --rm \
35-
-v $(pwd):/home/nixuser/kernelcode \
36-
ghcr.io/huggingface/kernel-builder:latest
36+
```bash
37+
docker build -t kernel-builder:local -f dockerfiles/Dockerfile .
38+
docker run --mount type=bind,source=$(pwd),target=/kernelcode kernel-builder:local
3739
```
3840

39-
This will build the kernel and save the output in the `build` directory in
40-
the activation folder.
41+
See [dockerfiles/README.md](./dockerfiles/README.md) for more options, including a user-level container for CI/CD environments.
4142

4243
# 📚 Documentation
4344

‎dockerfiles/Dockerfile‎

Lines changed: 207 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,207 @@
1+
FROM nixos/nix:2.18.8
2+
3+
# default build args
4+
ARG MAX_JOBS=4
5+
ARG CORES=4
6+
7+
# Combine RUN commands to reduce layers and improve caching
8+
RUN echo "experimental-features = nix-command flakes" >> /etc/nix/nix.conf \
9+
&& echo "max-jobs = $MAX_JOBS" >> /etc/nix/nix.conf \
10+
&& echo "cores = $CORES" >> /etc/nix/nix.conf \
11+
&& nix profile install nixpkgs#cachix nixpkgs#git-lfs \
12+
&& cachix use huggingface
13+
WORKDIR /app
14+
# Copy builder files
15+
COPY . /etc/kernel-builder/
16+
# Set environment variables
17+
ENV MAX_JOBS=${MAX_JOBS}
18+
ENV CORES=${CORES}
19+
# Create directory and setup script
20+
RUN mkdir -p /etc/kernel-builder && \
21+
cat <<'EOF' > /etc/kernel-builder/cli.sh
22+
#!/bin/sh
23+
set -e
24+
# Default values
25+
BUILD_URL=""
26+
DEV_SHELL=0
27+
HELP=0
28+
# CLI usage function
29+
function show_usage {
30+
echo "Kernel Builder CLI"
31+
echo ""
32+
echo "Usage: docker run [docker-options] kernel-builder:dev [command] [options]"
33+
echo ""
34+
echo "Commands:"
35+
echo " build Build the kernel extension (default if no command specified)"
36+
echo " dev Start a development shell"
37+
echo " fetch [URL] Clone and build from a Git URL"
38+
echo " help Show this help message"
39+
echo ""
40+
echo "Options:"
41+
echo " --jobs, -j NUMBER Set maximum number of parallel jobs (default: $MAX_JOBS)"
42+
echo " --cores, -c NUMBER Set number of cores per job (default: $CORES)"
43+
echo ""
44+
echo "Examples:"
45+
echo " docker run --mount type=bind,source=$(pwd),target=/kernelcode kernel-builder:root build"
46+
echo " docker run -it --mount type=bind,source=$(pwd),target=/kernelcode kernel-builder:root dev"
47+
echo " docker run --mount type=bind,source=$(pwd),target=/kernelcode kernel-builder:root fetch https://huggingface.co/user/repo.git"
48+
}
49+
50+
# Function to generate a basic flake.nix if it doesn't exist
51+
function ensure_flake_exists {
52+
local work_dir=$1
53+
if [ ! -f "${work_dir}/flake.nix" ]; then
54+
echo "No flake.nix found, creating a basic one..."
55+
cat <<'FLAKE_EOF' > "${work_dir}/flake.nix"
56+
{
57+
description = "Flake for Torch kernel extension";
58+
inputs = {
59+
kernel-builder.url = "github:huggingface/kernel-builder";
60+
};
61+
outputs = { self, kernel-builder, }:
62+
kernel-builder.lib.genFlakeOutputs {
63+
path = ./.;
64+
rev = self.shortRev or self.dirtyShortRev or self.lastModifiedDate;
65+
};
66+
}
67+
FLAKE_EOF
68+
echo "flake.nix created. You can customize it as needed."
69+
else
70+
echo "flake.nix already exists, skipping creation."
71+
fi
72+
}
73+
# Function to build the extension
74+
function build_extension {
75+
local work_dir=$1
76+
local output_dir=$2
77+
78+
echo "Building Torch Extension Bundle from ${work_dir}"
79+
cd "${work_dir}"
80+
81+
# Check if work_dir is a git repo and get hash if possible
82+
if [ -d "${work_dir}/.git" ]; then
83+
# Mark git as safe to allow commands
84+
git config --global --add safe.directory "${work_dir}"
85+
# Try to get git revision
86+
REV=$(git rev-parse --short=8 HEAD)
87+
88+
# Check if working directory is dirty
89+
if [ -n "$(git status --porcelain 2)" ]; then
90+
REV="${REV}-dirty"
91+
fi
92+
else
93+
# Generate random material if not a git repo
94+
REV=$(dd if=/dev/urandom status=none bs=1 count=10 2>/dev/null | base32 | tr '[:upper:]' '[:lower:]' | head -c 10)
95+
fi
96+
echo "Building with rev $REV"
97+
98+
# Check for flake.nix or create one
99+
ensure_flake_exists "${work_dir}"
100+
101+
# Make sure the build is up to date
102+
nix run .#update-build
103+
104+
# Pure bundle build
105+
echo "Building with Nix..."
106+
nix build \
107+
. \
108+
--max-jobs $MAX_JOBS \
109+
-j $CORES \
110+
-L
111+
112+
echo "Build completed. Copying results to ${output_dir}"
113+
mkdir -p "${output_dir}"
114+
cp -r --dereference ./result/* "${output_dir}/"
115+
# As root, ensure proper permissions for host access
116+
chmod -R 777 "${output_dir}"
117+
echo "Done - results available in ${output_dir}"
118+
}
119+
# Function to start a dev shell
120+
function start_dev_shell {
121+
echo "Starting development shell..."
122+
# Check for flake.nix or create one
123+
ensure_flake_exists "/kernelcode"
124+
cd /kernelcode
125+
/root/.nix-profile/bin/nix develop
126+
}
127+
# Function to fetch and build from URL
128+
function fetch_and_build {
129+
if [ -z "$1" ]; then
130+
echo "Error: URL required for fetch command"
131+
show_usage
132+
exit 1
133+
fi
134+
135+
local repo_url="$1"
136+
local src_dir="/tmp/kernel-src"
137+
local output_dir="/kernelcode/result"
138+
139+
echo "Fetching code from ${repo_url} to ${src_dir}"
140+
# Create a temporary directory for the clone
141+
mkdir -p "${src_dir}"
142+
143+
# Clone the repository to the temporary directory
144+
git lfs install
145+
git clone "${repo_url}" "${src_dir}"
146+
147+
# Build from the temporary directory and copy results to mounted output
148+
build_extension "${src_dir}" "${output_dir}"
149+
}
150+
# Parse arguments
151+
COMMAND="build" # Default command
152+
ARGS=()
153+
154+
while [[ $# -gt 0 ]]; do
155+
case $1 in
156+
build|dev|fetch|help)
157+
COMMAND="$1"
158+
shift
159+
;;
160+
--jobs|-j)
161+
MAX_JOBS="$2"
162+
shift 2
163+
;;
164+
--cores|-c)
165+
CORES="$2"
166+
shift 2
167+
;;
168+
-*)
169+
echo "Unknown option: $1"
170+
show_usage
171+
exit 1
172+
;;
173+
*)
174+
ARGS+=("$1")
175+
shift
176+
;;
177+
esac
178+
done
179+
# Execute the command
180+
case $COMMAND in
181+
build)
182+
# When building existing code, use the mounted directory
183+
build_extension "/kernelcode" "/kernelcode/build"
184+
;;
185+
dev)
186+
start_dev_shell
187+
;;
188+
fetch)
189+
fetch_and_build "${ARGS[0]}"
190+
;;
191+
help)
192+
show_usage
193+
;;
194+
*)
195+
echo "Unknown command: $COMMAND"
196+
show_usage
197+
exit 1
198+
;;
199+
esac
200+
EOF
201+
RUN chmod +x /etc/kernel-builder/cli.sh
202+
# Create output directory structure
203+
RUN mkdir -p /kernelcode/build
204+
# Set up volume for kernelcode
205+
VOLUME /kernelcode
206+
207+
ENTRYPOINT ["/etc/kernel-builder/cli.sh"]

Dockerfile renamed to dockerfiles/Dockerfile.user

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,9 @@ function build_extension {
117117
# Check for flake.nix or create one
118118
ensure_flake_exists
119119

120+
# Make sure the build is up to date
121+
nix run .#update-build
122+
120123
# Pure bundle build
121124
# TODO: remove the "bundle" after resolving
122125
echo "Building with Nix..."
@@ -231,4 +234,4 @@ RUN chmod +x /home/nixuser/bin/cli.sh && \
231234
USER nixuser
232235

233236
# Use the cli.sh script directly
234-
ENTRYPOINT ["/home/nixuser/bin/cli.sh"]
237+
ENTRYPOINT ["/home/nixuser/bin/cli.sh"]

‎dockerfiles/README.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Kernel Builder Docker Containers
2+
3+
This directory contains two Docker containers for different use cases:
4+
5+
## Root Container (`Dockerfile`)
6+
7+
This container runs as root and can modify file permissions when mounting volumes.
8+
9+
```bash
10+
# Build the container
11+
docker build -t kernel-builder:root -f Dockerfile ..
12+
13+
# Use the container
14+
docker run --mount type=bind,source=$(pwd),target=/kernelcode kernel-builder:root build
15+
```
16+
17+
## User Container (`Dockerfile.user`)
18+
19+
This container runs as a non-root user (nixuser with UID 1000) for more secure environments.
20+
21+
```bash
22+
# Build the container
23+
docker build -t kernel-builder:user -f Dockerfile.user ..
24+
25+
# Important: Prepare a directory with correct permissions
26+
mkdir -p ./build
27+
chown -R 1000:1000 ./build # Match the UID:GID of nixuser in the container
28+
29+
# Use with proper permissions for the build directory
30+
docker run --mount type=bind,source=$(pwd),target=/home/nixuser/kernelcode \
31+
--mount type=bind,source=$(pwd)/build,target=/home/nixuser/kernelcode/build \
32+
kernel-builder:user build
33+
```
34+
35+
## Environment Variables
36+
37+
Both containers support these build options:
38+
39+
```bash
40+
# Set options at build time
41+
docker build -t kernel-builder:custom --build-arg MAX_JOBS=8 --build-arg CORES=2 -f Dockerfile ..
42+
43+
# Or at runtime
44+
docker run -e MAX_JOBS=8 -e CORES=2 --mount type=bind,source=$(pwd),target=/kernelcode kernel-builder:root build
45+
```

0 commit comments

Comments
 (0)