Skip to content

Commit b7bd22e

Browse files
jasonrandrewsjasonrandrews
authored
Merge pull request #2776 from geremyCohen/main
Add fastpath learning path and kernel build guide
2 parents 784c836 + b190392 commit b7bd22e

13 files changed

Lines changed: 1424 additions & 0 deletions

‎content/install-guides/kernel-build.md‎

Lines changed: 235 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
---
2+
title: Compiling Linux Kernels for Arm
3+
additional_search_terms:
4+
- linux kernel
5+
- tuxmake
6+
- fastpath
7+
- Image.gz
8+
- modules.tar.xz
9+
minutes_to_complete: 60
10+
author: Geremy Cohen
11+
official_docs: https://tuxmake.org/
12+
test_images:
13+
- ubuntu:latest
14+
test_maintenance: true
15+
weight: 1
16+
tool_install: true
17+
multi_install: false
18+
multitool_install_part: false
19+
layout: installtoolsall
20+
---
21+
22+
This guide walks you through building and installing Linux kernels on Arm cloud VM instances using utility scripts available at [`arm_kernel_install_guide`](https://github.com/geremyCohen/arm_kernel_install_guide).
23+
24+
## What do I need before building Arm kernels?
25+
26+
Before you begin, choose a cloud provider, and spin up an instance with the following characteristics:
27+
28+
### Cloud Provider ###
29+
This guide uses AWS as the example platform, but you can follow the same steps on any cloud provider that offers 64-bit Arm Ubuntu instances.
30+
31+
### Instance Type ###
32+
This guide uses an AWS `c8g.24xlarge` instance for demonstration. Any sufficiently large instance on your chosen provider will work, however, smaller instances may take longer or risk running out of memory during compilation. If you choose to use a different instance type, the minimal requirements are an Arm instance running at least 24 vCPUs with 200 GB of free storage.
33+
34+
### Operating System ###
35+
Ubuntu 24.04 LTS (64-bit Arm) is the recommended OS for this guide. Other distributions may work but are not officially supported (yet). If you find a different distro that works well, or you'd like to request support for a different setup, please open an issue or pull request.
36+
37+
# Install and Clone
38+
39+
With your build instance running and accessible via SSH, install the required dependencies:
40+
41+
```bash
42+
sudo apt update
43+
sudo apt install -y git python3 python3-pip python3-venv build-essential bc rsync dwarves flex bison libssl-dev libelf-dev btop yq jq
44+
45+
cd
46+
git clone https://github.com/geremyCohen/arm_kernel_install_guide.git ~/arm_kernel_install_guide
47+
cd ~/arm_kernel_install_guide
48+
chmod +x scripts/*.sh
49+
```
50+
51+
All commands in this guide assume you are inside this directory. The important script is `scripts/kernel_build_and_install.sh`, which orchestrates cloning the upstream kernel tree, configuring tuxmake, building artifacts, and optionally installing the kernel. The script now runs non-interactively—once invoked it proceeds without confirmation prompts, and any install operation automatically reboots the system when it finishes.
52+
53+
## How do I build Arm kernels?
54+
55+
The `kernel_build_and_install.sh` script is intentionally modular. Most users fall into two buckets:
56+
57+
1. **General Usage (non-fastpath)** – build kernels for direct install or downstream packaging.
58+
2. **Fastpath Usage** – build kernels that add the *fastpath* headers/perf configuration needed by the *fastpath* validation tool. Aside from those extra configs, the workflow mirrors the general case.
59+
60+
The sections below start simple (demo flags) and progress toward advanced scenarios. Every flag referenced is shown in at least one example so you can mix and match confidently.
61+
62+
### Flag overview
63+
64+
#### General usage flags
65+
66+
| Flag | Description |
67+
| --- | --- |
68+
| `--demo-default-build` | Shortcut: builds `v6.18.1` with default configs and leaves *fastpath* disabled. |
69+
| `--tag <tag>` / `--tags <list>` / `--tag-latest` | Select one or more kernel tags. Multiple tags build in parallel; the latest stable release can be added via `--tag-latest`. |
70+
| `--install-from <dir>` / `--install-format <flat\|deb\|auto>` | Install an existing build (flat artifacts or `.deb` packages) without recompiling. |
71+
| `--dry-run` | Generate a self-contained plan script (stored in `/tmp/kernel_plan_*.sh`) with the resolved arguments and exit without running the build. |
72+
| `--kernel-install [tag\|bool]` | Install a kernel right after it finishes building. When multiple tags build, provide the specific tag to install. |
73+
| `--change-to-64k <bool>` | Generate a 64 KB page-size kernel. Often combined with the install flags to test high-page builds. |
74+
| `--config-file <path>` | Reuse a captured stock config instead of `/boot/config-$(uname -r)`. |
75+
| `--include-bindeb-pkg` | Adds the `bindeb-pkg` target so `.deb` packages are produced alongside `Image.gz` and `modules.tar.xz`. |
76+
| `--kernel-command-line <string>` | Override GRUB’s `GRUB_CMDLINE_LINUX` when installing a kernel. |
77+
| `--append-to-kernel-version <text>` | Attach custom suffixes to `EXTRAVERSION` (e.g., `--append "-lab"`). |
78+
| `--kernel-dir <path>` / `--venv-path <path>` | Control where the kernel git checkout lives and which Python venv hosts tuxmake. Build artifacts always land under `~/kernels/<tag>`. |
79+
80+
#### Fastpath usage flags
81+
82+
| Flag | Description |
83+
| --- | --- |
84+
| `--demo-fastpath-build` | Shortcut: builds `v6.18.1` and `v6.19-rc1` with *fastpath* configs enabled. |
85+
| `--fastpath <bool>` | Manually enable/disable the *fastpath* configuration overlay (installs Docker as needed). |
86+
87+
Run `./scripts/kernel_build_and_install.sh --help` anytime for the exhaustive list.
88+
89+
---
90+
91+
## General Usage
92+
93+
### Worked examples (general usage)
94+
95+
#### 1. Quick sanity check (demo)
96+
```bash
97+
./scripts/kernel_build_and_install.sh --demo-default-build
98+
```
99+
This demo builds `v6.18.1`, populates `~/kernels/6.18.1`, and leaves Docker as well as *fastpath* configs untouched.
100+
101+
#### 2. Specify your own tag
102+
```bash
103+
./scripts/kernel_build_and_install.sh --tags v6.19-rc1
104+
```
105+
This behaves like the demo while targeting a release candidate instead of the pinned stable tag, and it still runs without any interactive prompts.
106+
107+
#### 3. Produce both flat artifacts and Debian packages
108+
```bash
109+
./scripts/kernel_build_and_install.sh \
110+
--tags v6.18.1 \
111+
--include-bindeb-pkg
112+
```
113+
Running this command outputs `Image.gz`, `modules.tar.xz`, `perf.tar.xz`, and `.deb` files (headers, image, dbg) under `~/kernels/6.18.1`.
114+
115+
#### 4. Build and immediately install (single tag)
116+
```bash
117+
./scripts/kernel_build_and_install.sh \
118+
--tags v6.18.1 \
119+
--kernel-install true
120+
```
121+
This command installs the freshly built kernel, regenerates initramfs, updates GRUB, and then reboots automatically.
122+
123+
#### 5. Multi-tag build + targeted install
124+
```bash
125+
./scripts/kernel_build_and_install.sh \
126+
--tags v6.18.1,v6.19-rc1 \
127+
--kernel-install v6.18.1
128+
```
129+
Both kernels build in parallel, but only `v6.18.1` is installed (followed by an automatic reboot), leaving the `v6.19-rc1` artifacts untouched under `~/kernels`.
130+
131+
#### 6. 64K page-size build and install
132+
```bash
133+
./scripts/kernel_build_and_install.sh \
134+
--tags v6.18.1 \
135+
--change-to-64k true \
136+
--kernel-install true \
137+
--append-to-kernel-version "-64k"
138+
```
139+
This variation produces a 64 KB build, installs it, appends “-64k” to the reported kernel version, and reboots automatically so you can verify the new settings.
140+
141+
#### 7. Install-only workflow (reusing flat artifacts)
142+
```bash
143+
./scripts/kernel_build_and_install.sh \
144+
--install-from ~/kernels/6.18.1 \
145+
--install-format flat
146+
```
147+
Instead of compiling, the script installs the saved `Image.gz`, `modules.tar.xz`, and `config` from a prior run, which is ideal when the directory contains flat artifacts rather than `.deb` packages.
148+
149+
#### 8. Install-only with Debian packages
150+
```bash
151+
./scripts/kernel_build_and_install.sh \
152+
--install-from ~/kernels/6.18.1 \
153+
--install-format deb
154+
```
155+
Here the script installs the `.deb` artifacts produced earlier via `--include-bindeb-pkg`, expecting files such as `linux-image-*` and `linux-headers-*` to exist in the source directory.
156+
157+
#### 9. Install-only with auto-detection
158+
```bash
159+
./scripts/kernel_build_and_install.sh \
160+
--install-from ~/kernels/6.18.1
161+
```
162+
This form lets the script auto-detect whether the directory contains flat artifacts or `.deb` files, which simplifies reuse when you are not sure which format is present.
163+
164+
#### 10. Generate a runnable plan without executing
165+
```bash
166+
./scripts/kernel_build_and_install.sh \
167+
--tags v6.18.1 \
168+
--dry-run
169+
```
170+
Instead of performing the build, this command writes a self-contained plan such as `/tmp/kernel_plan_v6.18.1_<hash>.sh` that embeds the current script plus the resolved arguments (minus `--dry-run`). Running that plan file later—on the same host or another system with the required dependencies—replays the exact workflow.
171+
172+
---
173+
174+
## Fastpath Usage
175+
176+
*fastpath* builds use the same tuxmake pipelines but add a configuration fragment that exposes the interfaces needed by the *fastpath* testing framework (extra headers, perf tooling, and Docker so *fastpath* can drive the host). *fastpath* workflows are build-only: do not combine `--fastpath true` (or the demo shortcut) with `--kernel-install` or any `--install-from` commands. Instead, let the build finish, copy the flat artifacts (`Image.gz`, `modules.tar.xz`, and `config`) to the *fastpath* host, and let the *fastpath* tooling handle deployment to the SUT. Docker is still installed automatically whenever *fastpath* mode is enabled so the *fastpath* controller can manage the host.
177+
178+
*fastpath* runs can still take advantage of tuning flags such as `--change-to-64k`, alternate configs, or custom output directories. Even if you specify packaging flags such as `--include-bindeb-pkg`, *fastpath* tests consume the flat artifacts.
179+
180+
### Fastpath examples
181+
182+
#### 1. Demo (dual-tag baseline)
183+
```bash
184+
./scripts/kernel_build_and_install.sh --demo-fastpath-build
185+
```
186+
The demo builds `v6.18.1` and `v6.19-rc1` with *fastpath* configs enabled and installs Docker automatically if the host lacks it.
187+
188+
#### 2. Custom tags with Fastpath enabled
189+
```bash
190+
./scripts/kernel_build_and_install.sh \
191+
--tags v6.18.1,v6.19-rc1 \
192+
--fastpath true
193+
```
194+
This explicit version mirrors the demo while making it easy to swap tag sets or add additional flags.
195+
196+
#### 3. Fastpath build with additional tuning
197+
```bash
198+
./scripts/kernel_build_and_install.sh \
199+
--tags v6.18.1,v6.19-rc1 \
200+
--fastpath true \
201+
--change-to-64k true
202+
```
203+
This variation still produces build-only artifacts, but it proves that you can layer other build-time options (like a 64 KB page size) on top of *fastpath* runs before exporting the results to the *fastpath* host.
204+
205+
---
206+
207+
### Where are the artifacts stored?
208+
209+
Each kernel tag produces a directory under `~/kernels/<kernel_version>` containing:
210+
211+
- `Image.gz` – compressed kernel image.
212+
- `modules.tar.xz` – modules tree (untar to `/lib/modules/<version>` when installing elsewhere).
213+
- `perf.tar.xz`, `cpupower.tar.xz` – optional user-space tools.
214+
- `config` – final merged configuration.
215+
- `config.stock` – copy of the original base config used for the build.
216+
217+
The script also writes a copy of the base config to `~/kernels/stock-configs/`, named after the running host kernel. Preserve this directory if you want to reuse the stock configuration later (for example, pass it via `--config-file`).
218+
219+
### How do I verify the results?
220+
221+
After every run:
222+
223+
```bash
224+
ls ~/kernels
225+
ls ~/kernels/<version>
226+
```
227+
228+
If you installed the kernel, the script reboots automatically when the install finishes. After it comes back up, confirm:
229+
230+
```bash
231+
uname -r
232+
getconf PAGE_SIZE # expect 65536 for 64K builds, 4096 otherwise
233+
```
234+
235+
When using build-only workflows, copy `Image.gz`, `modules.tar.xz`, and the corresponding `config` file to the downstream environment that will consume the kernel.

‎content/learning-paths/servers-and-cloud-computing/fastpath/_index.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
---
2+
title: Fastpath Kernel Build and Install Guide
3+
4+
minutes_to_complete: 45
5+
6+
who_is_this_for: Software developers and performance engineers who want to explore benchmarking across different kernel versions with Fastpath on Arm.
7+
8+
learning_objectives:
9+
- Understand how Fastpath streamlines kernel experimentation workflows
10+
- Provision an Arm-based build machine and compile Fastpath-enabled kernels on it
11+
- Provision an Arm-based test system, also known as the System Under Test (SUT)
12+
- Create a test plan consisting of kernel versions and benchmark suites
13+
- Launch an Arm-based Fastpath host to orchestrate the kernel benchmarking process on the SUT
14+
15+
prerequisites:
16+
- An AWS account with permissions to create EC2 instances
17+
- Familiarity with basic Linux administration and SSH
18+
19+
author: Geremy Cohen
20+
21+
### Tags
22+
skilllevels: Intermediate
23+
subjects: Operating Systems
24+
armips:
25+
- Neoverse
26+
operatingsystems:
27+
- Linux
28+
tools_software_languages:
29+
- Fastpath
30+
- tuxmake
31+
- Linux kernel
32+
33+
further_reading:
34+
- resource:
35+
title: Fastpath documentation
36+
link: https://fastpath.docs.arm.com/en/latest/index.html
37+
type: documentation
38+
- resource:
39+
title: Kernel install guide
40+
link: /install-guides/kernel-build/
41+
type: guide
42+
- resource:
43+
title: AWS Compute Service Provider learning path
44+
link: /learning-paths/servers-and-cloud-computing/csp/
45+
type: guide
46+
47+
### FIXED, DO NOT MODIFY
48+
# ================================================================================
49+
weight: 1
50+
layout: "learningpathall"
51+
learning_path_main_page: "yes"
52+
---
53+
54+
*fastpath* accelerates the cycle of building, deploying, and benchmarking Linux kernels on Arm-based infrastructure.
55+
56+
Off-the-shelf distributions ship with general-purpose kernels, but when you want to maximize performance you often need to rebuild the kernel with custom configuration options, experimental patches, or prerelease code. Custom kernels let you validate questions like “does an RC fix my workload regression?” or “do these extra debug settings impose measurable overhead?” without waiting for distro updates.
57+
58+
This learning path focuses on a concrete use case: run the Speedometer browser benchmark on two different kernel versions and determine which kernel delivers the best score. The workflow mirrors what kernel engineers do every day—build, deploy, and compare—while *fastpath* keeps the process reproducible.
59+
60+
To make that manageable we split the work across three Arm-based nodes:
61+
62+
1. **Build host** – compiles the kernels with *fastpath*-specific options.
63+
2. **fastpath host** – orchestrates deployments, plan execution, and result collection.
64+
3. **System Under Test (SUT)** – runs each kernel and executes the benchmark workloads.
65+
66+
Arm’s `arm_kernel_install_guide` repository supplies wrapper scripts that streamline each step. You will use them to compile kernels on the build host, prepare the *fastpath* host and SUT, generate a plan, execute it, and then read the results without having to stitch together the workflow manually.
67+
68+
> **Tip:** The complete *fastpath* reference documentation is available at [fastpath.docs.arm.com](https://fastpath.docs.arm.com/en/latest/index.html).

0 commit comments

Comments
 (0)