Skip to content

Commit 53e0a8c

Browse files
committed
doc: add ISO boot documentation, efifb display chain, and BusyBox reference
New files: - doc/index.md: documentation index with one-line summary per doc file - doc/iso_boot.md: ISO boot parameter reference — details every kernel parameter injected, which initramfs framework uses it, parameter value rules, known limitations (Debian DVD installer, efifb framebuffer chain); credits u-root's boot/iso package as inspiration for the layered approach (u-root/u-root#3578) - doc/busybox_perks.md: BusyBox v1.36.1 vs GNU command differences for every tool used in initrd scripts; documents sort -k1 -u incompatibility where BusyBox sort -u deduplicates by entire line, not by sort key - doc/modules.md: module list from Makefile showing which tools are BusyBox applets vs standalone binaries Updated files: - doc/architecture.md: add display output after kexec section with ASCII diagram showing the full chain: coreboot/libgfxinit -> VIDEO_TYPE_EFI -> Heads efifb -> kexec-tools -> target kernel efifb; documents the smem_start limitation (Linux no longer leaks physical fb address); notes that current efifb approach is suboptimal (waiting for upstream fixes); TPM DUK is the only supported workaround for LUKS passphrase on blank display - doc/boot-process.md: add Stage 2b (USB ISO Boot) section covering kexec-iso-init.sh flow (Layer 1 initrd compat, Layer 2 loopback.cfg, Layer 3 kexec-select-boot), compat markers legend, and parameter injection - doc/logging.md: update ANSI policy — codes are preserved in debug.log for less -R rendering; remove _strip_ansi references and "MUST NOT appear in debug.log" language - doc/ux-patterns.md: document boot option format and compat markers - initrd/bin/kexec-iso-init.sh: add u-root boot/iso credit in header and Layer 2 section Signed-off-by: Thierry Laurion <insurgo@riseup.net>
1 parent 0bdd3b6 commit 53e0a8c

9 files changed

Lines changed: 758 additions & 3 deletions

File tree

doc/architecture.md

Lines changed: 89 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ Changes to user configuration are persisted by reflashing the ROM (CBFS operatio
8787
The top-level `Makefile` orchestrates:
8888

8989
- Cross-compiler (`musl-cross-make`, target: `x86_64-linux-musl` or `powerpc64le-linux-musl`)
90-
- Modules (coreboot, Linux, busybox, GPG, cryptsetup, kexec, LVM2, )
90+
- Modules (coreboot, Linux, busybox, GPG, cryptsetup, kexec, LVM2, zstd, … — see [modules.md](modules.md) for the full list)
9191
- Six CPIO archives assembled into the initrd:
9292
1. `dev.cpio` — device nodes
9393
2. `modules.cpio` — kernel modules
@@ -119,3 +119,91 @@ The CI pipeline's workspace and cache behavior is documented in
119119
- **Fail-closed** — failed verification drops to authenticated recovery shell, not an unverified OS boot
120120
- **Separation of duties** — the key that signs `/boot` lives on a hardware security dongle, never in the ROM
121121
- **Auditability** — all source is open, builds are reproducible, ROM images are verifiable
122+
123+
---
124+
125+
## Display output after kexec
126+
127+
```
128+
kexec boundary
129+
┌──────────────────────────────────────────────────┐
130+
│ DISPLAY CONTROLLER │
131+
│ (initialised by coreboot, continuously scans │
132+
│ out from framebuffer at physical address X) │
133+
└──────────────────────┬───────────────────────────┘
134+
│ reads pixels from
135+
136+
┌────────────────┐
137+
│ scanout buf │
138+
│ @ phys addr X │
139+
└────────────────┘
140+
141+
───────────────────────┼──────────────────────────────
142+
Heads side │ OS side
143+
144+
┌──────────────────────────┐ ┌───────────────────────────────┐
145+
│ coreboot/libgfxinit │ │ target kernel (after kexec) │
146+
│ - initialises display │ │ sysfb_init() reads boot │
147+
│ - sets VIDEO_TYPE_EFI │ │ params → VIDEO_TYPE_EFI │
148+
│ in screen_info │ │ → "efi-framebuffer" device │
149+
└────────────┬─────────────┘ │ → efifb binds │
150+
│ │ BUT: lfb_base == 0 │
151+
▼ │ → efifb_probe fails │
152+
┌──────────────────────────┐ │ → no /dev/fb0 │
153+
│ Heads kernel │ │ → initramfs display blank │
154+
│ sysfb_init() → sees │ └───────────────────────────────┘
155+
│ VIDEO_TYPE_EFI │
156+
│ → "efi-framebuffer" │ ┌───────────────────────────────┐
157+
│ → efifb binds │ │ DRM driver (post-switchroot) │
158+
│ → Heads GUI visible │ │ i915 reinitialises display │
159+
└────────────┬─────────────┘ │ → framebuffer works again │
160+
│ └───────────────────────────────┘
161+
162+
163+
┌──────────────────────────┐
164+
│ kexec-tools │
165+
│ opens /dev/fb0 → reads │
166+
│ fb parameters │
167+
│ fix.id = "EFI VGA" ✓ │
168+
│ smem_start = 0 ✗ │
169+
│ (kernel hid it) │
170+
│ writes to new kernel │
171+
│ boot params: │
172+
│ orig_video_isVGA = EFI │
173+
│ lfb_base = 0 │
174+
└──────────────────────────┘
175+
```
176+
177+
Heads' initrd runs under a kernel compiled with `CONFIG_FB_EFI=y`.
178+
coreboot/libgfxinit initialises the display hardware (PLL, timings,
179+
scanout buffer) and presents it to Linux as an efifb-compatible
180+
framebuffer (`screen_info` set to `VIDEO_TYPE_EFI`). Heads' kernel
181+
binds efifb, maps the scanout buffer, and the Heads console and
182+
whiptail GUI are visible. Without this, Heads would have no
183+
framebuffer at all.
184+
185+
After kexec, the display hardware remains in its initialised state
186+
(kexec does not reset devices — the controller keeps scanning out
187+
from the same physical address). The target kernel must also have
188+
`CONFIG_FB_EFI=y` to adopt the same framebuffer — simplefb and
189+
simpledrm are not compatible because the scanout buffer was set up
190+
as an efifb-compatible buffer by coreboot, not as a simplefb buffer.
191+
192+
**Known limitation — lost framebuffer address:**
193+
194+
Linux no longer exposes the physical framebuffer address to userspace.
195+
kexec-tools obtains `smem_start = 0` from `FBIOGET_FSCREENINFO` and
196+
writes `lfb_base = 0` into the new kernel's boot params. The target
197+
kernel's efifb sees a zero address and cannot map the framebuffer —
198+
the display stays blank even though the controller is still scanning
199+
out the correct memory (set up by coreboot, preserved through kexec).
200+
201+
Historically, Heads worked around this by using i915 with
202+
`CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM=y` (leaking the physical address),
203+
but that was suboptimal. The current efifb approach is also
204+
suboptimal — we are waiting for improvements in coreboot, Linux, and
205+
kexec-tools to properly convey the framebuffer state across kexec.
206+
207+
**TPM Disk Unlock Key** works around this for encrypted disks by
208+
injecting the LUKS key before kexec — the initramfs never prompts for
209+
a passphrase on a blank display. Serial console is unaffected.

doc/boot-process.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,97 @@ menu, system info, power off.
122122

123123
---
124124

125+
## Stage 2b: USB ISO Boot (`kexec-iso-init.sh`)
126+
127+
When booting from an ISO file on USB media, `kexec-iso-init.sh` handles the ISO
128+
boot flow. It is invoked from the "USB ISO Boot" option in the main menu.
129+
130+
### Flow
131+
132+
1. **Signature verification**: Check for `.sig` or `.asc` detached signature
133+
2. **Mount ISO**: Mount the ISO file as loopback device at `/boot`
134+
3. **Layer 1 — initramfs fs compatibility check** (`check_initrd_fs_compat`):
135+
Before presenting boot options, verify the ISO's initramfs contains kernel
136+
modules for the USB partition's filesystem (ext4/vfat/exfat). If the initrd
137+
can't read the USB filesystem, the kernel won't find the ISO after kexec.
138+
- Parsing boot configs for initrd paths (instead of searching the whole ISO)
139+
- Unpacking each initrd and checking for required `.ko` files and
140+
`modules.builtin`
141+
- Each initrd gets its own independent `[OK]` / `[!]` / (blank) marker in
142+
`/tmp/kexec_initrd_compat.txt` (the per-initrd flag `initrd_supports_fs` is tracked
143+
separately from the global `any_supported` flag, so no initrd is silently
144+
skipped)
145+
- `[OK]` = initrd has the needed module as `.ko`, has it in
146+
`modules.builtin`, or has no `.ko` files at all (minimal initrd with
147+
everything built into the kernel — nothing to check against).
148+
- `[!]` = initrd has loadable kernel modules but none for the USB
149+
filesystem type. No built-in assumption — we report what we find.
150+
- Read-only filesystems (iso9660/squashfs/udf) and unmapped fstypes skip
151+
- All initrds are checked (no early break) so the compat file is complete.
152+
4. **Layer 2 — loopback.cfg fast path**: If the ISO has a `loopback.cfg`, parse
153+
it and resolve GRUB variables (`${iso_path}`, `${isofile}`) to extract the
154+
ISO kernel params from loopback entries.
155+
5. **Boot param injection**: When Layer 2 resolves nothing (no GRUB vars found
156+
in loopback.cfg), all common ISO boot methods are injected unconditionally
157+
as kernel ADD params so the ISO initrd can pick whichever it supports:
158+
- `iso-scan/filename=/$ISO_PATH` — Ubuntu casper, Fedora dracut
159+
- `findiso=/$ISO_PATH` — Debian live-boot, NixOS stage-1
160+
- `img_dev=/dev/disk/by-uuid/$DEV_UUID` — block device containing the ISO
161+
- `img_loop=$ISO_PATH` — loopback file path (relative)
162+
- `iso=$DEV_UUID/$ISO_PATH` — UUID/path alternative
163+
- `live-media=/dev/disk/by-uuid/$DEV_UUID` — device filter (casper, live-boot)
164+
The kernel ignores parameters it doesn't understand.
165+
`fromiso=` is intentionally not injected because it conflicts with `findiso=`
166+
in Debian live-boot's `check_dev()``fromiso` mounts the ISO, then `findiso`
167+
looks for the ISO file inside the mounted ISO (not found), unmounts it,
168+
leaving orphaned loop devices that get re-scanned → infinite loop.
169+
`findiso=` alone covers Debian and NixOS.
170+
`live-media-path=` is intentionally not injected because the default differs
171+
per distro (`/live` for Debian, `/casper` for Ubuntu/PureOS, `/LiveOS` for
172+
Fedora); leaving it unset lets each distro use its own default.
173+
6. **Layer 3 — kexec-select-boot**: Launch the standard boot menu with `-u`
174+
(unique entries, dedup sorted by name).
175+
176+
### Initrd compatibility markers in the boot menu
177+
178+
During Layer 1, `check_initrd_fs_compat` writes per-initrd results to
179+
`/tmp/kexec_initrd_compat.txt`. `kexec-select-boot` reads this file and shows
180+
`[OK]` or `[!]` at the start of each menu line (before the entry name):
181+
182+
| Marker | Meaning | Behavior |
183+
|--------|---------|----------|
184+
| `[OK]` | Initrd has the USB fs module (as .ko or modules.builtin) | Boot should work |
185+
| `[!]` | Initrd has loadable modules but none for the USB fs type | May fail after kexec |
186+
| (blank) | Initrd has zero .ko files — can't verify either way | Assume OK (minimal initrd) |
187+
| (none) | Entry has no initrd (memtest, etc.) | No filesystem dependency |
188+
189+
A `NOTE` (3-second sleep, cannot scroll past) is displayed before the menu
190+
explaining the legend. Markers follow `doc/logging.md` accessibility rules:
191+
text-based, serial-safe, not color-dependent.
192+
193+
### Compatibility note for ext4 and vfat
194+
195+
Initrds with no `.ko` files at all get no marker at all (blank) — we can't
196+
verify either way, so nothing is displayed.
197+
198+
### Boot param injection
199+
200+
When Layer 2 (loopback.cfg) resolves no GRUB variables, the following
201+
parameters are injected unconditionally so the ISO initrd can find the USB
202+
partition and the ISO file after kexec, regardless of which distribution's
203+
init system it uses:
204+
205+
| Parameter | Example | Used by |
206+
|-----------|---------|---------|
207+
| `iso-scan/filename=` | `/ISOs/foo.iso` | Ubuntu casper, Fedora dracut |
208+
| `findiso=` | `/ISOs/foo.iso` | Debian live-boot, NixOS stage-1 |
209+
| `img_dev=` | `/dev/disk/by-uuid/UUID` | Block device hint |
210+
| `img_loop=` | `ISOs/foo.iso` | Loopback path |
211+
| `iso=` | `UUID/ISOs/foo.iso` | Alternative path |
212+
| `live-media=` | `/dev/disk/by-uuid/UUID` | Device filter (casper, live-boot) |
213+
214+
---
215+
125216
## Stage 3: kexec-select-boot
126217

127218
Called from the boot menu. Responsible for final verification and OS handoff.

doc/busybox_perks.md

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# BusyBox vs GNU: Heads usage reference
2+
3+
Heads initrd scripts run under BusyBox v1.36.1, not GNU coreutils.
4+
This documents every command usage across Heads scripts and the BusyBox
5+
adaptations required.
6+
7+
## dd
8+
9+
BusyBox `dd` supports `status=`, `iflag=`, `oflag=` the same as GNU.
10+
11+
**Usage in Heads**:
12+
```bash
13+
dd if="$file" bs=6 count=1 status=none # kexec-iso-init.sh:
14+
dd bs=1 count=1 status=none # unpack_initramfs.sh:38
15+
dd bs="$segment_end" count=1 status=none # unpack_initramfs.sh:101
16+
```
17+
18+
## xxd
19+
20+
**BusyBox quirk**: `xxd -p` pads the last line to 60 columns with spaces.
21+
GNU xxd does not pad.
22+
23+
**Usage in Heads**:
24+
```bash
25+
# Good — strips padding:
26+
next_byte="$(dd bs=1 count=1 status=none | xxd -p | tr -d '\n ')" # unpack_initramfs.sh:38
27+
magic="$(dd if="$f" bs=6 count=1 status=none | xxd -p | tr -d '\n ')" # unpack_initramfs.sh:68
28+
29+
# Reverse — needs fold workaround (from etc/functions.sh:2701):
30+
fold -w 60 | xxd -p -r
31+
```
32+
33+
## cpio
34+
35+
**BusyBox quirk**: Stops at first TRAILER. GNU reads past it and exits 2.
36+
37+
**Heads pattern**: `cpio -i -d "${CPIO_ARGS[@]}" 2>/dev/null || true`
38+
The `|| true` handles GNU's exit 2 on multi-segment archives.
39+
40+
For multi-segment extraction (`unpack_initramfs.sh:94-106`), the TRAILER offset
41+
is pre-computed and dd limits cpio's input to exactly one segment, so both
42+
BusyBox and GNU behave identically.
43+
44+
## grep
45+
46+
**BusyBox**: no `-a` flag, but treats binary as text by default.
47+
48+
**Heads usage**: `grep -F -b -o "TRAILER!!!" "$file" 2>/dev/null` at `unpack_initramfs.sh:76`.
49+
The `-a` flag is not needed — BusyBox treats binary as text by default; GNU grep handles it without `-a` too.
50+
51+
**Heads pattern**: `grep -F -b -o "TRAILER!!!" "$file" 2>/dev/null | head -1 | cut -d: -f1 || true`
52+
53+
## stat
54+
55+
Identical for `stat -c %s FILE`.
56+
57+
**Usage in Heads**:
58+
```bash
59+
orig_size="$(stat -c %s "$unpack_archive")" # unpack_initramfs.sh:127
60+
rest_size="$(stat -c %s "$rest_archive")" # unpack_initramfs.sh:128
61+
rest_size="$(stat -c %s "$next_archive" 2>/dev/null || echo 0)" # unpack_initramfs.sh:147
62+
```
63+
64+
## find
65+
66+
**Usage in Heads** (all supported by BusyBox):
67+
```bash
68+
find "$dir" -name "*.ko*" -type f 2>/dev/null | head -1 # kexec-iso-init.sh
69+
find "$dir" -name '*.cfg' -type f 2>/dev/null # kexec-iso-init.sh
70+
find "$dir" -name "*.ko*" 2>/dev/null | grep -q "ext4" # kexec-iso-init.sh
71+
```
72+
73+
## gunzip / gzip / zcat
74+
75+
Identical. Used via pipe in segment decompression:
76+
```bash
77+
gunzip | unpack_cpio # unpack_initramfs.sh:111
78+
```
79+
80+
## unxz / xzcat
81+
82+
Identical:
83+
```bash
84+
unxz | unpack_cpio # unpack_initramfs.sh:115
85+
```
86+
87+
## zstd / zstd-decompress
88+
89+
**Standalone binary** compiled at `build/x86/zstd-1.5.5/programs/zstd-decompress`
90+
and included in the initrd via `Makefile:745` (`CONFIG_ZSTD`). Not a BusyBox
91+
applet, but available in all boards with `CONFIG_ZSTD=y`.
92+
93+
Usage via pipe (`unpack_initramfs.sh:119`):
94+
```bash
95+
zstd-decompress -d < input.zst # reads stdin, writes stdout
96+
```
97+
98+
**Current code**: `(zstd-decompress -d 2>/dev/null || zstd -d 2>/dev/null || true) | unpack_cpio`
99+
— should work if `CONFIG_ZSTD=y` in the board config.
100+
101+
## sed
102+
103+
Identical for all patterns used in Heads:
104+
```bash
105+
sed 's|^/dev/||' # kexec-iso-init.sh (path stripping)
106+
sed 's/^append //' # kexec-iso-init.sh (param extraction)
107+
sed 's/^initrd //' # kexec-iso-init.sh (field extraction)
108+
sed "s|\${$var}|$val|g" # kexec-iso-init.sh (GRUB var resolution)
109+
```
110+
111+
## awk
112+
113+
BusyBox awk is minimal but sufficient for Heads usage:
114+
```bash
115+
awk -v dev="$dev" 'index($1, dev) == 1 { print $3; exit }' /proc/mounts
116+
awk '{print $2}' /proc/mounts
117+
```
118+
119+
## cut / head / tr / uniq / fold / basename / dirname / readlink
120+
121+
All identical for Heads usage. No special BusyBox workarounds needed.
122+
123+
## sort
124+
125+
**BusyBox quirk**: `sort -k` keyed sort with `-u` deduplicates based on the
126+
**entire line**, not the sort key. GNU sort deduplicates by the key alone.
127+
128+
**Heads pattern**: Use `awk -F'|' '!seen[$1]++'` instead of `sort -t\| -k1 -u`
129+
when deduplicating by a single field:
130+
```bash
131+
# Wrong under BusyBox (dedups by full line, not field 1):
132+
sort -t\| -k1 -u
133+
134+
# Correct under both:
135+
awk -F'|' '!seen[$1]++'
136+
```
137+
138+
## cat / mv / rm / mkdir / mktemp / printf / wc / xargs / echo
139+
140+
All identical. No BusyBox workarounds needed.
141+
142+
## Summary of required BusyBox workarounds
143+
144+
| Command | Workaround | Where |
145+
|---------|------------|-------|
146+
| `xxd -p` | `tr -d '\n '` strips 60-col padding | `unpack_initramfs.sh:38,68` |
147+
| `xxd -p -r` | `fold -w 60 \| xxd -p -r` | `etc/functions.sh:2701` |
148+
| `cpio` trailing data exit | `|| true` swallows GNU exit 2 | `unpack_initramfs.sh:52,101` |
149+
| `grep -a` | Omit or keep (no-op on both) | `unpack_initramfs.sh:78` |
150+
| `zstd` not available | `(zstd-decompress -d \|\| zstd -d \|\| true)` fails silently | `unpack_initramfs.sh:119` |

doc/index.md

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Heads documentation index
2+
3+
Quick reference — read the relevant doc when working on a topic.
4+
5+
| File | What it covers |
6+
|------|----------------|
7+
| `architecture.md` | System architecture: coreboot → kernel → initrd, build system, config hierarchy |
8+
| `boot-process.md` | Boot flow stages, ISO boot layers (Layer 1/2/3), [OK]/[!] marker legend |
9+
| `busybox_perks.md` | GNU vs BusyBox command differences for all tools used in initrd scripts |
10+
| `docker.md` | Docker-based build environment |
11+
| `logging.md` | Log levels (STATUS, WARN, NOTE, INFO, DEBUG, TRACE) usage conventions |
12+
| `modules.md` | Available tools: which are BusyBox applets vs standalone binaries |
13+
| `security-model.md` | TPM, measured boot, trust chain |
14+
| `tpm.md` | TPM 1.2 and 2.0 operations |
15+
| `ux-patterns.md` | User interaction patterns (whiptail, CLI menu, confirm dialogs) |
16+
| `iso_boot.md` | ISO boot parameter reference — what each kernel param does and which framework uses it |

0 commit comments

Comments
 (0)