Skip to content

Commit 48a3ea6

Browse files
ZhiXiao-Linclaude
ZhiXiao-Lin
and
claude
committed
docs(blog): add English version of libkrun/libkrunfw/WHPX article
Translate the technical deep-dive blog post to English for international audience. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent efad6a7 commit 48a3ea6

1 file changed

Lines changed: 281 additions & 0 deletions

File tree

‎docs/blog/2026-02-27-libkrun-libkrunfw-whpx-en.md‎

Lines changed: 281 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,281 @@
1+
# libkrun & libkrunfw: A Technical Deep Dive into Lightweight Virtualization
2+
3+
> From Linux/macOS to Windows: How a Dynamic Library Makes Process Isolation Accessible
4+
5+
---
6+
7+
## Background: Why Lightweight Virtualization?
8+
9+
Container technologies (like Docker) achieve process isolation through Linux namespaces and cgroups, but they share the host kernel, creating kernel escape security risks. Traditional virtual machines (like QEMU/KVM) provide strong isolation but are slow to start and resource-intensive, making them unsuitable for container scenarios.
10+
11+
**libkrun** takes a middle path: using virtualization technology to provide kernel-level isolation while maintaining container-like lightweight characteristics.
12+
13+
---
14+
15+
## What is libkrun?
16+
17+
libkrun is a **lightweight VMM (Virtual Machine Monitor) in dynamic library form** that allows any program to gain hardware virtualization-based process isolation capabilities simply by linking to this library.
18+
19+
```
20+
Application (crun / krunkit / muvm / a3s box)
21+
↓ links to
22+
libkrun.so / libkrun.dylib
23+
↓ calls
24+
KVM (Linux) / HVF (macOS) / WHPX (Windows)
25+
↓ runs
26+
Guest VM (isolated process)
27+
```
28+
29+
Its C API is extremely simple:
30+
31+
```c
32+
// Create a VM context
33+
uint32_t ctx = krun_create_ctx();
34+
35+
// Configure resources
36+
krun_set_vm_config(ctx, 2, 512); // 2 vCPUs, 512MB memory
37+
krun_set_root(ctx, "/path/to/rootfs"); // Root filesystem
38+
krun_set_exec(ctx, "/bin/sh", args, env); // Program to run
39+
40+
// Start VM (blocks until VM exits)
41+
krun_start_enter(ctx);
42+
```
43+
44+
### Core Components
45+
46+
libkrun internally integrates a complete VMM, including:
47+
48+
| Component | Purpose |
49+
|-----------|---------|
50+
| vCPU Management | Create, run, destroy virtual CPUs |
51+
| Memory Management | Allocate guest physical memory |
52+
| Device Emulation | virtio devices (console, fs, net, block, etc.) |
53+
| Interrupt Controller | Emulate APIC/GIC |
54+
| Boot Loader | Load kernel into guest memory and boot |
55+
56+
---
57+
58+
## What is libkrunfw?
59+
60+
libkrun needs a Linux kernel to run guest processes. **libkrunfw** is the carrier for this kernel—it packages a specially configured Linux kernel as a dynamic library.
61+
62+
### Clever Design
63+
64+
Traditional approaches require reading kernel files from disk, but libkrunfw leverages dynamic linker features:
65+
66+
```
67+
libkrunfw.so.5
68+
├── .data section: kernel binary image (vmlinux)
69+
├── krunfw_get_kernel(): returns pointer and size of kernel image
70+
└── krunfw_get_initrd(): returns initrd pointer (TEE variant)
71+
```
72+
73+
When libkrun loads libkrunfw, the dynamic linker directly maps the kernel image into the process address space. libkrun gets the pointer and directly injects this memory into the guest's physical memory, without any file I/O.
74+
75+
```rust
76+
// Code in libkrun for loading libkrunfw
77+
static KRUNFW: LazyLock<Option<libloading::Library>> =
78+
LazyLock::new(|| unsafe { libloading::Library::new("libkrunfw.so.5").ok() });
79+
80+
// Get kernel image pointer
81+
let get_kernel: Symbol<unsafe extern "C" fn(*mut u64, *mut u64, *mut size_t) -> *mut c_char>
82+
= krunfw.get(b"krunfw_get_kernel")?;
83+
```
84+
85+
### What's Special About the Kernel in libkrunfw?
86+
87+
The kernel in libkrunfw is not a standard distribution kernel; it includes special patches:
88+
89+
1. **TSI (Transparent Socket Impersonation) patches**: Allow the guest kernel to transparently forward socket calls to the VMM proxy, enabling network connectivity without virtual NICs.
90+
2. **Minimized configuration**: Removes many unnecessary drivers and features, reducing size and speeding up boot time.
91+
3. **CPU count limit**: `CONFIG_NR_CPUS=8`, optimized for lightweight scenarios.
92+
93+
### Multiple Variants
94+
95+
| Variant | Library Name | Purpose |
96+
|---------|--------------|---------|
97+
| Standard | `libkrunfw.so.5` | General virtualization |
98+
| SEV | `libkrunfw-sev.so.5` | AMD memory encryption |
99+
| TDX | `libkrunfw-tdx.so.5` | Intel Trust Domain Extensions |
100+
101+
---
102+
103+
## Technical Value
104+
105+
### 1. Extremely Low Barrier to Entry
106+
107+
No need to understand KVM ioctls, QEMU configuration, or any virtualization details. A single C function call can start an isolated VM. This allows projects like [crun](https://github.com/containers/crun), [krunkit](https://github.com/containers/krunkit), and [muvm](https://github.com/AsahiLinux/muvm) to easily gain virtualization isolation capabilities.
108+
109+
### 2. Fast Boot Time
110+
111+
Since the kernel is already mapped into memory through the dynamic linker, disk read time is eliminated. Combined with the kernel's minimized configuration, the entire VM boot time can be controlled at the millisecond level.
112+
113+
### 3. Strong Security Isolation
114+
115+
Compared to namespace isolation, libkrun provides hardware-level isolation boundaries. Even if the guest kernel has vulnerabilities, attackers need to break through the hypervisor layer to affect the host.
116+
117+
### 4. TSI Network Innovation
118+
119+
Traditional VMs require virtual NICs + userspace network proxies (like passt/gvproxy) for networking, with complex configuration. TSI intercepts socket system calls in the guest kernel and transparently forwards network requests to the VMM, without any virtual network devices, greatly simplifying network configuration.
120+
121+
### 5. Cross-Platform Capability
122+
123+
The same API uses KVM on Linux, HVF on macOS, and WHPX on Windows, with upper-layer applications unaware of underlying differences.
124+
125+
---
126+
127+
## Limitations
128+
129+
### Technical Limitations
130+
131+
**1. Dependency on Custom Kernel**
132+
Core features like TSI require the custom kernel in libkrunfw; standard distribution kernels cannot be used. This means kernel version and features are determined by libkrunfw, and users cannot freely choose.
133+
134+
**2. Limited Workload Compatibility**
135+
libkrun is designed to run single processes, not general-purpose VMs. It does not support:
136+
- Workloads requiring special kernel modules
137+
- OS installations requiring UEFI/BIOS (except EFI variant)
138+
- Scenarios requiring PCI passthrough
139+
140+
**3. CPU Count Ceiling**
141+
The libkrunfw kernel limits to a maximum of 8 vCPUs, unsuitable for high-concurrency compute-intensive scenarios.
142+
143+
**4. TDX Variant Memory Limit**
144+
The Intel TDX variant supports a maximum of 3072MB memory and 1 vCPU, with significant limitations.
145+
146+
### Security Model Limitations
147+
148+
libkrun's security model treats the guest and VMM as the same security context. Host resources accessible to the VMM can theoretically be accessed by the guest through the VMM. To achieve true isolation, restrictions must be applied to the VMM process itself at the host level (such as Linux namespaces).
149+
150+
### Platform Limitations
151+
152+
- **Windows support is still experimental**: WHPX backend is under development
153+
- **macOS only supports ARM64**: x86_64 macOS is not supported (HVF limitation)
154+
- **Linux requires KVM support**: Cannot be used in VMs or environments without KVM
155+
156+
---
157+
158+
## WHPX: Enabling libkrun on Windows
159+
160+
### What is WHPX?
161+
162+
**Windows Hypervisor Platform (WHPX)** is a userspace virtualization API introduced by Microsoft in Windows 10 version 2004, exposed to applications through `WinHvPlatform.dll`. It is the userspace interface to Hyper-V, similar to Linux's KVM and macOS's HVF.
163+
164+
```
165+
Windows Application
166+
↓
167+
WinHvPlatform API (userspace)
168+
↓
169+
Hyper-V Hypervisor (kernel)
170+
↓
171+
Hardware Virtualization (Intel VT-x / AMD-V)
172+
```
173+
174+
### WHPX Core APIs
175+
176+
| API | Purpose |
177+
|-----|---------|
178+
| `WHvCreatePartition` | Create VM partition |
179+
| `WHvSetupPartition` | Configure partition parameters |
180+
| `WHvMapGpaRange` | Map guest physical memory |
181+
| `WHvCreateVirtualProcessor` | Create vCPU |
182+
| `WHvRunVirtualProcessor` | Run vCPU until VM exit |
183+
| `WHvGetVirtualProcessorRegisters` | Read vCPU registers |
184+
| `WHvSetVirtualProcessorRegisters` | Write vCPU registers |
185+
| `WHvDeleteVirtualProcessor` | Destroy vCPU |
186+
| `WHvDeletePartition` | Destroy partition |
187+
188+
### VM Exit Handling Mechanism
189+
190+
WHPX uses a **synchronous VM exit** model: each time the guest executes an operation requiring VMM intervention, `WHvRunVirtualProcessor` returns, the VMM handles it, and then calls again to continue execution.
191+
192+
```rust
193+
// Core logic of WHPX vCPU run loop in libkrun
194+
pub fn run(&mut self) -> io::Result<VcpuExit<'_>> {
195+
let mut exit_context = WHV_RUN_VP_EXIT_CONTEXT::default();
196+
197+
unsafe {
198+
WHvRunVirtualProcessor(
199+
self.partition,
200+
self.index,
201+
&mut exit_context as *mut _,
202+
size_of::<WHV_RUN_VP_EXIT_CONTEXT>() as u32,
203+
)?;
204+
}
205+
206+
// Parse exit reason
207+
match exit_context.ExitReason {
208+
WHV_RUN_VP_EXIT_REASON_MEMORY_ACCESS => { /* MMIO handling */ }
209+
WHV_RUN_VP_EXIT_REASON_X64_IO_PORT_ACCESS => { /* IO port handling */ }
210+
WHV_RUN_VP_EXIT_REASON_X64_HALT => Ok(VcpuExit::Halted),
211+
WHV_RUN_VP_EXIT_REASON_CANCELED => Ok(VcpuExit::Shutdown),
212+
_ => Ok(VcpuExit::Shutdown),
213+
}
214+
}
215+
```
216+
217+
### libkrun WHPX Backend Architecture
218+
219+
To support libkrun on Windows, we implemented the following architecture:
220+
221+
```
222+
Vcpu::run() [vstate.rs]
223+
↓ loops calling
224+
WhpxVcpu::run() [whpx_vcpu.rs]
225+
↓ calls WHvRunVirtualProcessor
226+
↓ returns VcpuExit enum
227+
Vcpu::run_emulation()
228+
↓ dispatches based on exit type
229+
├── MmioRead/MmioWrite → mmio_bus (device emulation)
230+
├── IoPortRead/IoPortWrite → mmio_bus (IO port devices)
231+
├── Halted → VcpuEmulation::Halted (stop)
232+
└── Shutdown → VcpuEmulation::Stopped (stop)
233+
```
234+
235+
The **VcpuExit enum** design uses Rust's lifetime parameters to ensure memory safety of MMIO/IO data buffers:
236+
237+
```rust
238+
pub enum VcpuExit<'a> {
239+
MmioRead(u64, &'a mut [u8]), // address + mutable buffer (device fills data)
240+
MmioWrite(u64, &'a [u8]), // address + data
241+
IoPortRead(u16, &'a mut [u8]), // port + mutable buffer
242+
IoPortWrite(u16, &'a [u8]), // port + data
243+
Halted,
244+
Shutdown,
245+
}
246+
```
247+
248+
### Comparison with KVM/HVF
249+
250+
| Feature | KVM (Linux) | HVF (macOS) | WHPX (Windows) |
251+
|---------|-------------|-------------|----------------|
252+
| API Level | Kernel ioctl | Userspace framework | Userspace DLL |
253+
| Memory Mapping | `KVM_SET_USER_MEMORY_REGION` | `hv_vm_map` | `WHvMapGpaRange` |
254+
| vCPU Execution | `KVM_RUN` ioctl | `hv_vcpu_run` | `WHvRunVirtualProcessor` |
255+
| Exit Information | `kvm_run` shared memory | `hv_vcpu_exit_t` | `WHV_RUN_VP_EXIT_CONTEXT` |
256+
| Register Access | `KVM_GET/SET_REGS` | `hv_vcpu_get/set_reg` | `WHvGet/SetVirtualProcessorRegisters` |
257+
| Minimum Requirements | Linux + KVM module | macOS 11+ ARM64 | Windows 10 2004+ + Hyper-V |
258+
259+
### Significance of Windows Support
260+
261+
The WHPX backend implementation makes libkrun a truly cross-platform virtualization library:
262+
263+
1. **Windows Container Ecosystem**: Provides virtualization isolation capabilities for container runtimes on Windows (like Windows version of crun)
264+
2. **Development Environment Consistency**: Developers can use the same isolation mechanism on Windows as on Linux/macOS
265+
3. **a3s box Cross-Platform**: Enables a3s box to provide consistent security isolation experience on Windows as on other platforms
266+
267+
---
268+
269+
## Summary
270+
271+
libkrun and libkrunfw together form an elegant lightweight virtualization solution:
272+
273+
- **libkrunfw** solves the "where does the kernel come from" problem—packaging the kernel as a dynamic library, using the linker for zero-overhead loading
274+
- **libkrun** solves the "how to use virtualization" problem—wrapping complex VMM into a simple C API
275+
- **WHPX backend** solves the "what about Windows" problem—implementing virtualization capabilities equivalent to KVM/HVF through Windows Hypervisor Platform API
276+
277+
This design philosophy—**minimization, self-containment, cross-platform**—makes libkrun an important infrastructure in the container security isolation field.
278+
279+
---
280+
281+
*This article is based on libkrun source code analysis. Some implementation details are subject to the code.*

0 commit comments

Comments
 (0)