runtime: Add annotation-based block device mounting by antoine-gaillard · Pull Request #40 · DataDog/kata-containers

antoine-gaillard · 2026-01-21T20:01:58Z

Problem: CSI block mode PVCs are passed as raw device nodes. Containers must mount them manually, requiring privileged: true or CAP_SYS_ADMIN. Additionally, new volumes without a snapshot (e.g. fresh EBS) arrive unformatted, and the guest rootfs doesn't ship mkfs/blkid.

Solution: New annotation instructs kata-agent to mount the device:

annotations:
  io.katacontainers.volume.block-mounts: '{"/dev/xvda": {"mount": "/data", "fstype": "ext4"}}'

How it works:

1. Block device hotplugged via existing volumeDevices path
2. Runtime checks host-side device with blkid — if unformatted, runs mkfs.<fstype> on the host before passing to the agent
3. Runtime parses annotation → creates grpc.Storage objects
4. Device removed from OCI spec (no raw device in container)
5. Bind mount added → container sees mounted filesystem

Options:

- mount: destination path (required)
- fstype: ext4, xfs (default: ext4)
- options: mount options (default: ["rw"])
- fsGroup: optional gid for ownership

Testing:

- 27 unit tests (parsing, validation, storage creation, host-side formatting)
- Live tested on cluster with CSI block PVC + fresh unformatted EBS and EBS with existing data

zaymat

overall lgtm. The logic to check we're not mount devices outside of what's defined in the pod spec seems ok.

Small nit on optimization.

Now let's do the Rust part 😁

zaymat · 2026-02-19T17:46:37Z

+	var storages []*grpc.Storage
+	devicesToRemove := make(map[string]bool)
+
+	for devicePath, mountConfig := range blockMounts {


small optimization: if you instead go through container's devices and check if you found an entry for dev.ContainerPath in the blockMounts map, you go through each device once.

With the current option, you have a (# device)*(# auto mount) complexity.

microVM sandbox resources are computed from pod sandbox annotations. In particular, the number of vCPU is calculated by using CPU quota divided by CPU period. However, on clusters where CFS quotas are disabled, or if the pod doesn't specify any limit, the compute size is 0. When using resource hot pluging, the value value will be the size of the CPU set, which doesn't impact the performance of the microVM pod. But when using static sandbox management, the computed value will be 0 and the microVM will be dramatically undersized. This change takes into account CPU shares while computing the number of vCPU, and default the CPU Shares/1024 in case CPU quota and/or periods are zeros.

Co-authored-by: Maxime VISONNEAU <maxime.visonneau@gmail.com>

- Add scratch-based Dockerfile for kata data volume - Move Dockerfile to docker/ subdir and fix config file handling - Fix Dockerfile to extract only essential kata files - Add containerd runtime dropin configuration files 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>

An early call to closing the stdin channel made the stdout & stderr also closed. This waits for stdout & stderr to be properly finished by reading the whole buffer before closing everything. On the other, this also fixes a race condition where it was impossible to run multiple execs until the other one was over. This moves the lock only where it is necessary without locking exec processes. Fixes kata-containers#10387 Signed-off-by: Maxime Bertin <mbertin@luccasoftware.com> Co-authored-by: Maxime Bertin <mbertin@luccasoftware.com>

The WORKFLOW_TOKEN no longer exists, so artefact uploads fail. Use the built-in token instead.

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>

Add support for [`netkit`](https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/commit/?id=22360fad5889cbefe1eca695b0cc0273ab280b56) network devices similarly to how `veth` devices are currently handled. Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>

Netkit devices in L3 mode have no MAC address and require IP routing instead of L2 bridging. Since L3 routing is not currently implemented, reject these devices early with a clear error message directing users to use netkit L2 mode or veth devices instead. Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>

Add support for mounting block devices (volumeDevices) as filesystems inside the guest VM via annotation. This allows CSI block mode PVCs to be automatically mounted by kata-agent, eliminating the need for privileged containers. Annotation format: io.katacontainers.volume.block-mounts: '{"<devicePath>": {"mount": "<path>", "fstype": "<fs>"}}' Example: io.katacontainers.volume.block-mounts: '{"/dev/xvda": {"mount": "/data", "fstype": "ext4"}}' Supported options: - mount: destination path in container (required) - fstype: filesystem type - ext4, xfs, btrfs (default: ext4) - options: mount options array (default: ["rw"]) - fsGroup: optional gid for filesystem group ownership Implementation: - Delegates driver/source selection to handleBlockVolume() for DeviceBlock to ensure proper struct-based detection (e.g., blockDrive.Pmem takes precedence over config) - Extracts PCIPath directly for VhostUserBlk devices - removeDevicesFromOCISpec is a plain function (no receiver needed) - Comprehensive test coverage including pmem device test proving struct-based detection works (nvdimm driver despite VirtioBlock config) Fixes netkit_endpoint_test.go compilation by using proper constructors: - Use PciPathFromString() instead of struct literal - Use CcwDeviceFrom() instead of struct literal Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

…ount Fresh ephemeral EBS volumes arrive without a filesystem. The kata-agent fails with EINVAL when trying to mount an unformatted device, and the guest rootfs does not ship mkfs or blkid. Format on the host side in createAnnotationBlockStorages using the host device path from BlockDrive. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Iterate container devices once with map lookup instead of nested loops. Detect duplicate container devices and report unmatched annotation keys with device paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

antoine-gaillard force-pushed the agaillard/block-mount-annotation branch from 01bc551 to f6b741c Compare January 22, 2026 07:15

zaymat reviewed Jan 22, 2026

View reviewed changes

Comment thread src/runtime/virtcontainers/kata_agent.go Outdated

dd-octo-sts Bot force-pushed the datadog branch from b3e0ea1 to d8481bc Compare January 29, 2026 10:04

antoine-gaillard force-pushed the agaillard/block-mount-annotation branch from f6b741c to cbad5c6 Compare February 18, 2026 22:12

zaymat reviewed Feb 19, 2026

View reviewed changes

dd-octo-sts Bot force-pushed the datadog branch from d8481bc to 371561b Compare February 20, 2026 10:04

zaymat and others added 24 commits March 17, 2026 10:04

[local] Create CI for kata-containers

2499a65

[local] Add the dummy module to the kernel to support emissary

00d3931

[local] CI: build kernel 6.8

1700dfb

[local] Generate sbom of the rootfs

8c2f2d0

[local] CI in Github actions

d0ced0a

[local] Switch back to ubuntu 22.04

f8ebc89

[local] Add the option to add new files in the created rootfs

39f4453

[local] Change systemd's max process limit to 131072

b6d85fe

[local] Specific OS_VERSION when building OS

e997fcb

[local] Gzip SBOM file

d53ac06

Update src/runtime/virtcontainers/utils/utils.go

7642667

Co-authored-by: Maxime VISONNEAU <maxime.visonneau@gmail.com>

Fix after review

1f93164

Configuration files fixes

74a0f3e

Bump golang toolchain and set GOTOOLCHAIN to auto to avoid mismatch

8756296

Use GITHUB_TOKEN secret to upload release artefacts (#28)

8cfdc7b

The WORKFLOW_TOKEN no longer exists, so artefact uploads fail. Use the built-in token instead.

Harmonise sysctl between host and guest

78afa22

New compression format for Kata starting 3.21.0

78b469b

add eBPF support

7808ada

add btf

fe33fee

[local] Add libssl-dev to CI image

789ee8b

make it work on arm

294923f

safchain and others added 7 commits March 17, 2026 10:04

add pahole

31994d0

Bump ci image version

4b9c135

Bump CI runner resources

ac9ae9b

Use bootable image of the kernel instead of vmlinux

16309d6

kernel: Enable netkit device support

9166083

Signed-off-by: Hadrien Patte <hadrien.patte@datadoghq.com>

dd-octo-sts Bot force-pushed the datadog branch from 371561b to 4ebe9de Compare March 17, 2026 10:04

antoine-gaillard and others added 5 commits March 30, 2026 09:37

runtime: Only support ext4 and xfs for block mount annotation

dec76dd

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add doc

4861c33

antoine-gaillard force-pushed the agaillard/block-mount-annotation branch from a81694e to 4861c33 Compare March 30, 2026 13:37

dd-octo-sts Bot force-pushed the datadog branch from 4ebe9de to 77e6f1b Compare April 23, 2026 09:48

TheRayquaza mentioned this pull request Apr 30, 2026

runtime-rs: parse block-mounts annotation for volumeDevice passthrough #57

Draft

TheRayquaza force-pushed the datadog branch 2 times, most recently from 177fa81 to af4c8f5 Compare May 11, 2026 15:43

zaymat force-pushed the datadog branch from c860ad9 to 1a42dac Compare May 20, 2026 14:50

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

runtime: Add annotation-based block device mounting#40

runtime: Add annotation-based block device mounting#40
antoine-gaillard wants to merge 36 commits into
datadogfrom
agaillard/block-mount-annotation

antoine-gaillard commented Jan 21, 2026 •

edited

Loading

Uh oh!

Uh oh!

zaymat left a comment

Uh oh!

zaymat Feb 19, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

antoine-gaillard commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

zaymat left a comment

Choose a reason for hiding this comment

Uh oh!

zaymat Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

antoine-gaillard commented Jan 21, 2026 •

edited

Loading