os: add experimental reproducible mkosi backend

Author: kvinwang

os/README.md

@@ -9,15 +9,17 @@ os/
 ├── common/rootfs/            # backend-independent guest payload
 ├── image/                    # backend-independent release assembler
 ├── spec/                     # versioned backend artifact contract
-└── yocto/                    # the currently implemented backend
+├── mkosi/                    # experimental pinned Debian/mkosi backend
+└── yocto/                    # production backend
     ├── deps/                 # external Yocto layers (git submodules)
     ├── layers/               # dstack-owned Yocto layers
     └── tools/                # Yocto-specific host workarounds
 ```
 
-A future backend can be added at `os/<backend>/` (for example `os/mkosi/`)
-without moving shared payload or duplicating release packaging. No mkosi
-backend is implemented today.
+A backend can be added at `os/<backend>/` without moving shared payload or
+duplicating release packaging. The mkosi backend is experimental but implements
+the same artifact-manifest and common release-assembly contract as Yocto; see
+[`mkosi/README.md`](mkosi/README.md) for its scope and acceptance criteria.
 
 ## Build
 

os/build.sh

@@ -13,6 +13,7 @@ Build dstack guest OS release artifacts with a selected backend.
 
 Available backends:
   yocto    Reproducible Yocto build (default)
+  mkosi    Experimental pinned Debian/mkosi build
 
 A backend lives at os/<name>/build.sh and implements the "image" action.
 The common image contract is documented in os/README.md.

os/mkosi/.gitignore

@@ -0,0 +1,5 @@
+build*/
+repro*/
+*.raw
+*.raw.zst
+

os/mkosi/README.md

@@ -0,0 +1,59 @@
+# Experimental mkosi backend
+
+This is a deliberately small Debian/mkosi implementation of the dstack guest
+OS. It is an experiment, not yet a replacement for the release Yocto backend.
+It tracks the Yocto image's confidential-computing, container, storage,
+network, TPM, SSH, chrony and dstack guest-service capabilities.
+
+## Reproducibility model
+
+`versions.env` pins the stable 7.x kernel tarball by SHA-256 and selects an
+immutable Debian snapshot. `Cargo.lock`, `--locked --offline`, a fixed
+`SOURCE_DATE_EPOCH`, normalized file mtimes, fixed kernel build identity and
+mkosi's deterministic image construction close the remaining inputs. The two
+dstack binaries therefore require an already populated Cargo cache; use a
+vendored source tree in hermetic CI. `DSTACK_SKIP_RUST=1` exists only for
+rootfs/kernel development and does **not** produce a functional guest.
+
+The custom kernel starts from `x86_64_defconfig`, then applies the reviewed
+`kernel.config` fragment. This is the practical upstream equivalent of Yocto's
+`linux-yocto-tiny` plus explicit features: disabling arbitrary defconfig
+symbols without resolving Kconfig dependencies would be less auditable. Both
+the TDX DMA patch and the ACPI BadAML/SystemMemory sandbox patch are reused
+verbatim from `meta-dstack`; patch fuzz is forbidden. The final `.config` is
+checked before compilation.
+
+## Build and acceptance
+
+Host requirements include mkosi >= 26, systemd tools, a C/Rust kernel and EDK2
+build toolchain, `bc`, `bison`, `flex`, `nasm`, `iasl`, OpenSSL/ELF/UUID
+development headers, `patch`, `pax-utils` (`lddtree`), `squashfs-tools`,
+`cryptsetup`, `gdisk`, `dosfstools`, `mtools`, `curl`, `xz`, QEMU/KVM and root
+privileges (or a working user namespace). Full UKI release assembly also needs
+the pinned `nitro-tpm-pcr-compute` described by `os/image/assemble.sh`.
+
+```sh
+./os/mkosi/build.sh lint
+./os/build.sh --backend mkosi --build-dir "$PWD/os/mkosi/build"
+./os/mkosi/build.sh repro-check "$PWD/os/mkosi/repro"
+# QEMU smoke-test the assembled UKI disk (host OVMF path is distro-specific)
+qemu-system-x86_64 -machine q35 -m 2G -nographic \
+  -drive if=pflash,format=raw,readonly=on,file=/usr/share/OVMF/OVMF_CODE_4M.fd \
+  -drive if=virtio,format=raw,file=os/mkosi/build/out/prod/dstack-0.6.0/disk.raw
+```
+
+Acceptance means: the static contract passes; a disk with systemd-boot/UKI
+boots on x86_64 QEMU; `/proc/config.gz` contains the checked TDX/SNP, TPM,
+ACPI, dm-verity/crypt, virtio, container and hardening options; dstack services
+are enabled; and two clean builds compare byte-for-byte. The backend exports
+artifact-manifest schema v1 and delegates final assembly to
+`os/image/assemble.sh`, exactly like Yocto. Its output contains the same
+`dstack-0.6.0/` directory, bare-metal and UKI tarballs, partitioned combined
+squashfs/dm-verity image, metadata, measurements, checksums, kernel, initramfs,
+OVMF and UKI. Package contents differ because this backend uses Debian, but
+consumers see the same release format and partition labels.
+
+The firmware is not Debian's generic OVMF: `build-ovmf.sh` builds the same
+EDK2 stable-202502 revision and `pre202505` TDX measurement layout selected by
+the Yocto recipe. A generic OVMF cannot be substituted because `dstack-mr`
+would produce invalid or unparseable TDX measurement material.

os/mkosi/build.sh

@@ -0,0 +1,69 @@
+#!/bin/bash
+# SPDX-License-Identifier: Apache-2.0
+set -euo pipefail
+SELF=$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)
+ROOT=$(cd "$SELF/../.." && pwd)
+source "$SELF/versions.env"
+action=${1:-image}
+BUILD_DIR=${2:-$SELF/build}
+case "$action" in
+  image|repro-check|lint) ;;
+  *) echo "Usage: $0 {image|repro-check|lint} [build-dir]" >&2; exit 2 ;;
+esac
+if [[ $action == lint ]]; then exec "$SELF/tests/acceptance.sh"; fi
+command -v mkosi >/dev/null || { echo 'mkosi >= 26 is required' >&2; exit 1; }
+actual=$(mkosi --version | awk '{print $2}' | cut -d. -f1)
+(( actual >= MKOSI_MIN_VERSION )) || { echo "mkosi >= $MKOSI_MIN_VERSION required" >&2; exit 1; }
+export SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH:-$(git -C "$ROOT" log -1 --format=%ct)}
+export TZ=UTC LC_ALL=C
+
+build_one() {
+  local out=$1 work=$2 flavor=$3
+  local stage="$work/rootfs-stage" kstage="$work/kernel-stage" tree="$work/rootfs"
+  rm -rf "$work" "$out"; mkdir -p "$stage" "$kstage" "$out"
+  "$SELF/scripts/stage-rootfs.sh" "$stage"
+  "$SELF/scripts/build-kernel.sh" "$work" "$kstage"
+  "$SELF/scripts/build-ovmf.sh" "$work/ovmf-build" "$kstage/ovmf.fd"
+  cat > "$work/mkosi.local.conf" <<EOF
+[Content]
+ExtraTrees=$stage
+Bootable=no
+[Output]
+Format=directory
+OutputDirectory=$work
+Output=rootfs
+CompressOutput=no
+EOF
+  if [[ $flavor == dev ]]; then
+    cat >> "$work/mkosi.local.conf" <<EOF
+[Content]
+Packages=strace,tcpdump,gdb,vim
+EOF
+  fi
+  devargs=()
+  if [[ $flavor == dev ]]; then
+    devargs+=(--package=strace --package=tcpdump --package=gdb --package=vim)
+  fi
+  mkosi --directory "$SELF" --force --extra-tree="$stage" "${devargs[@]}" \
+    --format=directory --output-directory="$work" --output=rootfs \
+    --compress-output=no --bootable=no build
+  mkdir -p "$tree/usr/lib/modules"
+  cp -a "$kstage/usr/lib/modules/." "$tree/usr/lib/modules/"
+  artifact_dir="$work/artifacts/$flavor"
+  "$SELF/scripts/make-release-artifacts.sh" "$tree" "$kstage" "$artifact_dir" "$flavor"
+  DIST_DIR="$out" \
+    TAR_OPTIONS="--sort=name --mtime=@$SOURCE_DATE_EPOCH --owner=0 --group=0 --numeric-owner" \
+    "$ROOT/os/image/assemble.sh" --manifest "$artifact_dir/artifact-manifest.json"
+  release_name=dstack
+  [[ $flavor == dev ]] && release_name=dstack-dev
+  "$SELF/tests/check-output.sh" "$out/$release_name-$DSTACK_VERSION" "$flavor"
+}
+
+if [[ $action == image ]]; then
+  for flavor in $FLAVORS; do build_one "$BUILD_DIR/out/$flavor" "$BUILD_DIR/work-$flavor" "$flavor"; done
+  exit
+fi
+build_one "$BUILD_DIR/a" "$BUILD_DIR/work-a" prod
+build_one "$BUILD_DIR/b" "$BUILD_DIR/work-b" prod
+cmp "$BUILD_DIR/a/dstack-0.6.0.tar.gz" "$BUILD_DIR/b/dstack-0.6.0.tar.gz"
+echo 'reproducibility check passed'

os/mkosi/kernel.config

@@ -0,0 +1,120 @@
+# dstack 7.x kernel: x86_64 defconfig baseline plus explicitly audited guest features.
+CONFIG_LOCALVERSION="-dstack"
+CONFIG_LOCALVERSION_AUTO=n
+CONFIG_MODULES=y
+CONFIG_IKCONFIG=y
+CONFIG_IKCONFIG_PROC=y
+CONFIG_EFI=y
+CONFIG_EFI_STUB=y
+CONFIG_EFI_PARTITION=y
+CONFIG_ACPI=y
+CONFIG_ACPI_TABLE_UPGRADE=y
+CONFIG_HYPERVISOR_GUEST=y
+CONFIG_PARAVIRT=y
+CONFIG_KVM_GUEST=y
+CONFIG_INTEL_TDX_GUEST=y
+CONFIG_TDX_GUEST_DRIVER=y
+CONFIG_TSM_REPORTS=y
+CONFIG_AMD_MEM_ENCRYPT=y
+CONFIG_SEV_GUEST=y
+CONFIG_EFI_COCO_SECRET=y
+CONFIG_EFI_SECRET=y
+CONFIG_VMGENID=y
+CONFIG_VIRT_DRIVERS=y
+CONFIG_TCG_TPM=y
+CONFIG_TCG_TIS=y
+CONFIG_TCG_CRB=y
+CONFIG_VIRTIO=y
+CONFIG_VIRTIO_PCI=y
+CONFIG_VIRTIO_BLK=y
+CONFIG_VIRTIO_NET=y
+CONFIG_VIRTIO_CONSOLE=y
+CONFIG_VIRTIO_VSOCKETS=y
+CONFIG_VSOCKETS=y
+CONFIG_NET_9P=y
+CONFIG_NET_9P_VIRTIO=y
+CONFIG_9P_FS=y
+CONFIG_BLK_DEV_NVME=y
+CONFIG_NVME_CORE=y
+CONFIG_NET_VENDOR_GOOGLE=y
+CONFIG_GVE=y
+CONFIG_NET_VENDOR_AMAZON=y
+CONFIG_ENA_ETHERNET=y
+CONFIG_SERIAL_8250=y
+CONFIG_SERIAL_8250_CONSOLE=y
+CONFIG_MD=y
+CONFIG_BLK_DEV_DM=y
+CONFIG_DM_CRYPT=y
+CONFIG_DM_VERITY=y
+CONFIG_CRYPTO_SHA256=y
+CONFIG_CRYPTO_SHA512=y
+CONFIG_CRYPTO_AES_NI_INTEL=y
+CONFIG_EXT4_FS=y
+CONFIG_SQUASHFS=y
+CONFIG_SQUASHFS_ZSTD=y
+CONFIG_XFS_FS=m
+CONFIG_BTRFS_FS=m
+CONFIG_OVERLAY_FS=y
+CONFIG_OVERLAY_FS_METACOPY=y
+CONFIG_FUSE_FS=m
+CONFIG_BLK_DEV_LOOP=y
+CONFIG_TMPFS_XATTR=y
+CONFIG_TMPFS_POSIX_ACL=y
+CONFIG_QUOTA=y
+CONFIG_QFMT_V2=y
+CONFIG_CGROUPS=y
+CONFIG_BLK_CGROUP=y
+CONFIG_CGROUP_BPF=y
+CONFIG_CGROUP_CPUACCT=y
+CONFIG_CGROUP_DEVICE=y
+CONFIG_CGROUP_FREEZER=y
+CONFIG_CGROUP_SCHED=y
+CONFIG_CFS_BANDWIDTH=y
+CONFIG_NAMESPACES=y
+CONFIG_USER_NS=y
+CONFIG_SECCOMP=y
+CONFIG_SECCOMP_FILTER=y
+CONFIG_BPF=y
+CONFIG_BPF_SYSCALL=y
+CONFIG_BPF_JIT=y
+CONFIG_BRIDGE=m
+CONFIG_BRIDGE_NETFILTER=m
+CONFIG_VETH=m
+CONFIG_VLAN_8021Q=m
+CONFIG_TUN=m
+CONFIG_WIREGUARD=m
+CONFIG_NETFILTER=y
+CONFIG_NETFILTER_ADVANCED=y
+CONFIG_NF_TABLES=m
+CONFIG_NFT_COMPAT=m
+CONFIG_NF_NAT=m
+CONFIG_NETFILTER_XTABLES=m
+CONFIG_IP_SET=m
+CONFIG_IP_VS=m
+CONFIG_IPVLAN=m
+CONFIG_VXLAN=m
+CONFIG_CONFIGFS_FS=y
+CONFIG_SECURITY=y
+CONFIG_SECURITY_DMESG_RESTRICT=y
+CONFIG_HARDENED_USERCOPY=y
+CONFIG_FORTIFY_SOURCE=y
+CONFIG_INIT_ON_ALLOC_DEFAULT_ON=y
+CONFIG_SLAB_FREELIST_RANDOM=y
+CONFIG_SLAB_FREELIST_HARDENED=y
+CONFIG_IO_STRICT_DEVMEM=y
+CONFIG_STACKPROTECTOR_STRONG=y
+CONFIG_STRICT_KERNEL_RWX=y
+CONFIG_STRICT_MODULE_RWX=y
+CONFIG_RANDOMIZE_BASE=y
+CONFIG_CPU_MITIGATIONS=y
+CONFIG_MITIGATION_PAGE_TABLE_ISOLATION=y
+CONFIG_MITIGATION_RETPOLINE=y
+CONFIG_LEGACY_TIOCSTI=n
+CONFIG_MAGIC_SYSRQ=n
+CONFIG_INET_ESP=n
+CONFIG_INET6_ESP=n
+CONFIG_AF_RXRPC=n
+CONFIG_SUSPEND=n
+CONFIG_BT=n
+CONFIG_WLAN=n
+CONFIG_MMC=n

os/mkosi/mkosi.conf

@@ -0,0 +1,61 @@
+# SPDX-License-Identifier: Apache-2.0
+[Config]
+MinimumVersion=26
+
+[Distribution]
+Distribution=debian
+Release=trixie
+Architecture=x86-64
+Snapshot=20260721T000000Z
+Repositories=main,contrib,non-free-firmware
+
+[Output]
+Format=disk
+Output=dstack-mkosi.raw
+CompressOutput=no
+ImageId=dstack
+ImageVersion=0.1
+
+[Content]
+Bootable=yes
+Bootloader=systemd-boot
+Hostname=dstack
+Packages=
+        systemd
+        systemd-boot
+        systemd-resolved
+        dbus
+        ca-certificates
+        bash
+        coreutils
+        iproute2
+        iptables
+        nftables
+        docker.io
+        docker-compose
+        containerd
+        chrony
+        openssh-server
+        wireguard-tools
+        cryptsetup
+        dmsetup
+        tpm2-tools
+        curl
+        jq
+        fuse3
+        pigz
+        xfsprogs
+        e2fsprogs
+        gdisk
+        parted
+        zfsutils-linux
+        busybox-static
+        cryptsetup-bin
+        ovmf
+        systemd-boot-efi
+SkeletonTrees=mkosi.skeleton
+PostInstallationScripts=mkosi.postinst
+KernelCommandLine=console=ttyS0,115200n8 console=tty0 panic=30 coherent_pool=64M
+
+[Validation]
+SecureBoot=no

os/mkosi/mkosi.postinst

@@ -0,0 +1,21 @@
+#!/bin/bash
+# SPDX-License-Identifier: Apache-2.0
+set -euo pipefail
+
+# mkosi executes post-installation scripts outside the image with BUILDROOT.
+B=${BUILDROOT:?}
+mkdir -p "$B/dev" "$B/proc" "$B/sys" "$B/run" "$B/tmp" "$B/dstack" \
+  "$B/etc/wireguard" "$B/var/lib/docker" "$B/var/lib/containerd"
+rm -f "$B/etc/machine-id" "$B/var/lib/dbus/machine-id"
+: > "$B/etc/machine-id"
+rm -rf "$B/var/cache/apt/"* "$B/var/lib/apt/lists/"* "$B/var/log/"*
+ln -sfn /run/systemd/resolve/stub-resolv.conf "$B/etc/resolv.conf"
+ln -sfn dstack "$B/tapp"
+ln -sfn ../lib/systemd/systemd "$B/usr/sbin/init"
+
+systemctl --root="$B" enable systemd-networkd systemd-resolved chrony docker \
+    dstack-firstboot.service dstack-prepare.service \
+    dstack-guest-agent.socket dstack-guest-agent.service \
+    app-compose.service wg-checker.service
+systemctl --root="$B" mask getty@.service serial-getty@.service debug-shell.service \
+    sleep.target suspend.target hibernate.target hybrid-sleep.target

os/mkosi/mkosi.skeleton/etc/apt/apt.conf.d/99dstack-reproducible

@@ -0,0 +1,4 @@
+Acquire::Check-Valid-Until "false";
+APT::Install-Recommends "false";
+APT::Install-Suggests "false";
+

os/mkosi/mkosi.skeleton/etc/docker/daemon.json

@@ -0,0 +1,6 @@
+{
+  "iptables": true,
+  "ip6tables": true,
+  "storage-driver": "overlay2"
+}
+