Bug
ostree system with LUKS fails to boot after a systemctl soft-reboot
Adding x-initrd.attach to crypttab when provisioning LUKS on ostree systems would fix the issue.
Operating System Version
Fedora CoreOS 43, CentOS Stream 10 bootc, Fedora Silverblue 43 — any ostree/bootc system with LUKS root.
Ignition Version
All versions that provision LUKS (the storage.luks config section).
Environment
Any — qemu, bare metal, cloud. Reproducible on FCOS 43 qemu with the LUKS root-reprovision kola test.
Expected Behavior
After Ignition provisions a LUKS root device on an ostree system, systemctl soft-reboot should work. The LUKS device should survive soft-reboot shutdown so that /sysroot, /var, and /boot remain available.
Actual Behavior
Soft-reboot fails with var.mount failure and the system drops to emergency shell. The LUKS device is torn down during soft-reboot shutdown because systemd-cryptsetup-generator writes Conflicts=umount.target into the cryptsetup service unit. This causes a cascade: cryptsetup is stopped, the dm-crypt device is removed, /sysroot loses its backing device, and /var (a bind mount from /sysroot) fails to mount.
On non-ostree systems this cascade is harmless — the kernel root mount (-.mount, Perpetual=yes) pins the LUKS device and cryptsetup detach fails with EBUSY. On ostree, /sysroot is a regular mount (not the kernel root), so the detach actually succeeds.
The systemd-cryptsetup-generator skips Conflicts=umount.target when the crypttab entry has the x-initrd.attach option. Ignition writes /etc/crypttab but does not include this option.
Reproduction Steps
- Boot FCOS with an Ignition config that provisions LUKS root (e.g., the
root-reprovision/luks kola test config with storage.luks + clevis TPM2)
- After provisioning completes and the system reboots, run
systemctl soft-reboot
- System drops to emergency shell —
var.mount fails
Other Information
Proposed fix
When Ignition provisions LUKS on an ostree system (detected via /run/ostree-booted), add x-initrd.attach to the crypttab entry's options field in createCrypttabEntries(). This tells systemd-cryptsetup-generator not to add Conflicts=umount.target, preventing the teardown cascade during soft-reboot.
Alternatively, Ignition could always add x-initrd.attach for LUKS devices that back the root filesystem, since on any system where the initrd manages LUKS setup, the device should persist across soft-reboot.
Why rd.luks.options=x-initrd.attach as a kernel argument does not work
The systemd-cryptsetup-generator processes /etc/crypttab first via add_crypttab_devices(). When it finds the device in crypttab, it creates the unit using crypttab options (which don't include x-initrd.attach) and marks the device as created. Later, add_proc_cmdline_devices() skips the device because it was already created from crypttab. The rd.luks.options= default options are never applied. The crypttab must contain the option directly.
Current workaround
There is no shipped workaround. The ostree PR ostreedev/ostree#3571 fixes soft-reboot for var, sysroot, and boot mounts on plain disk and RAID, but does not address the LUKS case. A prototype fix was validated during development but removed from the PR because the proper fix belongs in the provisioning tool that writes the crypttab. That lives on https://github.com/jmarrero/ostree/tree/fix-soft-reboot-var-mount-2
Users can work around the issue manually by adding x-initrd.attach to their /etc/crypttab options field.
Related issues
Bug
ostree system with LUKS fails to boot after a
systemctl soft-rebootAdding
x-initrd.attachto crypttab when provisioning LUKS on ostree systems would fix the issue.Operating System Version
Fedora CoreOS 43, CentOS Stream 10 bootc, Fedora Silverblue 43 — any ostree/bootc system with LUKS root.
Ignition Version
All versions that provision LUKS (the
storage.luksconfig section).Environment
Any — qemu, bare metal, cloud. Reproducible on FCOS 43 qemu with the LUKS root-reprovision kola test.
Expected Behavior
After Ignition provisions a LUKS root device on an ostree system,
systemctl soft-rebootshould work. The LUKS device should survive soft-reboot shutdown so that/sysroot,/var, and/bootremain available.Actual Behavior
Soft-reboot fails with
var.mountfailure and the system drops to emergency shell. The LUKS device is torn down during soft-reboot shutdown becausesystemd-cryptsetup-generatorwritesConflicts=umount.targetinto the cryptsetup service unit. This causes a cascade: cryptsetup is stopped, the dm-crypt device is removed,/sysrootloses its backing device, and/var(a bind mount from/sysroot) fails to mount.On non-ostree systems this cascade is harmless — the kernel root mount (
-.mount,Perpetual=yes) pins the LUKS device andcryptsetup detachfails withEBUSY. On ostree,/sysrootis a regular mount (not the kernel root), so the detach actually succeeds.The
systemd-cryptsetup-generatorskipsConflicts=umount.targetwhen the crypttab entry has thex-initrd.attachoption. Ignition writes/etc/crypttabbut does not include this option.Reproduction Steps
root-reprovision/lukskola test config withstorage.luks+ clevis TPM2)systemctl soft-rebootvar.mountfailsOther Information
Proposed fix
When Ignition provisions LUKS on an ostree system (detected via
/run/ostree-booted), addx-initrd.attachto the crypttab entry's options field increateCrypttabEntries(). This tellssystemd-cryptsetup-generatornot to addConflicts=umount.target, preventing the teardown cascade during soft-reboot.Alternatively, Ignition could always add
x-initrd.attachfor LUKS devices that back the root filesystem, since on any system where the initrd manages LUKS setup, the device should persist across soft-reboot.Why
rd.luks.options=x-initrd.attachas a kernel argument does not workThe
systemd-cryptsetup-generatorprocesses/etc/crypttabfirst viaadd_crypttab_devices(). When it finds the device in crypttab, it creates the unit using crypttab options (which don't includex-initrd.attach) and marks the device as created. Later,add_proc_cmdline_devices()skips the device because it was already created from crypttab. Therd.luks.options=default options are never applied. The crypttab must contain the option directly.Current workaround
There is no shipped workaround. The ostree PR ostreedev/ostree#3571 fixes soft-reboot for var, sysroot, and boot mounts on plain disk and RAID, but does not address the LUKS case. A prototype fix was validated during development but removed from the PR because the proper fix belongs in the provisioning tool that writes the crypttab. That lives on https://github.com/jmarrero/ostree/tree/fix-soft-reboot-var-mount-2
Users can work around the issue manually by adding
x-initrd.attachto their/etc/crypttaboptions field.Related issues
rd.luksduring shutdown systemd/systemd#38803 — systemd RFE: cryptsetup generator should automatically implyx-initrd.attachforrd.luksdevices (would make this Ignition change unnecessary if implemented)