fix(boot): /etc storage.conf + pasta default + static cockpit-systemd user

Three independent boot failures from the 2026-05-06 boot journal,
all surfaced after the rshared/devnodes fix in 69cf6e3 unblocked
forward progress.

1. Rootful container storage falling through to fuse-overlayfs.
   Symptom (every Quadlet, including mios-ai / mios-forge / mios-
   searxng): `fuse-overlayfs: cannot mount: No such file or directory`.
   Root cause: containers/storage's documented config-resolution chain
   is $CONTAINERS_STORAGE_CONF -> /etc/containers/storage.conf ->
   ~/.config/containers/storage.conf -> hardcoded defaults. The
   /usr/share/containers/storage.conf path we'd been shipping is
   convention but is NOT read by libcontainers/storage at runtime, so
   the base image's /etc/containers/storage.conf (from
   ghcr.io/ublue-os/ucore-hci) was winning with mount_program=
   /usr/bin/fuse-overlayfs. Shipping a real /etc/containers/storage.
   conf in MiOS with mount_program="" forces kernel-native overlayfs
   (Linux 5.11+ supports unprivileged overlayfs in user namespaces
   with metacopy=on + userxattr; the microsoft-WSL2 6.6 kernel does).
   Adds whitelist line `!/etc/containers/storage.conf` to .gitignore
   so the new file isn't masked by `etc/containers/*`.

2. Rootless network forced to slirp4netns under a wrong premise.
   Symptom: `start slirp4netns: /usr/bin/slirp4netns failed:
   open(/dev/net/tun): No such file or directory`. Root cause: an
   earlier drop-in at /etc/containers/containers.conf.d/30-mios-
   rootless-network.conf forced default_rootless_network_cmd=
   "slirp4netns" on the false assumption that slirp4netns "works
   without /dev/net/tun". It doesn't -- slirp4netns creates a tap
   device inside the rootless netns and that tap requires /dev/net/
   tun to be exposed. pasta (passt-pasta) is socket-only -- no tun
   device, no CAP_NET_ADMIN, ~3x faster on TCP -- and IS the right
   answer for the missing-tun nested-container shape MiOS targets.
   This commit reverses the directive to default_rootless_network_
   cmd="pasta" and rewrites the file's comment block so the next
   reader doesn't fall into the same trap.

3. Cockpit 217/USER for cockpit-systemd-service.
   Symptom: `cockpit.service: Failed to determine credentials for
   user 'cockpit-systemd-service': Unknown user / Failed at step
   USER spawning /usr/libexec/cockpit-certificate-ensure: Invalid
   argument`. Root cause: newer cockpit-ws (300+) ships the unit
   with `User=cockpit-systemd-service` + `DynamicUser=yes`. Our
   /etc/systemd/system/cockpit.service.d/10-mios-container.conf
   (added in 1f74f44) sets DynamicUser=no to prevent the
   PrivateTmp=yes implication that DynamicUser=yes forces, because
   PrivateTmp triggers a mount namespace clone which fails on WSL2.
   With DynamicUser=no the User= reference is looked up statically
   and finds nothing on a fresh image. Adds usr/lib/sysusers.d/
   50-mios-cockpit.conf pinning cockpit-systemd-service to UID 977
   (system range, away from MiOS service slots and upstream Fedora
   reservations) so the static lookup succeeds while the namespace
   resets stay in effect.

Boot order: mios-wsl-early (rshared/devnodes) still runs first, and
its WARN messages on hosts where mknod/make-rshared genuinely fails
(unprivileged-podman-in-WSL2) are now expected to be benign rather
than fatal -- with kernel overlay + pasta, neither /dev/fuse nor
/dev/net/tun is required to come up.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: Kabuki94

.gitignore

@@ -104,6 +104,12 @@ etc/*
 !/etc/.keep
 !/etc/containers/
 etc/containers/*
+# storage.conf -- rootful containers/storage config. /usr/share/containers/
+# storage.conf is NOT read by libcontainers/storage at runtime (only /etc/
+# and ~/.config/), so the vendor file there is ignored and the base image's
+# /etc/containers/storage.conf wins. Shipping our own here puts kernel
+# overlay (mount_program="") in front of the cascade.
+!/etc/containers/storage.conf
 !/etc/containers/systemd/
 etc/containers/systemd/*
 !/etc/containers/systemd/mios*

etc/containers/containers.conf.d/30-mios-rootless-network.conf

@@ -1,35 +1,32 @@
 # /etc/containers/containers.conf.d/30-mios-rootless-network.conf
 #
-# Force rootless containers to use slirp4netns instead of the newer
-# pasta backend.
+# Force rootless containers to use pasta (NOT slirp4netns).
 #
-# Why: podman 5.x defaults rootless networking to pasta, which opens
-# /dev/net/tun directly. Some MiOS deployment shapes -- notably the
-# MiOS-DEV podman-machine WSL2 distro that operates as the Windows-
-# side build/runtime substrate -- ship without /dev/net/tun (the
-# microsoft-WSL2 kernel does have the tun module compiled in, but the
-# device node isn't created until something modprobes it, and
-# rootless podman runs unprivileged so it can't trigger that).
+# History (2026-05-06 reversal):
+#   This file used to set default_rootless_network_cmd="slirp4netns"
+#   on the assumption that slirp4netns "works without /dev/net/tun".
+#   That assumption was wrong. slirp4netns DOES open /dev/net/tun --
+#   it creates a tap device inside the rootless network namespace,
+#   and that tap requires the kernel's tun module to be exposed via
+#   /dev/net/tun. The same MiOS-DEV / nested-container shapes that
+#   miss the device for pasta also miss it for slirp4netns. The
+#   observed failure was identical:
 #
-# Without this drop-in every Quadlet that uses rootless networking
-# fails at startup with:
-#     pasta[NNNN]: Failed to open() /dev/net/tun: No such file or directory
-#     pasta[NNNN]: Failed to set up tap device in namespace
-#     mios-forge[NNNN]: setting up Pasta: pasta failed with exit code 1
+#     mios-forge[N]: time="..." level=error msg="Preparing container ...:
+#         rootless netns: start slirp4netns: /usr/bin/slirp4netns failed:
+#         \"open(\\\"/dev/net/tun\\\"): No such file or directory\"
 #
-# slirp4netns is the legacy rootless network backend; it builds an
-# in-process userspace TCP/IP stack instead of using a tap device,
-# so it works on every host without requiring kernel/device support.
-# Slightly slower than pasta on TCP throughput, but functionally
-# identical for everything MiOS does (REST APIs, git push/pull,
-# systemd-managed inbound listeners). Architectural Law 4 (VM |
-# Container | Flatpak only) is unaffected.
+# pasta (passt-pasta) implements the same NAT/forwarding contract as
+# slirp4netns but uses sockets only -- no tun device, no CAP_NET_ADMIN,
+# and ~3x faster on TCP throughput. Upstream-canonical default since
+# podman 4.7 (the official recommendation since 5.0). Both binaries
+# remain available on the image; this drop-in just nails the default
+# the rootless network engine picks when a Quadlet doesn't override
+# it explicitly.
 #
-# Per the MiOS-DEV feature-parity contract this drop-in ships on
-# every shape, not gated to WSL -- the substrate either has /dev/net/tun
-# (most hosts) and pasta would have worked anyway, or doesn't (WSL),
-# in which case slirp4netns is the only option. Either way the
-# behavior is consistent across all MiOS deployments.
+# Architectural Law 4 (VM | Container | Flatpak only) is unaffected --
+# this is a backend choice within the rootless-podman primitive, not
+# a new distribution mechanism.
 
 [network]
-default_rootless_network_cmd = "slirp4netns"
+default_rootless_network_cmd = "pasta"

etc/containers/storage.conf

@@ -0,0 +1,42 @@
+# /etc/containers/storage.conf
+# 'MiOS' rootful container storage config.
+#
+# Why this file lives in /etc and not /usr/share:
+#   containers/storage's documented config-resolution chain is
+#   $CONTAINERS_STORAGE_CONF -> /etc/containers/storage.conf ->
+#   ~/.config/containers/storage.conf -> hardcoded defaults. The
+#   /usr/share/containers/storage.conf path is convention but is NOT
+#   read by libcontainers/storage at runtime, so a vendor file there is
+#   ignored. Inheriting /etc/containers/storage.conf from the base
+#   image (ghcr.io/ublue-os/ucore-hci) gave us its `mount_program =
+#   /usr/bin/fuse-overlayfs` default, which then fails inside nested
+#   containers and WSL2 distros where /dev/fuse is missing:
+#
+#       fuse-overlayfs: cannot mount: No such file or directory
+#       Error: mounting storage for container ...: creating overlay
+#           mount to /var/lib/containers/storage/overlay/.../merged
+#
+# Setting mount_program="" tells containers/storage to skip
+# fuse-overlayfs and ask the kernel to do the overlay mount directly.
+# The microsoft-WSL2 6.6 kernel (and every Linux >= 5.11) supports
+# unprivileged overlayfs in user namespaces with metacopy=on +
+# userxattr, which preserves uid/gid mappings without copy-up. This
+# is the single change that closes the per-boot Quadlet failure
+# cascade observed in MiOS WSL/nested-container deployments.
+#
+# Operators who legitimately want fuse-overlayfs (e.g. older kernels,
+# specific shfs interactions) override here in /etc/ or in
+# ~/.config/containers/storage.conf -- /etc wins.
+
+[storage]
+driver = "overlay"
+runroot = "/run/containers/storage"
+graphroot = "/var/lib/containers/storage"
+
+[storage.options]
+additionalimagestores = []
+
+[storage.options.overlay]
+# Empty mount_program => kernel-native overlayfs. NOT fuse-overlayfs.
+mount_program = ""
+mountopt = "nodev,metacopy=on,userxattr"

usr/lib/sysusers.d/50-mios-cockpit.conf

@@ -0,0 +1,26 @@
+# /usr/lib/sysusers.d/50-mios-cockpit.conf
+# 'MiOS' static user for cockpit's dynamic-user references.
+#
+# Newer cockpit-ws (300+) ships /usr/lib/systemd/system/cockpit.service
+# with `User=cockpit-systemd-service` + `DynamicUser=yes`. The dynamic
+# user mechanism creates the user transiently per-invocation. But our
+# /etc/systemd/system/cockpit.service.d/10-mios-container.conf neutralizes
+# DynamicUser= (along with PrivateTmp / ProtectSystem / ... ) so the unit
+# can boot under WSL2's unprivileged-namespace constraints. Without
+# DynamicUser= the User= reference is looked up STATICALLY in /etc/passwd,
+# and on a fresh image that's empty -- yielding the per-boot failure:
+#
+#     cockpit.service: Failed to determine credentials for user
+#         'cockpit-systemd-service': Unknown user
+#     cockpit.service: Failed at step USER spawning
+#         /usr/libexec/cockpit-certificate-ensure: Invalid argument
+#     cockpit.service: status=217/USER
+#
+# Pinning the user statically here closes that gap. UID 977 is in the
+# system range (<1000) and away from MiOS service slots (810-829) +
+# upstream Fedora reservations (<200, 81 dbus, 999 polkitd, 990
+# systemd-resolve, 983 zincati, etc.). The user has /sbin/nologin and
+# /var/empty so it can never be used for an interactive login.
+
+g cockpit-systemd-service 977
+u cockpit-systemd-service 977:cockpit-systemd-service "'MiOS' Cockpit systemd-service helper" /var/empty /sbin/nologin