oci: Empty /run and use /usr mtime for emptied directories

Empty /run in create_filesystem() when processing OCI images. This is
a tmpfs at runtime and should always be empty in images.

Emptying /run also works around podman/buildah's RUN --mount issue where
cache mounts can leave incomplete directory entries in OCI tar layers,
causing mtime inconsistencies. Container authors can now safely use:

    RUN rm -rf /var/cache/dnf && ln -sr /run/dnfcache /var/cache/dnf
    RUN --mount=type=cache,target=/run/dnfcache dnf install -y ...

The /run directory is emptied for all OCI images, not just bootable ones,
since it should always be empty regardless of boot configuration.

Also change emptied directories in transform_for_boot() (/boot, /sysroot)
to use /usr's mtime instead of 0, for consistency with how root directory
metadata is handled via copy_root_metadata_from_usr().

Closes: #132
Assisted-by: OpenCode (Opus 4.5)
Signed-off-by: Colin Walters <walters@verbum.org>

Author: cgwalters

crates/composefs-boot/src/lib.rs

@@ -14,6 +14,8 @@ pub mod selabel;
 pub mod uki;
 pub mod write_boot;
 
+use std::ffi::OsStr;
+
 use anyhow::Result;
 
 use composefs::{fsverity::FsVerityHashValue, repository::Repository, tree::FileSystem};
@@ -67,9 +69,14 @@ impl<ObjectID: FsVerityHashValue> BootOps<ObjectID> for FileSystem<ObjectID> {
         repo: &Repository<ObjectID>,
     ) -> Result<Vec<BootEntry<ObjectID>>> {
         let boot_entries = get_boot_resources(self, repo)?;
+
+        // Get /usr's mtime to use as the canonical mtime for emptied directories.
+        // This matches how we handle the root directory in copy_root_metadata_from_usr().
+        let usr_mtime = self.root.get_directory(OsStr::new("usr"))?.stat.st_mtim_sec;
+
         for d in REQUIRED_TOPLEVEL_TO_EMPTY_DIRS {
             let d = self.root.get_directory_mut(d.as_ref())?;
-            d.stat.st_mtim_sec = 0;
+            d.stat.st_mtim_sec = usr_mtime;
             d.clear();
         }
 

crates/composefs-oci/src/image.rs

@@ -128,9 +128,9 @@ pub fn create_filesystem<ObjectID: FsVerityHashValue>(
         }
     }
 
-    // Copy root metadata from /usr to ensure consistent digests across different
-    // container runtimes and tar implementations.
-    filesystem.copy_root_metadata_from_usr()?;
+    // Apply OCI container transformations for consistent digests.
+    // See https://github.com/containers/composefs-rs/issues/132
+    filesystem.transform_for_oci()?;
 
     Ok(filesystem)
 }

crates/composefs/src/fs.rs

@@ -400,27 +400,29 @@ pub fn is_allowed_container_xattr(name: &OsStr) -> bool {
 /// Load a container root filesystem from the given path.
 ///
 /// This is a convenience wrapper around [`read_filesystem_filtered`] that also
-/// copies metadata from `/usr` to the root directory.
+/// applies OCI container transformations via [`FileSystem::transform_for_oci`].
 ///
 /// Equivalent to calling:
 /// ```ignore
 /// let mut fs = read_filesystem_filtered(dirfd, path, repo, is_allowed_container_xattr)?;
-/// fs.copy_root_metadata_from_usr()?;
+/// fs.transform_for_oci()?;
 /// ```
 ///
 /// This is the recommended way to read a container filesystem because:
 /// - OCI container runtimes don't preserve root directory metadata from layer tars
 /// - Host xattrs (especially `security.selinux`) can leak into mounted filesystems
+/// - `/run` should be empty (it's a tmpfs at runtime)
+/// - Podman/buildah's `RUN --mount` can leave directory stubs
 ///
-/// By filtering xattrs and copying `/usr` metadata to root, we ensure consistent
+/// By filtering xattrs and applying OCI transformations, we ensure consistent
 /// and reproducible composefs digests between build-time and install-time.
 pub fn read_container_root<ObjectID: FsVerityHashValue>(
     dirfd: impl AsFd,
     path: &Path,
     repo: Option<&Repository<ObjectID>>,
 ) -> Result<FileSystem<ObjectID>> {
     let mut fs = read_filesystem_filtered(dirfd, path, repo, is_allowed_container_xattr)?;
-    fs.copy_root_metadata_from_usr()?;
+    fs.transform_for_oci()?;
     Ok(fs)
 }
 

crates/composefs/src/generic_tree.rs

@@ -547,6 +547,52 @@ impl<T> FileSystem<T> {
             stat.xattrs.borrow_mut().retain(|k, _| predicate(k));
         });
     }
+
+    /// Empties the `/run` directory if present, using `/usr`'s mtime.
+    ///
+    /// `/run` is a tmpfs at runtime and should always be empty in container images.
+    /// This also works around podman/buildah's `RUN --mount` behavior where bind
+    /// mount targets leave directory stubs in the filesystem that shouldn't be
+    /// part of the image content.
+    ///
+    /// The mtime is set to match `/usr` for consistency with [`Self::copy_root_metadata_from_usr`].
+    ///
+    /// NOTE: If changing this behavior, also update `doc/oci.md`.
+    ///
+    /// # Errors
+    ///
+    /// Returns an error if `/usr` does not exist (needed to get the mtime).
+    pub fn canonicalize_run(&mut self) -> Result<(), ImageError> {
+        if self.root.get_directory_opt(OsStr::new("run"))?.is_some() {
+            let usr_mtime = self.root.get_directory(OsStr::new("usr"))?.stat.st_mtim_sec;
+            let run_dir = self.root.get_directory_mut(OsStr::new("run"))?;
+            run_dir.stat.st_mtim_sec = usr_mtime;
+            run_dir.clear();
+        }
+        Ok(())
+    }
+
+    /// Transforms the filesystem for OCI container image consistency.
+    ///
+    /// This applies the standard transformations needed to ensure consistent
+    /// composefs digests between build-time (mounted filesystem) and install-time
+    /// (OCI tar layers) views:
+    ///
+    /// 1. [`Self::copy_root_metadata_from_usr`] - copies `/usr` metadata to root directory
+    /// 2. [`Self::canonicalize_run`] - empties `/run` directory
+    ///
+    /// This is the recommended single entry point for OCI container processing.
+    ///
+    /// NOTE: If changing this behavior, also update `doc/oci.md`.
+    ///
+    /// # Errors
+    ///
+    /// Returns an error if `/usr` does not exist.
+    pub fn transform_for_oci(&mut self) -> Result<(), ImageError> {
+        self.copy_root_metadata_from_usr()?;
+        self.canonicalize_run()?;
+        Ok(())
+    }
 }
 
 #[cfg(test)]
@@ -921,4 +967,93 @@ mod tests {
         assert_eq!(root_xattrs.len(), 1);
         assert!(root_xattrs.contains_key(OsStr::new("user.custom")));
     }
+
+    #[test]
+    fn test_canonicalize_run() {
+        let mut fs = FileSystem::<FileContents>::new(default_stat());
+
+        // Create /usr with specific mtime
+        let usr_dir = Directory::new(stat_with_mtime(12345));
+        fs.root
+            .insert(OsStr::new("usr"), Inode::Directory(Box::new(usr_dir)));
+
+        // Create /run with content and different mtime
+        let mut run_dir = Directory::new(stat_with_mtime(99999));
+        run_dir.insert(OsStr::new("somefile"), Inode::Leaf(new_leaf_file(11111)));
+        let mut subdir = Directory::new(stat_with_mtime(22222));
+        subdir.insert(OsStr::new("nested"), Inode::Leaf(new_leaf_file(33333)));
+        run_dir.insert(OsStr::new("subdir"), Inode::Directory(Box::new(subdir)));
+        fs.root
+            .insert(OsStr::new("run"), Inode::Directory(Box::new(run_dir)));
+
+        // Verify /run has content before
+        assert_eq!(
+            fs.root
+                .get_directory(OsStr::new("run"))
+                .unwrap()
+                .entries
+                .len(),
+            2
+        );
+
+        // Canonicalize
+        fs.canonicalize_run().unwrap();
+
+        // Verify /run is now empty with /usr's mtime
+        let run = fs.root.get_directory(OsStr::new("run")).unwrap();
+        assert!(run.entries.is_empty());
+        assert_eq!(run.stat.st_mtim_sec, 12345);
+    }
+
+    #[test]
+    fn test_canonicalize_run_no_run_dir() {
+        let mut fs = FileSystem::<FileContents>::new(default_stat());
+
+        // Create /usr but no /run
+        let usr_dir = Directory::new(stat_with_mtime(12345));
+        fs.root
+            .insert(OsStr::new("usr"), Inode::Directory(Box::new(usr_dir)));
+
+        // Should succeed without error
+        fs.canonicalize_run().unwrap();
+    }
+
+    #[test]
+    fn test_transform_for_oci() {
+        let mut fs = FileSystem::<FileContents>::new(default_stat());
+
+        // Create /usr with specific metadata
+        let usr_stat = Stat {
+            st_mode: 0o750,
+            st_uid: 100,
+            st_gid: 200,
+            st_mtim_sec: 54321,
+            xattrs: RefCell::new(BTreeMap::from([(
+                Box::from(OsStr::new("user.test")),
+                Box::from(b"val".as_slice()),
+            )])),
+        };
+        fs.root
+            .insert(OsStr::new("usr"), new_dir_inode_with_stat(usr_stat));
+
+        // Create /run with content
+        let mut run_dir = Directory::new(stat_with_mtime(99999));
+        run_dir.insert(OsStr::new("file"), Inode::Leaf(new_leaf_file(11111)));
+        fs.root
+            .insert(OsStr::new("run"), Inode::Directory(Box::new(run_dir)));
+
+        // Transform for OCI
+        fs.transform_for_oci().unwrap();
+
+        // Verify root metadata copied from /usr
+        assert_eq!(fs.root.stat.st_mode, 0o750);
+        assert_eq!(fs.root.stat.st_uid, 100);
+        assert_eq!(fs.root.stat.st_gid, 200);
+        assert_eq!(fs.root.stat.st_mtim_sec, 54321);
+
+        // Verify /run is emptied with /usr's mtime
+        let run = fs.root.get_directory(OsStr::new("run")).unwrap();
+        assert!(run.entries.is_empty());
+        assert_eq!(run.stat.st_mtim_sec, 54321);
+    }
 }

doc/oci.md

@@ -77,3 +77,51 @@ This ensures that:
  - Other host xattrs (overlayfs internals, etc.) don't pollute the image
 
 See: https://github.com/containers/storage/pull/1608#issuecomment-1600915185
+
+# The /run directory
+
+When processing OCI images via `create_filesystem()`, the `/run` directory
+is emptied if present. This is a tmpfs at runtime and should always be
+empty in images. Its mtime is set to match `/usr` for consistency with
+how root directory metadata is handled.
+
+This makes it possible to work around podman/buildah's `RUN --mount` issue where cache
+mounts can leave incomplete directory entries in OCI tar layers (directories
+without explicit tar entries inherit incorrect mtimes) by pointing all
+such mounts into `/run`, and then redirecting from their final location
+via e.g. symlinks into `/run`.
+
+## Container build cache mounts
+
+A practical implication of emptying `/run` is that container authors can
+use it for cache mounts without worrying about polluting the final image.
+
+Instead of:
+```dockerfile
+RUN --mount=type=cache,target=/var/cache/dnf dnf install -y ...
+```
+
+Consider:
+```dockerfile
+RUN rm -rf /var/cache/dnf && ln -sr /run/dnfcache /var/cache/dnf
+RUN --mount=type=cache,target=/run/dnfcache dnf install -y ...
+```
+
+This avoids potential mtime inconsistencies in `/var/cache` while still
+benefiting from build caching.
+
+See: https://github.com/containers/composefs-rs/issues/132
+
+# Emptied directories for boot
+
+When preparing a filesystem for boot via `transform_for_boot()`, certain
+additional directories are emptied because their contents should not be
+part of the final verified image:
+
+- `/boot`: Contains the UKI which embeds the composefs digest, so including
+  it would create a circular dependency
+- `/sysroot`: Only has content in ostree-container cases, and traversing
+  it for SELinux labeling causes problems
+
+These directories are emptied and their mtime is set to match `/usr` for
+consistency with how the root directory metadata is handled.