Skip to content

Releases: composefs/composefs-rs

v0.6.0

11 Mar 18:24
@cgwalters cgwalters
Immutable release. Only release title and notes can be modified.
v0.6.0
b4fe6d4

Choose a tag to compare

View all tags
v0.6.0 Latest
Latest

A lot of cool stuff landing here, and even more in the pipeline! In particular we have a completely revamped interface for oci, our tar parsing is now much more strongly hardened, and much improved EROFS handling.

What's Changed

Full Changelog: v0.5.0...v0.6.0

Contributors

cgwalters and Johan-Liebert1
Assets 3
Loading

v0.5.0

16 Feb 14:11
@cgwalters cgwalters
v0.5.0
b9163a8

Choose a tag to compare

View all tags
v0.5.0

This project has made huge strides since its creation. We didn't create formal release tags as we were working, so here's the first one! There are two big themes going on:

Format stability

The goal of this effort is to finalize the on-disk format and especially precisely how we generate the EROFS so that checksums are stable.

Replacing composefs-c

The original composefs project was written in C and had a different functionality set (includes a CLI too and C shared library, but does not scope in any specific support for OCI or logic that runs in the initramfs).

Changes in last two months

Featured changes

Splitstream format rework (ostree prep)

The splitstream file format was updated to support future ostree use and to improve GC. The header is no longer compressed. All referenced fs-verity objects (external chunks, mapped splitstreams, and optional references not used in chunks) are stored in the header. The mapping table is separate from the reference table. The format now has a magic value and content-type magic for detection, stores the ObjectID format tag and total stream size in the header, and allows referencing file objects in the repo even when they are not part of the splitstream content. GC can enumerate referenced objects without parsing the entire splitstream.

  • Preparatory splitstream format changes for ostree support by @alexlarsson in #185

OCI-native manifest and config storage

OCI images are now stored and identified by explicit oci/ tags that point to a manifest, which points to a config. Manifest and config are still written as splitstreams but the original bytes are stored as external objects, aligning with a future proposal for composefs OCI sealing (fsverity signatures for manifest/config as a detached object). Downstream such as bootc can stop storing the manifest on their own (see bootc#1830). This change also adds general support for OCI artifacts and referrers (e.g. sigstore, future composefs signatures) and an initial sketch for multi-arch image storage.

  • oci: Add OCI-native manifest/config storage and image management by @cgwalters in #216

Parallel object storage in tar layers

Tar processing now writes large files to the object store in parallel. Files are streamed to O_TMPFILE via a channel while fs-verity digests are computed in background blocking tasks, avoiding blocking the async runtime. New repository helpers include create_object_tmpfile, finalize_object_tmpfile, and a shared write semaphore for backpressure. For a 2GB tar (10,000 files × 200KB), this yields roughly a 7× speedup (~980ms → ~140ms).

  • oci: Parallelize object storage within tar layers by @cgwalters in #208

Container root metadata from /usr

OCI layer tars often omit a root directory entry, and container runtimes ignore it. Root metadata (mode, uid, gid, mtime, xattrs) is now defined from /usr so it is deterministic. New APIs: copy_root_metadata_from_usr() and read_container_root() (which calls read_filesystem() then copies metadata). The previous have_root_stat / ensure_root_stat logic (which only handled mtime) was removed. cfsctl commands use read_container_root() by default; --no-propagate-usr-to-root uses raw read_filesystem() for testing.

  • fs: Add read_container_root() and copy metadata from /usr -> / by @cgwalters in #209

Garbage collection fix

The previous GC implementation never added any object IDs to the live set and would mark every object for deletion. It assumed an older repository layout where non-first-level entries under streams/ and images/ could link directly to the object store; in the current layout, refs link to first-level entries which then link to objects. The fix walks all named references into the live set and performs a full recursive walk of the stream tree (config splitstream → layer splitstreams → layer contents) so that only unreferenced objects are marked for deletion. The implementation remains dry-run by default; output can be piped to a shell to perform deletions. Relevant for downstream such as bootc (see bootc#1808).

Containers-storage: use uncompressed layer digest

When pulling from containers-storage, the code was requesting the layer by its compressed digest, which led to skopeo errors (“file does not exist”). The pull path now uses get_blob with the uncompressed digest for the layer. Fixes bootc#1703.

  • skopeo: Get uncompressed layer when pulling from containers-storage by @Johan-Liebert1 in #210

Important Bugfixes

Bugfixes

  • tar: Use next_multiple_of for 512-byte alignment by @cgwalters in #201
  • Use u64 for file sizes to handle >4GB on 32-bit by @cgwalters in #204
  • composefs-oci: Fix wrong OCI opaque whiteout filename by @chaserhkj in #205
  • Fix for ensure_config + re-enable bootc revdep CI by @cgwalters in #207

Other changes

Contributors

giuseppe, cgwalters, and 4 other contributors
Loading