tar: Preserve trailing record-padding bytes after end-of-archive #292
Merged
jeckersb merged 3 commits intocomposefs:mainfrom May 7, 2026
Merged
tar: Preserve trailing record-padding bytes after end-of-archive #292jeckersb merged 3 commits intocomposefs:mainfrom
jeckersb merged 3 commits intocomposefs:mainfrom
Conversation
The mirror-fixture-images.yml workflow only runs on pushes to main, so during the PR that adds a new entry to ci/fixture-images.txt the ghcr.io mirror does not yet exist. Without a fallback, the integration test would fail trying to pull from ghcr.io. Add an upstream_ref field to ContainerImage and teach pull_image() to try the mirror first, then warn and retry against the upstream registry if the mirror pull fails. Fill in upstream_ref for existing images (UBI10 and centos-bootc) so the struct is complete. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
GNU tar pads archives to a "record size" (typically 20 × 512 = 10240 bytes). The archive ends with two 512-byte zero blocks that tar-core emits as ParseEvent::End, but any additional zero-padding blocks that fill out the record were silently dropped. split_async() stored only what ParseEvent::End::consumed covered (1024 bytes for the two zero blocks), leaving the remaining trailing bytes unread. When cat() later reconstructed the archive, the output was shorter than the original, causing the sha256 to differ from the layer's diff_id. This broke the checksum validation path in create_filesystem() — specifically the @digest reference path that lacks a verity proof and must re-hash the reconstructed tar. The fix reads any remaining bytes after the End event until true EOF and stores them inline, making cat() byte-for-bit faithful to the original archive. This resolves "Layer has incorrect checksum" errors for images where the tar ends with GNU-style record padding, including Ubuntu 26.04 (resolute) which uses umoci/PAX format. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
Ubuntu 26.04 uses umoci/PAX format tars produced by Rockcraft, which include GNU-style record padding after the end-of-archive blocks. Before the preceding fix, this caused 'Layer has incorrect checksum' errors on the @digest compute-id path because the reconstructed tar was shorter than the original. Pin the image by manifest digest (d31acef2) and record the expected composefs image ID so regressions are caught automatically. Assisted-by: OpenCode (Claude Sonnet 4.6) Signed-off-by: Colin Walters <walters@verbum.org>
|
thanks @cgwalters for the quick fix. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes: #290