Full cache for offline mode#497
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends OS package caching to better support “offline” rebuilds by reusing previously downloaded repository metadata, package lists, and cached artifacts across runs.
Changes:
- Add persistent caching for RPM metadata (repomd-derived primary location + parsed primary metadata) and DEB parsed package metadata (plus URL existence / package-list URL lookup caches).
- Add package-cache “up-to-date” checks to short-circuit downloads when all required DEBs/RPMs are already present locally.
- Isolate initrd/chrootenv package download directories and sync initrd packages back into the shared cache; add/adjust tests and release notes.
Reviewed changes
Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/ospackage/rpmutils/resolver_test.go | Adds offline-cache tests for RPM metadata parsing and repomd primary location resolution. |
| internal/ospackage/rpmutils/resolver.go | Implements RPM metadata caching under cache_dir/rpm-metadata with offline-first reads. |
| internal/ospackage/rpmutils/helper_test.go | Simplifies SPDX filename tests using a regex-based timestamp check. |
| internal/ospackage/rpmutils/download_cache_test.go | Adds tests for RPM package-cache staleness and cache-clearing behavior. |
| internal/ospackage/rpmutils/download.go | Adds RPM package-cache staleness detection, cache clearing helpers, and cache short-circuiting. |
| internal/ospackage/debutils/zip.go | Adds cached package-list URL lookup to avoid repeated probing of Packages.{gz,xz}. |
| internal/ospackage/debutils/resolver.go | Adds parsed package metadata cache keyed by Release checksum for DEB repos. |
| internal/ospackage/debutils/helper_test.go | Tightens SPDX filename tests to validate timestamped format via regex. |
| internal/ospackage/debutils/download_test.go | Adds tests for offline usage of URL/package-list caches and DEB package-cache staleness/clearing. |
| internal/ospackage/debutils/download.go | Adds persisted URL existence + package-list URL caches, user-repo config init, and DEB cache short-circuiting/clearing. |
| internal/image/initrdmaker/initrdmaker.go | Downloads initrd packages into an isolated cache dir and syncs them into the shared cache. |
| internal/image/imageos/imageos_test.go | Updates DEB local repo test harness and mock chroot env to validate metadata refresh call path. |
| internal/image/imageos/imageos.go | Refreshes local DEB repo metadata as part of DEB local repo initialization. |
| internal/config/apt_sources_test.go | Adds tests for stable GPG-key cache path and cache-hit behavior. |
| internal/config/apt_sources_integration_test.go | Minor formatting-only changes. |
| internal/config/apt_sources.go | Adds persistent caching of downloaded repository GPG keys under cache_dir/gpg-keys. |
| internal/chroot/chrootbuild/chrootbuild.go | Isolates chrootenv package downloads into a subdir to avoid cache invalidation collisions. |
| docs/release-notes.md | Documents new GPG key and RPM metadata caching locations/behavior. |
| cmd/os-image-composer/version_test.go | Fixes ignored errors when closing pipes in tests. |
| cmd/os-image-composer/completion.go | Handles close/remove errors in dirWritable. |
| cmd/os-image-composer/cache.go | Handles fmt.Fprintln write errors when printing cache output. |
| .coverage-threshold | Updates coverage threshold value. |
srmungar
approved these changes
May 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merge Checklist
All boxes should be checked before merging the PR
Description
This PR extends OS package caching to better support “offline” rebuilds by reusing previously downloaded repository metadata, package lists, and cached artifacts across runs.
Changes:
Add persistent caching for RPM metadata (repomd-derived primary location + parsed primary metadata) and DEB parsed package metadata (plus URL existence / package-list URL lookup caches).
Add package-cache “up-to-date” checks to short-circuit downloads when all required DEBs/RPMs are already present locally.
Isolate initrd/chrootenv package download directories and sync initrd packages back into the shared cache; add/adjust tests and release notes.
Any Newly Introduced Dependencies
How Has This Been Tested?