Skip to content

Full cache for offline mode#497

Merged
samueltaripin merged 20 commits into
mainfrom
full_cache
May 19, 2026
Merged

Full cache for offline mode#497
samueltaripin merged 20 commits into
mainfrom
full_cache

Conversation

@samueltaripin
Copy link
Copy Markdown
Contributor

@samueltaripin samueltaripin commented Mar 30, 2026

Merge Checklist

All boxes should be checked before merging the PR

  • The changes in the PR have been built and tested
  • Documentation has been updated to reflect the changes (or no doc update needed)
  • Ready to merge

Description

This PR extends OS package caching to better support “offline” rebuilds by reusing previously downloaded repository metadata, package lists, and cached artifacts across runs.

Changes:

Add persistent caching for RPM metadata (repomd-derived primary location + parsed primary metadata) and DEB parsed package metadata (plus URL existence / package-list URL lookup caches).
Add package-cache “up-to-date” checks to short-circuit downloads when all required DEBs/RPMs are already present locally.
Isolate initrd/chrootenv package download directories and sync initrd packages back into the shared cache; add/adjust tests and release notes.

Any Newly Introduced Dependencies

How Has This Been Tested?

  1. run build
  2. disconnect from network
  3. re-run same build

@samueltaripin samueltaripin marked this pull request as ready for review April 22, 2026 05:12
@samueltaripin samueltaripin requested a review from a team as a code owner April 22, 2026 05:12
Copilot AI review requested due to automatic review settings April 22, 2026 05:12
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends OS package caching to better support “offline” rebuilds by reusing previously downloaded repository metadata, package lists, and cached artifacts across runs.

Changes:

  • Add persistent caching for RPM metadata (repomd-derived primary location + parsed primary metadata) and DEB parsed package metadata (plus URL existence / package-list URL lookup caches).
  • Add package-cache “up-to-date” checks to short-circuit downloads when all required DEBs/RPMs are already present locally.
  • Isolate initrd/chrootenv package download directories and sync initrd packages back into the shared cache; add/adjust tests and release notes.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
internal/ospackage/rpmutils/resolver_test.go Adds offline-cache tests for RPM metadata parsing and repomd primary location resolution.
internal/ospackage/rpmutils/resolver.go Implements RPM metadata caching under cache_dir/rpm-metadata with offline-first reads.
internal/ospackage/rpmutils/helper_test.go Simplifies SPDX filename tests using a regex-based timestamp check.
internal/ospackage/rpmutils/download_cache_test.go Adds tests for RPM package-cache staleness and cache-clearing behavior.
internal/ospackage/rpmutils/download.go Adds RPM package-cache staleness detection, cache clearing helpers, and cache short-circuiting.
internal/ospackage/debutils/zip.go Adds cached package-list URL lookup to avoid repeated probing of Packages.{gz,xz}.
internal/ospackage/debutils/resolver.go Adds parsed package metadata cache keyed by Release checksum for DEB repos.
internal/ospackage/debutils/helper_test.go Tightens SPDX filename tests to validate timestamped format via regex.
internal/ospackage/debutils/download_test.go Adds tests for offline usage of URL/package-list caches and DEB package-cache staleness/clearing.
internal/ospackage/debutils/download.go Adds persisted URL existence + package-list URL caches, user-repo config init, and DEB cache short-circuiting/clearing.
internal/image/initrdmaker/initrdmaker.go Downloads initrd packages into an isolated cache dir and syncs them into the shared cache.
internal/image/imageos/imageos_test.go Updates DEB local repo test harness and mock chroot env to validate metadata refresh call path.
internal/image/imageos/imageos.go Refreshes local DEB repo metadata as part of DEB local repo initialization.
internal/config/apt_sources_test.go Adds tests for stable GPG-key cache path and cache-hit behavior.
internal/config/apt_sources_integration_test.go Minor formatting-only changes.
internal/config/apt_sources.go Adds persistent caching of downloaded repository GPG keys under cache_dir/gpg-keys.
internal/chroot/chrootbuild/chrootbuild.go Isolates chrootenv package downloads into a subdir to avoid cache invalidation collisions.
docs/release-notes.md Documents new GPG key and RPM metadata caching locations/behavior.
cmd/os-image-composer/version_test.go Fixes ignored errors when closing pipes in tests.
cmd/os-image-composer/completion.go Handles close/remove errors in dirWritable.
cmd/os-image-composer/cache.go Handles fmt.Fprintln write errors when printing cache output.
.coverage-threshold Updates coverage threshold value.

Comment thread internal/ospackage/debutils/resolver.go
Comment thread internal/ospackage/debutils/resolver.go Outdated
Comment thread internal/ospackage/rpmutils/resolver.go
Comment thread internal/ospackage/debutils/download.go
Comment thread internal/image/imageos/imageos.go Outdated
@samueltaripin samueltaripin requested a review from a team as a code owner May 6, 2026 08:45
@samueltaripin samueltaripin merged commit 78eefe8 into main May 19, 2026
3 checks passed
@samueltaripin samueltaripin deleted the full_cache branch May 19, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants