Save disk space by only caching dev packages into the internal 'sources' directory#6440
Open
kit-ty-kate wants to merge 2 commits into
Open
Save disk space by only caching dev packages into the internal 'sources' directory#6440kit-ty-kate wants to merge 2 commits into
kit-ty-kate wants to merge 2 commits into
Conversation
kit-ty-kate
added
AREA: PORTABILITY
PR: WIP
Member
Author
|
As a demonstration, |
kit-ty-kate
force-pushed
the
sources-save-disk-space
branch
3 times, most recently
from
0781bfc to
ae3a100
Compare
kit-ty-kate
changed the title
kit-ty-kate
force-pushed
the
sources-save-disk-space
branch
from
ae3a100 to
45235f3
Compare
Member
Author
|
xref live test results: #6906 (comment) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5448
Fixes #4056
The
.opam-switch/sourcesdirectory currently stores the source of every installed packages in the following format:<pkgname>: are for pinned packages<pkgname>.<version>: are for the restIn #2825 (opam 2.0.0~beta) the change from
packages.devtosourceswas introduced to support (as i understand it) non-pinned dev packages (e.g.ocaml-variants.5.4.0+trunk). This is a good change but this also had for side-effect to store the source of every packages regardless of whether it's a dev package or not, which i think isn't a good change given that it duplicates data that's already in the archive cache that could instead be extracted very easily (the cost of extraction vs. copy is negligible).As a side note, extracting the sources from their tarball would be way faster on Windows that extraction + copy for the same reason detailed in #5741
As it currently stands this PR is a WIP. It compiles but still has many unsolved issues (in particular it breaks caching for non-pinned dev packages) and the reftests fail, but i'm opening it anyway to show the rough area of the code that needs to change and in case anyone wants to take over this work while i'm doing something else.Post-WIP edit: Further improvements can be made by extracting the tarball directly into the build directory and would also f.i.x #6693, but i think it is simpler to do that separately in a future PR.