Skip to content

Lazy trees v2 #27

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
edolstra merged 34 commits into from
May 7, 2025
Merged

Lazy trees v2 #27

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
3751c06
coerceToStorePath(): Improve error message
edolstra Feb 6, 2025
f24ff05
Make `nix flake metadata|update|lock` lazy
edolstra Feb 6, 2025
9e6b89c
lockFlake(): Always compute a NAR hash for inputs
edolstra Feb 7, 2025
2890a2e
Typo
edolstra Feb 7, 2025
3432184
Compute NAR hash for Git archive flakes if --no-trust-tarballs-from-g…
edolstra Feb 10, 2025
307ce9b
Add NAR hash mismatch test
edolstra Feb 10, 2025
228dac4
Merge remote-tracking branch 'origin/master' into lazy-flake-commands
edolstra Feb 13, 2025
c891554
Merge branch 'lazy-flake-commands' into lazy-trees-v2
edolstra Apr 8, 2025
febd28d
Lazily copy trees to the store
edolstra Apr 8, 2025
fa5cb62
Revert unneeded test change
edolstra Apr 8, 2025
f058567
Fix printAmbiguous() / printValueAsJSON()
edolstra Apr 8, 2025
f45db85
Actually ignore system/user registries during locking
edolstra Apr 9, 2025
0cb06d7
Rename FlakeCache -> InputCache and key it on Inputs instead of Flake…
edolstra Apr 9, 2025
3bbf917
Move the input cache into libfetchers
edolstra Apr 9, 2025
dd15c8a
Move getAccessorCached() to InputCache
edolstra Apr 9, 2025
62565ce
Remove unused variable
edolstra Apr 10, 2025
e099a5b
Move the InputCache to EvalState
edolstra Apr 14, 2025
0c0dda3
Devirtualize double-copied paths
edolstra Apr 15, 2025
43a2691
unsafeGetAttrPos: Set string context on store paths
edolstra Apr 18, 2025
a6faa69
Merge remote-tracking branch 'detsys/detsys-main' into lazy-trees-tmp
edolstra Apr 23, 2025
ff85b34
Temporarily run all flake regression tests
edolstra Apr 23, 2025
182edb4
Move mountInput into EvalState
edolstra Apr 23, 2025
9d87ab1
Add a setting to enable lazy trees
edolstra Apr 24, 2025
2aa3655
computeBaseName(): Respect the original store path name
edolstra Apr 24, 2025
88cd822
Fix the nix-community/patsh/0.2.1 flake regression test (again)
edolstra Apr 25, 2025
ae5ac8a
Limit parallelism
edolstra Apr 25, 2025
fef193f
Try namespace runner
edolstra May 6, 2025
577b331
Merge remote-tracking branch 'detsys/detsys-main' into lazy-trees-v2
edolstra May 6, 2025
6f5cfaf
Run flake-regressions with --lazy-trees
edolstra May 6, 2025
630bdff
Re-enable _NIX_TEST_FAIL_ON_LARGE_PATH tests
edolstra May 7, 2025
91cde8c
EvalState::mountInput(): Throw an error if there is a NAR hash mismatch
edolstra May 7, 2025
9bab483
Improve error message
edolstra May 7, 2025
d0a89fa
Put flake_regressions back in the merge queue
edolstra May 7, 2025
f6ad629
nix flake metadata: Show store path if available
edolstra May 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions src/libexpr/paths.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,16 @@ StorePath EvalState::mountInput(
}

// FIXME: what to do with the NAR hash in lazy mode?
Copy link
Copy Markdown
Member

@grahamc grahamc May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this fixme'd?

Copy link
Copy Markdown
Collaborator Author

@edolstra edolstra May 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but we can postpone this for now.

if (!settings.lazyTrees)
assert(!originalInput.getNarHash() || storePath == originalInput.computeStorePath(*store));
if (!settings.lazyTrees && originalInput.getNarHash()) {
auto expected = originalInput.computeStorePath(*store);
if (storePath != expected)
throw Error(
(unsigned int) 102,
"NAR hash mismatch in input '%s', expected '%s' but got '%s'",
originalInput.to_string(),
store->printStorePath(storePath),
store->printStorePath(expected));
}

return storePath;
}
Expand Down
5 changes: 2 additions & 3 deletions tests/functional/fetchGit.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,14 +142,13 @@ path4=$(nix eval --impure --refresh --raw --expr "(builtins.fetchGit file://$rep
[[ $(nix eval --impure --expr "builtins.hasAttr \"dirtyRev\" (builtins.fetchGit $repo)") == "false" ]]
[[ $(nix eval --impure --expr "builtins.hasAttr \"dirtyShortRev\" (builtins.fetchGit $repo)") == "false" ]]

# FIXME: check narHash
#expect 102 nix eval --raw --expr "(builtins.fetchGit { url = $repo; rev = \"$rev2\"; narHash = \"sha256-B5yIPHhEm0eysJKEsO7nqxprh9vcblFxpJG11gXJus1=\"; }).outPath"
expect 102 nix eval --raw --expr "(builtins.fetchGit { url = $repo; rev = \"$rev2\"; narHash = \"sha256-B5yIPHhEm0eysJKEsO7nqxprh9vcblFxpJG11gXJus1=\"; }).outPath"

path5=$(nix eval --raw --expr "(builtins.fetchGit { url = $repo; rev = \"$rev2\"; narHash = \"sha256-Hr8g6AqANb3xqX28eu1XnjK/3ab8Gv6TJSnkb1LezG9=\"; }).outPath")
[[ $path = $path5 ]]

# Ensure that NAR hashes are checked.
#expectStderr 102 nix eval --raw --expr "(builtins.fetchGit { url = $repo; rev = \"$rev2\"; narHash = \"sha256-Hr8g6AqANb4xqX28eu1XnjK/3ab8Gv6TJSnkb1LezG9=\"; }).outPath" | grepQuiet "error: NAR hash mismatch"
expectStderr 102 nix eval --raw --expr "(builtins.fetchGit { url = $repo; rev = \"$rev2\"; narHash = \"sha256-Hr8g6AqANb4xqX28eu1XnjK/3ab8Gv6TJSnkb1LezG9=\"; }).outPath" | grepQuiet "error: NAR hash mismatch"

# It's allowed to use only a narHash, but you should get a warning.
expectStderr 0 nix eval --raw --expr "(builtins.fetchGit { url = $repo; ref = \"tag2\"; narHash = \"sha256-Hr8g6AqANb3xqX28eu1XnjK/3ab8Gv6TJSnkb1LezG9=\"; }).outPath" | grepQuiet "warning: Input .* is unlocked"
Expand Down