Skip to content

Improve lazy trees backward compatibility#56

Merged
grahamc merged 8 commits into
detsys-mainfrom
references-without-context
May 8, 2025
Merged

Improve lazy trees backward compatibility#56
grahamc merged 8 commits into
detsys-mainfrom
references-without-context

Conversation

@edolstra
Copy link
Copy Markdown
Collaborator

@edolstra edolstra commented May 7, 2025

Motivation

When you apply builtins.toString to a path value representing a path in the Nix store (as is the case with flake inputs), historically you got a string without context (e.g. /nix/store/...-source). This is broken, since it allows you to pass a store path to a derivation/toFile without a proper store reference. This is especially a problem with lazy trees, since the store path is a virtual path that doesn't exist and can be different every time.

For backwards compatibility, and to warn users about this unsafe use of toString, we now keep track of such strings as a special type of context.

(The original lazy trees branch had something similar, but there the context was keeping track of SourcePaths rather than virtual StorePaths.)

This makes Nix with --lazy-trees pass the entire flake-regressions test suite.

Context

edolstra added 2 commits May 7, 2025 18:24
When you apply `builtins.toString` to a path value representing a path
in the Nix store (as is the case with flake inputs), historically you
got a string without context (e.g. `/nix/store/...-source`). This is
broken, since it allows you to pass a store path to a
derivation/toFile without a proper store reference. This is especially
a problem with lazy trees, since the store path is a virtual path that
doesn't exist and can be different every time.

For backwards compatibility, and to warn users about this unsafe use
of `toString`, we now keep track of such strings as a special type of
context.
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 7, 2025

@github-actions github-actions Bot temporarily deployed to pull request May 7, 2025 17:09 Inactive
@github-actions github-actions Bot temporarily deployed to pull request May 7, 2025 17:42 Inactive
@github-actions github-actions Bot temporarily deployed to pull request May 7, 2025 19:08 Inactive
@github-actions github-actions Bot temporarily deployed to pull request May 8, 2025 07:27 Inactive
This is already done by consumers of builtins.toJSON (like
builtins.toFile or builtins.derivation), so we can delay this until
it's actually needed.
@github-actions github-actions Bot temporarily deployed to pull request May 8, 2025 13:44 Inactive
@edolstra edolstra marked this pull request as ready for review May 8, 2025 13:54
Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/ci.yml Outdated
edolstra and others added 2 commits May 8, 2025 18:12
Co-authored-by: Graham Christensen <graham@grahamc.com>
@edolstra edolstra added the flake-regression-test Run the flake regressions test suite on this PR label May 8, 2025
@github-actions github-actions Bot temporarily deployed to pull request May 8, 2025 16:50 Inactive
@grahamc grahamc enabled auto-merge May 8, 2025 16:54
@grahamc grahamc added this pull request to the merge queue May 8, 2025
Merged via the queue into detsys-main with commit 7c47777 May 8, 2025
36 checks passed
@grahamc grahamc deleted the references-without-context branch May 8, 2025 17:28
@MattSturgeon
Copy link
Copy Markdown

MattSturgeon commented Feb 4, 2026

For backwards compatibility, and to warn users about this unsafe use of toString, we now keep track of such strings as a special type of context.

The warning added seems to also trigger when intentionally removing context using builtins.unsafeDiscardStringContext. We're seeing this in nixpkgs and home-manager,* as we intentionally remove any accidentally added string context from the documentation's options.json

* and anywhere else the usual NixOS options-doc pattern is used

@Malix-Labs
Copy link
Copy Markdown

@grahamc @edolstra please see the comment above, related to nix-community/home-manager#7935 and NixOS/nixpkgs#482766

friedenberg added a commit to amarbel-llc/eng that referenced this pull request May 1, 2026
Make the home-manager default (`manual.manpages.enable = true`) explicit
so the warning seen during `just build-home` has a documented anchor.

The warning fires inside nixpkgs' `make-options-doc/default.nix`, where
`optionsJSON` does `builtins.unsafeDiscardStringContext (builtins.toJSON
optionsNix)` to keep the manual's closure from dragging in every nixpkgs
module file referenced by `options.<...>.declarations`.
`disallowedReferences` is the actual correctness guard; the warning is
cosmetic and Determinate-Nix-specific (added in
DeterminateSystems/nix-src#56; upstream Nix does not emit it).

Tracking upstream:
  - NixOS/nixpkgs#485682
  - nix-community/home-manager#7935
  - NixOS/nix#14011

If/when this becomes unacceptable, flip the line to `false` (loses
`man home-configuration.nix` but eliminates the trigger end-to-end).

:clown: https://github.com/amarbel-llc/clown
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

flake-regression-test Run the flake regressions test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants