[perf] Cache nixpkgs resolution

[mikeland73]

Summary

This caches resolutions for unlocked nixpkgs (the default) for 90 days on the user's machine. Otherwise, everytime devbox tries to resolve github:NixOS/nixpkgs/nixpkgs-unstable it will download a new version of nixpkgs. This is slow (40+ seconds) and usually nothing changes.

If a user wants to update nixpkgs there are two ways of doing it:

• devbox update without any arguments will update nixpkgs. This is existing functionality.
• devbox update nixpkgs [...pkgs] will also update nixpkgs.

I'm not convinced updating nixpkgs when using update without arguments is best, but I didn't want to break past functionality. It adds 40+ seconds to update operations when there is no package to update.

An alternative command syntax considered was devbox update --nixpkgs. The benefit of doing this would be to avoid conflicts in the future.

I did not use stdenv because it already exists and it is a different package.

Bug fixes:

• Updating a specific package (e.g. devbox update go) no longer updates nixpkgs

Alternative approaches:

• We could try to search the local nix database to try to find existing nixpkgs, but this seemed more trouble than it was worth.

How was it tested?

• Created a new project and logged the cache being used.
• Replaced the cache file data with an old version, created a new project and observed the old version used.
• Tried several permutations of devbox update with and without packages

Community Contribution License

All community contributions in this pull request are licensed to the project
maintainers under the terms of the
Apache 2 License.

By creating this pull request, I represent that I have the right to license the
contributions to the project maintainers under the Apache 2 License as stated in
the
Community Contribution License.

[Allsimon]

This issue is slightly related: #2560

Is it possible to pin nixpkgs to a specific version ?

[gcurtis]

Could we shorten the cache time to something less? Maybe about a week?

There are times where an outdated stdenv breaks packages. This recently happened on macOS where the stdenv was overhauled, leading to weird runtime errors about git or frameworks not being found.

[gcurtis]

This issue is slightly related: #2560

Is it possible to pin nixpkgs to a specific version ?

@Allsimon you can try setting the top-level { "nixpkgs": { "commit": "<hash>" } } in devbox.json or manually editing devbox.lock. I know this is a bit hacky though. Devbox generally uses a search index to map package versions (e.g., go@1.19) to nixpkgs commit hashes, so pinning a specific hash is a little tricky.

[tlelson]

@gcurtis I'm interested in your suggestion but am not quite sure how best to use it in my context.

I have been moving my team to devbox which has been great for reproducibility but bad for CICD build times.

For instance. I have a simple job that uses 3 packages (python3.12, pip, zip) to download some packages and zip them up.

When I run this using the devbox run command in gitlab it spends 3.5 minutes just downloading the packages from cache.nixos.org.

When I run it in a python3.12 container and apt-get install zip it take 30 seconds for the whole job.

What I would like to do, is pin the nixpkgs-unstable version to a specific hash so that I can prebuild a container and ensure that the nix store has the common packages in it.

Is this a viable strategy? Is there a better way to improve slow jobs times?

P.S
There are no builds happening in this 3.5 minutes, just copies.