re-remove dependency on nixpkgs

[leotaku]

I have previously used niv to bootstrap systems that do not have the nixpkgs repository downloaded/setup as well as a replacement for the non-reproducible nix-channel. niv's newly added dependence on nixpkgs for its fetchers makes this impossible.

From what I understand we use the nixpkgs fetchers in order to please hydra, which needs a "real" derivation instead of one that is manually created.

However in #32 it is mentioned that hydra can also be pleased by simply adding the outPath key to the derivation record.

So maybe we can lose the dependence on nixpkgs again?

[leotaku]

@nlewo

[nmattia]

@leotaku yeah I'm having issues with that as well. I have a solution which I'll probably implement tonight. Basically it'll use the builtin fetchers for nixpkgs, though it'll still use the nixpkgs fetchers for the other packages. Would that work for you?

[leotaku]

@nmattia Hmmm... I suppose it would, but it wouldn't really be ideal (as it would mean that niv would always have to pull some nixpkgs checkout, even if that isn't really needed for the project at hand).

What is the reason we are using the nixpkgs fetchers? (How) do they differ from builtins.fetchurl and builtins.fetchTarball?

[nlewo]

@leotaku builtins.fetch* don't work if the restricted evaluation mode (documented in the nix man page) is enabled which is the case on Hydra. Also, the builtins.fetch* don't provide derivations but only output paths.

By using nixpkgs fetchers, we also get better error messages, and it'd allow us to fetch more stuffs, such as Docker images #34.

But I agree it is not ideal to first have to fetch nixpkgs and it would be nice to find something else...

[nmattia]

Quick summary of the issue:

We cannot use <nixpkgs> to bootstrap pkgs in two cases:

1. If there is no nixpkgs=... set in the NIX_PATH (@leotaku 's use case)
2. If NIX_PATH=nixpkgs=./nix is set, where ./nix is the dir with a default.nix fetching nixpkgs from the sources.nix (@nmattia 's use case)

We cannot use the builtins fetchers when restricted eval is on (@nlewo 's use case)

We can check if <nixpkgs> is set; if not, we assume the user has a nixpkgs package in sources.json and use the builtins fetchers for that. That would work for @nlewo and @leotaku , assuming @leotaku has a nixpkgs package. That would still break for @nmattia . We could make it work for @nmattia if we check if <nixpkgs> is set and if <nixpkgs> is not equal to ./..

Generally I believe we still want to use the pkgs fetchers for most packages that aren't nixpkgs, since we will need all fetchers for packages with exotic types (docker, file, etc). We could enforce the convention that there must be a nixpkgs package, that is always fetched with builtin fetchers. That breaks @nlewo . Alternatively we could enforce the convention that there must be a nixpkgs package, that is fetched with builtin fetchers iff <nixpkgs> is not set. That breaks @nmattia .

The good news is that we appear to have an equivalent of the CAP theorem for niv; the bad news is that I can't see a general solution. So maybe this has to be set per-project; we could extend the sources.nix annotations (like the do_update) with extra params. But then that means that sources.nix needs to parse itself... alternatively we could allow the user to tweak parts of sources.nix (e.g. set config in the nix code).

[leotaku]

@nmattia If you choose to have this configurable per project, one easy fix would surely be to just offer multiple sources.nix templates for most common use-cases through niv init.

So for example: (--sources might not be a good flag name)

niv init # equivalent to niv-init --sources default
niv init --sources default # import derivation with key "nixpkgs"
niv init --sources system # simply use "NIX_PATH" nixpkgs
niv init --sources builtin # only allow builtin fetchers, abort on other fetchers

To me this sounds like a reasonable and relatively easy to implement method, opinions?