> Is [the benefit of this approach] just a matter of it being quicker to do nix-env -iA instead of updating your config and running home-manager switch?
That's definitely a factor. I think since I keep my Nix configurations in source control, somehow modifying the configuration feels more ‘official’, and it also usually comes with extra steps like committing and pushing.
The other thing I like is that it makes it very easy to tell if I actually want/need something: if I find myself installing something over and over (because I periodically purge my profile), I know for sure that it's time to add it to my config. This way I end up pulling less crap I don't actually use into my setup in an enduring way.
Maybe I'd also feel the same way about invoking things via `nix run` or `nix-shell` over time, and that would motivate me to incorporate them into my config ‘for realsies’ by declaring them.
> instead of updating your config and running home-manager switch?
I'm not currently a home-manager user on NixOS. Before home-manager was a thing, I used to define groups of packages using buildEnv and store them in an overlay for very simple de-facto declarative package management, so something like:
nix-env -iA pxc-tui-apps
would install my whole CLI environment at once, and then on NixOS, I'd include `pxc-tui-apps` in my `environment.systemPackages`.
I'm switching to home-manager in my new setup, but one thing I still like about `nix profile install ...` (the flakes-based/next-gen replacement for nix-env, currently) is that it's user-mode/unprivileged, and it doesn't involve rebuilding my whole system (or anything other than the dependencies of just the package I want), even if my nixpkgs checkout/channel/flake registry or whatever has changed. `home-manager rebuild switch` is also unprivileged and also doesn't involve updating my whole system, but unfortunately home-manager doesn't support flakes just yet. You can use it on flakes-based setups on NixOS and macOS via the home-manager NixOS and nix-darwin modules, respectively... but then you're giving up the other benefits I like, because you have to invoke nixos-rebuild after all!
If `nix profile` were some day removed along with `nix-env`, but I had a user-level declarative environment management tool (like home-manager or something more tightly integrated), I could probably get by with just a little discipline about how I choose to edit my configurations and manage sources of Nix expressions and be pretty happy.
But I think the problems with `nix-env`/`nix profile` are pretty solvable, and I think lacking any imperative solution at all will likely put some ‘winnable’ new users off.
I do agree that `nix-env` itself sucks and needs to go, though. `nix-env --upgrade` doesn't really do what people expect, and there's no real reason to use/prefer it over just removing/reinstalling a package. The way that `nix-env` thinks about versions is basically insane, since Nixpkgs doesn't have any real version metadata and `nix-env` just parses attribute names to get the versions back out. `nix-env --query` is clunky and slow (but the new `nix search` is awesome and crazy fast!). `nix-env` was a cool thing for its time, and it's actually how `home-manager` manages its profiles on the backend (which is why flake support is still lacking; enabling flakes disables `nix-env` for your user). But it's like an imitation of `rpm`, and it was made before Nix had a real userbase and opportunities to think about what operations/abstractions/metadata were desirable at that level.
One of the things that's cool about Nix's design is that its design allows you to just bypass the hardest and most annoying problem that faces traditional package managers: dependency resolution. If you want a package manager that's guaranteed to give you solutions that are correct and complete, you need a SAT solver for dependency resolution, and that's NP-complete. By leveraging its quasi-content-addressable derivation approach, Nix gets to avoid resolving dependencies like traditional package managers do— the thing you need is the thing you were built against, and that's that! Similarly, by leaning hard into the Nixpkgs monorepo so that almost all packages live on it and all third-party package sources are built as de facto overlays on top of it, Nix has been able to totally avoid having to think about versions.
Compared to formats like DEB or RPM, Nix packages have very, very little metadata. Nix packagers don't have to declare things like acceptable version ranges for dependencies, what packages provide, what package names ought to be considered equivalent, what other packages it's incompatible with, or even what version number a package has. But `nix-env`'s `-q`, -i`, and `-u` options all imitate the `rpm` CLI, where all of that kind of metadata is necessary and present. And `nix-env` makes all that stuff work by assuming the structure of Nixpkgs and operating on Nix attributes representing packages directly. And it doesn't really make sense in the Nix world as it exists.
Flakes is the first attempt to revisit all these questions the community has basically punted on like ‘how do we want to relate packages to each other in a way that's not monorepo-centric?’, ‘what kind of metadata do we actually want to have for publishable Nix source package artifacts?’, ‘how should Nix code repositories advertise what features/tooling (packages, shell environments, modules, overlays, whatever) they support?’, ‘do we want Nix to actually be able to reason about versions?’, etc. (This also possibly reintroduces the question of dependency resolution. Hopefully not?) I think once we have real, considered answers for questions of that kind, grounded in the experience of the community so far, we can build an imperative frontend to Nix that makes sense and is nice to use.
In the context of within nix packages that's correct. In the context of nix usage that is not.
I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.
Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.
> and it doesn't involve rebuilding my whole system
True. One thing I'm 100% sure is that nix-env -i will do just that and nothing else, whereas nixos-rebuild switch might include something else pending I might have forgotten about because it relies on globals.
> I start a project, I need it to use Ruby 2.7.x, OpenSSL 1.1, node 14.x because that's what the project is compatible with, and I need that to happen on both Darwin, Linux, on whatever cpu arch. Pinning the hashes of "whatever I built it with" won't work.
> Worse, some software such as e.g Ruby encodes their platform at build time (because it matters, because #ifdefs) so currently I'm on darwin20 but specifying "ruby" pulls in RUBY_PLATFORM==darwin17. Nix is currently helpless in face of that.
Right, right. That's a real problem. Gentoo Portage has mostly a big monorepo kinda like Nixpkgs, and in it you can find multiple versions of many pieces of software. Maybe the sensible future in the Nix world would be:
1. Our package attributes include version metadata, perhaps through something like ‘subflakes’ within the repo.
2. Nixpkgs includes multiple versions of major pieces of software.
3. Nixpkgs' top-level remains basically as it is now, in that for the most part only the latest version of something is used as a dependency. Alternatively, we use some kind of a ‘lock file’ that gets published with Nixpkgs. This way `nix profile install` still doesn't have to perform any dependency resolution for packages inside nixpkgs, so its behavior stays fast and predictable.
4. You can add version constraints in defining packages for use outside Nixpkgs, in `nix shell` environments, etc.
Does that seem like the way to go for you, or is your ideal picture something else?
That's definitely a factor. I think since I keep my Nix configurations in source control, somehow modifying the configuration feels more ‘official’, and it also usually comes with extra steps like committing and pushing.
The other thing I like is that it makes it very easy to tell if I actually want/need something: if I find myself installing something over and over (because I periodically purge my profile), I know for sure that it's time to add it to my config. This way I end up pulling less crap I don't actually use into my setup in an enduring way.
Maybe I'd also feel the same way about invoking things via `nix run` or `nix-shell` over time, and that would motivate me to incorporate them into my config ‘for realsies’ by declaring them.
> instead of updating your config and running home-manager switch?
I'm not currently a home-manager user on NixOS. Before home-manager was a thing, I used to define groups of packages using buildEnv and store them in an overlay for very simple de-facto declarative package management, so something like:
would install my whole CLI environment at once, and then on NixOS, I'd include `pxc-tui-apps` in my `environment.systemPackages`.I'm switching to home-manager in my new setup, but one thing I still like about `nix profile install ...` (the flakes-based/next-gen replacement for nix-env, currently) is that it's user-mode/unprivileged, and it doesn't involve rebuilding my whole system (or anything other than the dependencies of just the package I want), even if my nixpkgs checkout/channel/flake registry or whatever has changed. `home-manager rebuild switch` is also unprivileged and also doesn't involve updating my whole system, but unfortunately home-manager doesn't support flakes just yet. You can use it on flakes-based setups on NixOS and macOS via the home-manager NixOS and nix-darwin modules, respectively... but then you're giving up the other benefits I like, because you have to invoke nixos-rebuild after all!
If `nix profile` were some day removed along with `nix-env`, but I had a user-level declarative environment management tool (like home-manager or something more tightly integrated), I could probably get by with just a little discipline about how I choose to edit my configurations and manage sources of Nix expressions and be pretty happy.
But I think the problems with `nix-env`/`nix profile` are pretty solvable, and I think lacking any imperative solution at all will likely put some ‘winnable’ new users off.
I do agree that `nix-env` itself sucks and needs to go, though. `nix-env --upgrade` doesn't really do what people expect, and there's no real reason to use/prefer it over just removing/reinstalling a package. The way that `nix-env` thinks about versions is basically insane, since Nixpkgs doesn't have any real version metadata and `nix-env` just parses attribute names to get the versions back out. `nix-env --query` is clunky and slow (but the new `nix search` is awesome and crazy fast!). `nix-env` was a cool thing for its time, and it's actually how `home-manager` manages its profiles on the backend (which is why flake support is still lacking; enabling flakes disables `nix-env` for your user). But it's like an imitation of `rpm`, and it was made before Nix had a real userbase and opportunities to think about what operations/abstractions/metadata were desirable at that level.
One of the things that's cool about Nix's design is that its design allows you to just bypass the hardest and most annoying problem that faces traditional package managers: dependency resolution. If you want a package manager that's guaranteed to give you solutions that are correct and complete, you need a SAT solver for dependency resolution, and that's NP-complete. By leveraging its quasi-content-addressable derivation approach, Nix gets to avoid resolving dependencies like traditional package managers do— the thing you need is the thing you were built against, and that's that! Similarly, by leaning hard into the Nixpkgs monorepo so that almost all packages live on it and all third-party package sources are built as de facto overlays on top of it, Nix has been able to totally avoid having to think about versions.
Compared to formats like DEB or RPM, Nix packages have very, very little metadata. Nix packagers don't have to declare things like acceptable version ranges for dependencies, what packages provide, what package names ought to be considered equivalent, what other packages it's incompatible with, or even what version number a package has. But `nix-env`'s `-q`, -i`, and `-u` options all imitate the `rpm` CLI, where all of that kind of metadata is necessary and present. And `nix-env` makes all that stuff work by assuming the structure of Nixpkgs and operating on Nix attributes representing packages directly. And it doesn't really make sense in the Nix world as it exists.
Flakes is the first attempt to revisit all these questions the community has basically punted on like ‘how do we want to relate packages to each other in a way that's not monorepo-centric?’, ‘what kind of metadata do we actually want to have for publishable Nix source package artifacts?’, ‘how should Nix code repositories advertise what features/tooling (packages, shell environments, modules, overlays, whatever) they support?’, ‘do we want Nix to actually be able to reason about versions?’, etc. (This also possibly reintroduces the question of dependency resolution. Hopefully not?) I think once we have real, considered answers for questions of that kind, grounded in the experience of the community so far, we can build an imperative frontend to Nix that makes sense and is nice to use.