Thanks for taking the time to write all this out, including the examples.
> I really don't see how it is "unacceptably bad"
1) Like you pointed out, documentation will advise against this approach. Is it it even portable (what the original thread of discussion was about)? After my initial usage of nix, I switched to following the "best practice" of writing derivations and specifying system dependencies in the configuration.nix .
2) The nix commands are undeniably more complex than their brew equivalents. If you've used nix enough to memorize them, this probably goes away. But to a casual user who only interacts with nix once in awhile, it's way easier to remember "search" than "--query --available" or "-qa". "search" also feels like a higher-level unstructured-human-readable command.
3) Even "nix-env" is sort of weird as a name, because it begs explanation of why "-env", and that starts pulling in more advanced concepts that maybe a user doesn't initially need to be exposed to. It also means then you have to remember when to use "nix" and when to use "nix-env".
As for the rest, consider the use case of setting up an environment with Python packages:
And the problem is, by this point you've long ago lost most software developers. They gave up, wrote a dockerfile with `pip install` commands, and used `sed` or `patch` or something to make the small-scale changes.
And I have to admit, while the procedural solution is not as elegant nor reproducible nor crossplatform, there's less cognitive overhead and novel syntactic constructs required than switching to a language and library that forces you to distinguish between `override`, `overrideAttrs`, and `overridePythonAttrs`.
> Like you pointed out, documentation will advise against this approach
If by "documentation" you mean "community documentation", indeed people like to point out that You-Re-Holding-It-Wrong-And-Should-Not-Do-This-As-It-Is-Not-The-Proper-Nix-Way, with which I disagree because Nix is a tool, and wielding it in a way that solves one's problem is one's prerogative. I do agree that doing it in other more complete ways (e.g configuration.nix, shell.nix, flakes) unlocks further helpful stuff but that does not mean doing it this way is not helpful in its own right, especially as the more complete ways come with more knowledge needed, thus facing newcomers with a steeper learning curve.
> After my initial usage of nix, I switched to following the "best practice" of writing derivations and specifying system dependencies in the configuration.nix .
Which is how it should be, walking along the learning curve: start small, walk forward, and stop at any point that solves your problems in satisfactory ways. configuration.nix is nixos/nix-darwin, but to me that's already a bridge too far to get simple early adoption from newcoming folks.
I find it more friendly to let people start with nix-env, possibly shoving a sequence of nix-env -iA foo in a shell script, then as they get familiar with it, progressively have them realise that they can set up their dev packages per project easily with shell.nix, or their system ones with nixos/nix-darwin configuration.nix instead of lecturing them in doing it "The Right Way".
> If you've used nix enough to memorize them, this probably goes away.
That's my "learning curve" point, and true for any tool. I've seen the same resistance with Docker back then and now everyone takes it for granted, but it was equally mystifying for people at first.
> it's way easier to remember "search" than "--query --available" or "-qa"
I would agree, nonetheless the situation is really not much better for some other package managers that folks seem to have no problem with, e.g pacman is loved yet has an eerily similar --query/-Q, apt-get/apt-cache/dpkg are all over the place in general Debian-based management, including Dockerfiles.
By "have no problem with" I mean I never heard anyone saying "OMG apt/pacman is absolutely inscrutable I cannot use this", which Nix seems to trigger every time.
I will readily admit though is that whatever https://search.nixos.org/packages is doing should be available via command line. Big gripe on my side but it's not an insurmountable barrier.
> Even "nix-env" is sort of weird as a name, because it begs explanation of why "-env"
Is it? People do not seem that taken aback about using "rbenv" or "pyenv" or "virtualenv". I don't think "env" is exactly uncharted territory.
> As for the rest, consider the use case of setting up an environment with Python packages [...] And the problem is, by this point you've long ago lost most software developers
Whoa, this is going much farther than what prompted my initial comment. Fair enough though, I'm the one who made it devolve into project-level management with Python/Ruby dependencies. That said, folding language dependency management into Nix is definitely a very rough area of Nix, one that needs much development to be made easier.
That is exactly why I am advocating for this less dogmatic approach of doing things: have Nix handle your systemwide (via simple nix-env) or per-project (via shell.nix) tools and delegate language dependencies to the dependency tool that people use, reaping both the benefit of nix for setting up a dev env and the benefit of people knowing their tool of choice for their language.
Language dependencies have a huge impedance mismatch between dependency management tools and package managers. There's exactly the same kind of impedance mismatch problem with trying to package some or all language dependencies with a package manager such as apt or pacman or homebrew. Trying to package a gem or wheel in a deb is an exercise in frustration. I don't know if it's still the case but there was a time where npm install -g, pip install --global, or gem install would install stuff where e.g apt or pacman would, and thus screw things up badly.
So I would recommend for a long while: do not attempt to have nix handle everything down to language dependencies at the beginning because the impedance mismatch between these dependency management tools and nix is making it hard. The current state of Nix has no descriptive abstraction on top of it so you are faced with injecting stuff and getting to grips with Nix internals.
I do believe that over time this is solvable though, e.g NixOS modules generally provide nice override points that don't require one to dance with overrideAttrs and self: super.
> They gave up, wrote a dockerfile with `pip install` commands
Interestingly enough, that's sort of what I recommend for newcomers, except not with Docker:
# shell.nix
{
pkgs ? import <nixpkgs> {},
}:
let
# get these python packages from nix
python_packages = python-packages: [
python-packages.pip
];
# use this pyhton version, and include the above packages
python = pkgs.python39.withPackages python_packages;
in pkgs.mkShell {
buildInputs = [
python
];
shellHook = ''
# get python version
export PYTHON_VERSION="$(python -c 'import platform; import re; print(re.sub(r"\.\d+$", "", platform.python_version()))')"
# replicate virtualenv behaviour
export PIP_PREFIX="$PWD/vendor/python/$PYTHON_VERSION/packages"
export PYTHONPATH="$PIP_PREFIX/lib/python$PYTHON_VERSION/site-packages:$PYTHONPATH"
unset SOURCE_DATE_EPOCH
export PATH="$PIP_PREFIX/bin:$PATH"
'';
}
Which makes nix-shell an equivalent of source venv/bin/activate, and then just pip install -r requirements.txt. The interesting bit is that one can control non-python dependencies for python things, e.g if a native python package depends on some C library or a compiler or a CFLAGS. Also, compared to a Dockerfile it is not RUN order dependent. There's also much less fuss about darwin vs linux or intel vs arm. You can stuff CLI stuff in there, be it shellcheck or rsync or fswatch and be sure that everyone is on the same page, no it-fails-oh-wait-my-homebrew-python-is-not-on-the-same-version-as-everyone-else, no it-fails-oh-wait-I-forgot-to-docker-pull-or-docker-build. It walks around a ton of little issues that increase friction with extremely little fuss.
My point is not that Nix is not hard, Nix can be hard... if you go full-tilt, but it can also be immediately useful even with only small, approachable bits here and there. I remain convinced that the people that spend hours on end cobbling up Dockerfiles full of sed hacks and conjuring contrived docker run commands to run on a dog-slow Docker for Mac should not be objectively turned away by 'nix-env -iA' instead of 'brew install' or even making sense of that shell.nix.
That's why I feel like the community's response in the lines of "don't use nix-env" or "flakes!" combined with "here's this nixlang eldritch horror and I can't seem to be able to do this" is unhelpful in reducing the general sentiment that Nix is only approachable - and thus useful - if you write Haskell in your sleep.
That's why I'm trying to make it better with my limited ways so that curious newcomers are not scared away and can benefit from a little bit of nix goodness with little effort.
> I really don't see how it is "unacceptably bad"
1) Like you pointed out, documentation will advise against this approach. Is it it even portable (what the original thread of discussion was about)? After my initial usage of nix, I switched to following the "best practice" of writing derivations and specifying system dependencies in the configuration.nix .
2) The nix commands are undeniably more complex than their brew equivalents. If you've used nix enough to memorize them, this probably goes away. But to a casual user who only interacts with nix once in awhile, it's way easier to remember "search" than "--query --available" or "-qa". "search" also feels like a higher-level unstructured-human-readable command.
3) Even "nix-env" is sort of weird as a name, because it begs explanation of why "-env", and that starts pulling in more advanced concepts that maybe a user doesn't initially need to be exposed to. It also means then you have to remember when to use "nix" and when to use "nix-env".
As for the rest, consider the use case of setting up an environment with Python packages:
https://nixos.wiki/wiki/Python
This requires memorizing more commands and syntax.
And then if you want to change something about the configuration of that Python package:
https://stackoverflow.com/questions/70395839/how-to-globally...
And the problem is, by this point you've long ago lost most software developers. They gave up, wrote a dockerfile with `pip install` commands, and used `sed` or `patch` or something to make the small-scale changes.
And I have to admit, while the procedural solution is not as elegant nor reproducible nor crossplatform, there's less cognitive overhead and novel syntactic constructs required than switching to a language and library that forces you to distinguish between `override`, `overrideAttrs`, and `overridePythonAttrs`.