This might get down voted, but here goes. I agree that the Nix documentation can be confusing and even contradictory at times. I agree that there is a steep learning curve to getting started with Nix and NixOS. However, I do not think that Nix has ever claimed or intended to be a tool that can be used easily by everyone.
If the ideas seem pointless and confusing to you, that's fine, don't use it. There are bunch of great package managers and linux distros that will probably suit you better. But, if you see the need for declarative package management, then Nix is for you.
I'm a phd student in computational science. I use a bunch of different packages on my computer and I need to know exactly what I've installed and I do not have time to debug problems from dependency hell. Nix and NixOS have been two of the most interesting and useful tools I have every used. Not only have a gained huge productivity boost from having reproducible environments and declarative package management, but I've thoroughly enjoyed the process of learning these unique tools that, for me, are leagues better than any package management system I've used in the past.
If it didn't work for you, I hope you find a tool that suits you better. But that doesn't mean its bad. It might just mean it wasn't intended for you.
I agree with ngruhn that the integration of Nix with my entire development environment is much more seamless then docker images. I think what chriswarbo is getting at is that docker and Nix often have very different use cases.
In my case, I have a few different computers, and I declare all of their environments with NixOS in the same directory. Then I use syncthing so that all the builds are automatically updated and in sync. This means that I can develop something on my desktop, and open up my laptop and run it without configuring anything. If it works on my desktop, it works on my laptop and vise-versa.
I do not, although that would be cool. If I make changes to my nixos config I just type "rebuild" in the terminal on which ever computer was affected. I have "rebuild" aliased to "sudo nixos-rebuild switch --flake /home/user/nixos/#machine_name" on each machine.
For example, I make a development environment on my desktop using a flake.nix file and a .envrc file in directory. I type direnv allow in that directory (actually all this is done with a script I wrote so I can just type "dev-env" in the terminal in the corresponding directory". Then open my laptop, syncthing makes it so the entire environment and all the files are already on my laptop. I type "direnv allow" in the directory in my laptop, and then "rebuild" if the NixOS configuration was changed. Then I can run the software that I wrote on the desktop.
For me the big advantage is that it integrates much easier with my development environment. For example if I have a language server for some programming language running in Docker, it’s not obvious to me how to make it talk to my editor on the host. I can expose a port but both the editor and the language server need access to the code base I’m working on. I can mount the code base as a volume but the language server gets confused about absolute paths that are not matching and stuff like this.
Docker images of what? Docker cannot read minds; it has to be told what to put in those images. Nix is a tool for saying what you want; in a more sensible way than the de facto standard of "grab an entire Ubuntu installation, then run `apt-get -y` commands because I don't understand dpkg".
I want to use nix it's the language that hurts for me. I keep trying to get into it, I have it on a mac laptop but Brew has gotten so good since I last used it almost 4 years ago that I go back to that.
Of course this is a whole env thing and not just a package thing but I dunno I'd almost rather build a container and work within that than write nix to get this functionality, the ergonomics of teh language are that galling for me.
I read this a lot and it prevented me from trying nix many years ago, instead I looked to guix and guile. Turns out, I don’t like schemes all that much.
Started using nix this year, language is easy to use. Has very few concepts and syntax, it’s a standard functional language with all those goodies and a clean syntax in my opinion.
I agree. I think the language gets blamed when it's a docs problem, and the docs get blamed because nix scratches the itch for so many different sorts of people. If you're a dev trying to replace poetry you're in a different world than a sysadmin trying to replace ansible, but documenting nix means addressing the needs of both parties. That's hard.
The language is fine, especially compared with its alternatives (e.g. templated yaml, yuck).
Devenv is often used with direnv, which just means that the environment is activated when you cd in to the folder. This is arguably a bit more than just a docker.
> I'd almost rather build a container and work within that than write nix to get this functionality
If you're happy writing shell commands/scripts, then just stick them in the following template:
with import <nixpkgs> {};
runCommand "name-of-your-thing"
{
someEnvVar = "some value";
someOtherEnvVar = "some other value";
buildInputs = [ programs you want to run ]; # e.g. see search.nixos.org/packages
}
''
run any bash code you like here
just make sure the file/folder you want to output
uses the env var "$out" as its path
''
You can always defunctionalise some particular code, to get a more static/declarative DSL. That's already quite a common pattern in Nix, where we can write Nix functions that act as interpreters for some more "declarative" DSL; indeed, many of the "foo2nix" utilities are just interpreters, written in Nix, for some third-party DSL; e.g. the "callCabal2nix" function in Nixpkgs can read the .cabal file of a Haskell project, and there are many other examples. The `builtins.readJSON` function is a particularly useful building block for such functions.
I don't think it's a good idea to defunctionalise the Nix language itself, since the result would either be too inflexibile or incredibly complex. Keep in mind that the Nix language has no idea what a shell is, or any concept of "package", etc. yet it's flexible enough to write functions like `runCommand`, which is used in the example above.
Also, it's reasonable to argue that Nix is already declarative, since that's what the `.drv` files are for! However, they're so tedious to work with that it's preferable to generate them using a "proper" language like the Nix expression language (or Guile Scheme if you're using Guix)!
I agree. Something that is emblematic of the problem: Nix has an implication operator. That's something that virtually no other language supports - it's normally something found in formal verification systems.
The Nix language is designed by people that think everyone is familiar with lambda calculus and function programming. That's fine. But it's crazy to use a language like that for something that everyone has to use.
I bet Nix would be way more popular if it used something like Starlark instead.
> Something that is emblematic of the problem: Nix has an implication operator. That's something that virtually no other language supports - it's normally something found in formal verification systems.
I wholeheartedly disagree. Nix's `a -> b` is just a boolean function, like `&&`, `||`, `==`, etc. It's meant for assertions, e.g.
assert format == "csv" -> depth data == 2;
Most languages allow assertions like this; they'd just be more awkward, like
assert(depth(data) == 2 if format == "csv" else True)
In fact, a common complaint about the Nix language is that it's untyped: which shows just how informal it is!
> The Nix language is designed by people that think everyone is familiar with lambda calculus and function programming. That's fine. But it's crazy to use a language like that for something that everyone has to use.
The Nix language is basically just "JSON with functions" (although the syntax is unfortunately different, since JSON wasn't so popular back around 2003; it does support XML though!). There's only an emphasis placed on functions since they're the USP of the language; otherwise we could just use plain JSON. Lambda calculus is just a simple model of functions; you don't need to know or care about it in order to write functions in the Nix language. In the same way that you can write integer arithmetic in Nix without knowing Peano's axioms, or von Neumann's set encoding of ordinals, or whatever.
I've not used Starlark, but it seems to be waaaay more complicated than "JSON with functions"; e.g. it has loops, which makes the semantics dependent on some notion of time/temporal-ordering, which is sounds like a massive headache.
I guess. Been using Nix for two years and never knew it had an implication operator. It's mostly just json-like config with occasional code to do more complex things. You almost never have to use the heavier parts of the language unless you are developing Nix itself.
I don’t think there’s a beginner-friendly guide to NixOS. I personally learned it (and continue to!) through a number of blog posts and examples on Github. The NixOS manual is a key reference to rely on, but not really a tutorial: https://nixos.org/manual/nixos/stable/
Flakes = combine explicit refs to deps as well as a build description for anything you're making (which can be a lib, app, or entire machine). IMO if you're starting now, use this.
Channel = group of named deps. 'Traditional' nix tracks a channel so when you update it, anything pointed to it also updates.
Home Manager = NixOS lite. Good option to define your preferred environment deterministically if you like another OS.
A dependency at a particular version vs. the latest version of that dependency. The latter relies on a „channel“ snapshotting all available dependencies at a particular version. This is considered to be a mistake in hindsight, hence flakes
The idea of Nix is great, implementation not so much. If everything you need is within their package/ecosystem then things will go fine. But God help you if you want to run some binary that isn't in their package manger, the amount of time in effort and struggle to get a simply binary to run is absolutely insane.
> Good luck getting answers on those questions other than "read the source code" and then followed by "no, not that source code, this branch here".
I experienced a similar situation last week with git-hooks.nix[1], a pre-commit integration for Nix.
I wanted to run biome[2] checks on my repository during pre-push so I wrote a custom hook because git-hooks.nix has pre-defined integrations with prettier and rome, but not biome.
Or that's what I thought. I eventually found out that the rome hook is actually referred as "rome" everywhere but calls biome instead[3]. This wasn't documented anywhere, so I opened an issue[4] suggesting to rename the hook to "biome" and keep the former for backwards compatibility reasons.
As of today, this has been acknowledged by one of the maintainers, whose sole feedback has been to "thumb down" the issue.
TL;DR: It's not just the documentation, but also the code not doing what you would expect. It also seems there's no means to improve the situation other than just forking the project since there's also clearly some kind of communication problem.
There are a fair few nix based dev environment systems (devenv, devshell, devbox) and I've tried out a few, but devenv is the one I've stuck with. I don't typically have complicated dev environments, I just use it for installing packages and adding convenience scripts which will automatically be activated and deactivated with direnv.
I have been using this for a while, great project!
Though something that annoyed me with the recent v1 release is that it changed the default repository where it pulls the package definitions from Nix's official to a fork made by the author.
That is dangerous and also lags behind an incredibly active and large upstream.
If you want to patch things, use proper Nix overrides or apply the patches using devenv code, don't fork a +80,000-big rolling release package repository.
One of my favorite things about Rust is that it uses the file system as the basis for a fast, reproducible, and composable developer environment.
I can unzip a directory, cd into it and cargo build it and for the most part it just works.
The fact that we have added shared libraries and environmental variables and so many other magic incantations that we need massively complicated second systems is an indictment of our culture of complexity.
I believe Go also embraces that level of simplicity.
And when it comes to deployment, instead of needing a simulation of a whole operating system (containers), I can just remote copy a directory and just have it work.
Lots of rust software has dependencies on non-rust software and build scripts that need various things to be in the path or in other environment variables.
For example, anything using bindgen needs libclang, and to make matters worse, doesn't look in the conventional LIBRARY_PATH but requires a special environment variable LIBCLANG_PATH.
There may still be arguments around shared cache between programs though the security inspired rearranging things might thwart that.
I like static binaries a lot. Whole load of failure modes just gone. Building a clang that targets musl by default on glibc systems is loosely practical these days. That builds self contained binaries that only depend on syscall.
(It took me a few days, a lot of cursing at cmakes ideas about cross compilation and patching trunk slightly, but it can be done and enough of the patches stuck.)
Downvote though you may, you misunderstand containers - they are not simulating a whole operating system (that would be a VM), they are using features built into Linux to isolate tasks running on the same OS kernel to be isolated from each other.
I've been using devenv + flake parts and so far it's done it's job perfectly. I build a rust and a node application in the ci. Both local and ci build every time. It will only fail from time to time when i make an update, otherwise no errors at all
I wish just one of these would allow for multi-instantiation of services. I have projects where I need to spin up multiple independent postgres. Instead, many of these tools repeat the patterns used in nixpkgs.
If you’re on NixOS (the distro), the only way to run multiple services of the same type (eg, Nginx) is using containers. These can either be NixOS containers or standard Docker/Podman containers [1]. If you want to run a Compose project, I maintain a tool that help simplify this [2].
As GP implies, NixOS supports running multiple instances of any service you like, but the way some of the modules are written doesn't make it easy. Thankfully one of the advantages of NixOS is that it's pretty easy to add a configuration module to your personal configuration!
In general, you can configure systemd yourself through `systemd.services`, where you can write systemd service files ‘directly’ in Nix syntax. Services for which multiple instances make sense often (unfortunately, not always) provide configuration that allows you to specify multiple instances (i.e. their top-level configuration object will be a list or attrset of instance configurations). I've written a little bit about patterns to do this here: https://twey.io/nix-patterns/inputs-and-outputs/
If you needed, say, multiple instances of Postgres, it's not too challenging to copy-paste the nixpkgs implementation, change it a bit to parameterize the config on (e.g.) a service name for namespacing, then import that module into your NixOS configuration to allow you to define multiple instances. For example, I did this here for the Rainloop email client: https://github.com/Twey/dotfiles/blob/main/nixos/modules/ser...
Grumble, I know, I don't like using containers when I don't need to, though. And I still think it's too bad that greenfield nix projects haven't thought about this more a-priori. Either by making the isolation a default, hidden foundation, or by creating the new module to be multi-instantiable.
I've been a long time user of devenv.sh and can definitely recommend it as a nice wrapper around nix.
I often want to have some packages/tools installed when I'm working on a project and docker isn't always the right fit, which is where devenv comes it.
I basically use it as a generalized version of venv.
- Was working on a paper, set up texstudio w/ devenv.
- Needed to use vagrant and virtualbox, however apt installing them was impossible due to a dependency issue (incompatible on pop os at the time). Trivially fixed by just: devenv init -> add to devenv.nix file -> devenv shell
Not fast at all in my experience. Just running (for the first time) what would take seconds with something like `asdf` could take up to half an hour with Nix.
* Fast -- in what way? How can you even measure the speed of an environment?..
* Declarative -- as a developer, I don't care... Why is this important? Also, declarative means there's a bunch of imperative code hiding behind it. It just makes it harder to debug when things break.
* Composable -- in over 25 years of being a developer not even once did I want my dev. environment to compose with another...
And the article opens up with:
> Simple JSON-like language
Why on earth?.. And things down the road that announce "features" like "run processes"... Is this some kind of April 1st joke that is meant to be funny because its two weeks too late?
> Fast -- in what way? How can you even measure the speed of an environment?..
How fast it installs, how fast it's ready when needed. This can be relevant depending on how you use them. For example, if I let every script run in a separate environment, it would matter if it's ready in 10 seconds or 10 milliseconds.
> Declarative -- as a developer, I don't care... Why is this important? Also, declarative means there's a bunch of imperative code hiding behind it. It just makes it harder to debug when things break.
As a developer, I do care how much work something will offload on me. And declarative is more about the state of the world, than the execution of a process. If there is a trustable system which can manage that world for me, then I will have less work, and fewer problems.
I'm sorry that you've been a dev for 25 years and you've never worked anywhere where microservices and distributed systems are the norm, being able to modularly combine and recombine systems can be invaluable for rapid development and testing.
You really set up your development environment using microservices? How is this subject relevant to development environment setup?
If you are interested in my job history: it includes two of the top-ten largest US s/w companies. My area of expertise is storage (more in the direction of SDS as this was a fashion when I got into this field) and more recently HPC. I'm less familiar with the aspects of modern programming s.a. making Web sites for example, and somehow, I have a feeling that programmers who make Web sites really like talking about microservices. So, maybe this will explain my lack of understanding of how you set up your development environment using microservies.
> You really set up your development environment using microservices? How is this subject relevant to development environment setup?
If you are working on something that involves microservices then you might have several different projects with different development environment needs, all checked out locally. It doesn't mean you're "setting up your development environment" using microservices, it just means that's literally what you're developing.
I think, you are talking about what I'd call the "testing environment". For me, development environment is made of the editor, tools that work on my source code s.a. compilers, linters, maybe language server etc. and things that help me with administrative tasks related to programming (eg. bug tracker, version control, communication channels).
In my practice, it's very uncommon to be able to run even a small portion of the project I'm working on on my personal computer. There's usually no way for me to even attempt to set up anything resembling the system I'm building on my computer. Which is unfortunate, and I'd love to be able to, but it's just physically impossible.
I don't know how typical my case is. I'd assume that programmers working on desktop applications or phone / Web stuff are more likely to be able to fit their entire product or at least a significant portion of it on their own computer. In which case they could think about their testing environment as being part of their development environment. Still, I'd not call it that... I mean, conceptually, it's for testing, so it's a testing environment. You don't need it to do development, it's just coincidental that you have both in close proximity.
Are you saying you never run the code you work on at all? You can't e.g. step through it in a debugger? For testing even the simplest thing you have to submit your code to some external system? If that's what you mean then indeed I think it's pretty unusual.
I've worked on stuff that's not at all web or desktop related (e.g. Materialize, hosted cloud data warehouse software) and still, we had ways of running it locally; otherwise most people would have found it impossible to develop.
If the ideas seem pointless and confusing to you, that's fine, don't use it. There are bunch of great package managers and linux distros that will probably suit you better. But, if you see the need for declarative package management, then Nix is for you.
I'm a phd student in computational science. I use a bunch of different packages on my computer and I need to know exactly what I've installed and I do not have time to debug problems from dependency hell. Nix and NixOS have been two of the most interesting and useful tools I have every used. Not only have a gained huge productivity boost from having reproducible environments and declarative package management, but I've thoroughly enjoyed the process of learning these unique tools that, for me, are leagues better than any package management system I've used in the past.
If it didn't work for you, I hope you find a tool that suits you better. But that doesn't mean its bad. It might just mean it wasn't intended for you.