I love nix. I have also wasted far too much time trying to write my own expressions when just typing “brew install ___” would have let my get back to work.
My biggest issue with homebrew is you’re always at head and you always upgrade the world. This is not good when you want repeatable things to happen. My second biggest issue with nix is you can pin things, but it’s pinned to some obtuse hash of the world. I build embedded code and I just want my toolchains to work perfectly, including avoiding the weird cmake bug in >3.25 that keeps dropping —std=20 from my arguments and then failing to build my “modern” c++ code. The amount of work this takes is too damn high with both brew and nix.
Previously, I worked on and delivered a developer command line Swiss Army knife type tool, you have probably seen them. They do everything from build, docker, package, test, submit code, tail logs, etc. These have a ton of dependencies, and when you support Macs you end up using homebrew. Your install can then be broken by someone brew installing something mundane in their own machine. Not good. I tried to use nix for this, but packaging coverage wasn’t enough so I ended up wasting time packaging my own dependencies. Not good. I ended up putting a parallel brew repo up in my private git repo, then using a hidden brew that wasn’t on the user’s regular path to install the tools I needed at the right versions, and only upgrade them after I’d confirmed they worked. Still a waste of time and space but at least it didn’t suck up weeks of my time.
I’m back in a similar space. I’ve got a complex set of tooling I want to share the install with to my teammates so we don’t have random issues with it, and we’re on Mac, Linux, and Windows. I just want to pin to version numbers (and a bunch of it is python with version pinned requirements, installed by scripts that someone wrote who doesn’t use nix and expects a global imperative environment) so I’m back to hosting my own brew repo. This is really what nix should make trivial but to me it’s a leaky abstraction. I keep having to look under the hood and do my own work to make it function.
At some point is it easier to use Docker to maintain a stable runtime environment for the tools than to try to deal with three different native OS environments? That is, wrapping “docker run” with scripts so it looks like native commands. People do this to make a consistent “k8s toolbox”, for example.
I’ve been intrigued by Fedora Silverblue but never got round to trying it out, though I’m tired of potentially fighting with NVIDIA drivers whenever I update my kernel (linus_torvalds.mp4)
Have you looked into it all? I think it uses flatpak and containerizaton to separate the base OS from everything else.
Not sure if it would help with my driver issues though.
Don't you just need to mount the UART device file to the container using a `-v` volume flag? That doesn't seem like a big deal if your container saves you from doing the PITA setup part.
The tools being used often expect to be on the machine, with access to the remote device to debug, directly.
Lots of MCUs ship an IDE that you have to use, with a specific OS and setup. You can’t just roll your own without literally re-implementing the hacks and magic someone did to make the IDE work. Talk about a leaky abstraction. These are not world class tools, they work in one specific way using one golden path (that barely works.)
Little to none of it will be helped by docker. Trust me, if there was an easy button for this I would have pressed it already.
Flakes help a lot with granular version pinning. It feels weird to have multiple nixpkgs at different commits as inputs, but it's not too bad if you imagine them each as a distinct flake repo for the version of package you need (and name them accordingly).
NixOS is complicated, it’s a new paradigm. But one thing I found very easy: getting started. I just installed NixOS Gnome on my laptop, everything worked. I added vscode , git and Firefox to the packages and I was productive. Now I try to learn bit by bit, using Nix and learning. But all the while I do have a useable laptop that just works. When I feel like I exploring, I explore and learn.
Even better for getting started: Using nix as an additional package manager that won't fight with whatever pre-existing OS uses. My first exposure was using nix instead of homebrew on OSX.
This seems emblematic of the general direction of technology. It's an approach to managing system complexity that works by adding additional complexity and handling that combination well as long as you use their tools. Nix isn't special in this -- all (non-derivative) Linux distros do exactly that, though that's probably little more than an extension of having written a package manager. I dislike higher level tools like this because they discourage understanding of anything below them. The filesystem is no longer organized in a human-readable fashion, it is an implementation detail.
I'd love to see more concerted efforts going the other direction -- managing complexity by working to make the system simpler and compartmentalizing additional functionality. System snaps, now packaging lxd and cups, are probably the closest mainstream example today.
IMO, the complexity is an important aspect of the Sisyphean drive that life seems to have to reverse the action we call the second law of thermodynamics. I highly recommend reading The Animate and the Inanimate[0] by William Sidis.
However, I do agree that there is a lot of unnecessary complexity in our tech stacks today; though I blame much of this on how young the concept of computing and information theory still is. We're still exploring, we're still building, and it's really exciting to me to have the opportunity to work in a space that is only in its infancy. There is so much room for almost anyone in the field to bring innovations and improvements, if only so many of us weren't slaves to grinding our bones, and finite time, to dust creating things of dubious moral value under an almost singular focus on monetary wealth and the threat of death and ostracism.
I must say that as far as I'm concerned, Nix is very much emblematic of simpler technology. Nix tries to solve problems (such as build impurities) very much by trying to fix things at the source (and contributing back to upstream!), rather than slapping things that work on top of each other, mindlessly using magic like containers. That's one of the reasons why I adore nixpkgs, that they have heroically attempted to fix problems in packages at their sources, and have very much succeeded.
> rather than slapping things that work on top of each other, mindlessly using magic like containers.
I have exactly this problem. I need to run centos7-era binary application on rocky9, and of course, it does not compile on the new gcc compiler in rocky9, and also some libraries are missing/have changed too much.
I was thinking I will run the app in a centos7 container on rocky9 machine, but this creates lots of unwelcome complications and additional work.
I'm not very familiar with Nix, but it seems one could install Nix on rocky9 and then somehow use it to build my application against the centos7 devel libraries. Do you think this is a plausible pathway for compiling and running such an old application? It would be great if I could just compile the old app on the new system and forget about containers.
> "Nix is quirky, unique, and a little rough around the edges. Debugging it, like a lot of other things, is frustrating. But the benefits I get from using it currently outweigh the disadvantages, providing enough incentive for me to keep on investing in it..."
OMG, this is almost exactly the thinking/feeling that has kept me happily addicted to linux distros (and FLOSS in general) for almost 2 decades! And, now, hearing this about Nix, seems quite tempting. :-)
> "What I find to be most useful for me is declarative system management via NixOS. My memory span is virtually nonexistent nowadays, and having a way to declare my servers completely idempotently is an incredible way to avoid confusion and stress. I like having that philosophy extended to my personal projects and user environment, as it ensures a level of consistency and reproducibility that I haven’t found anywhere else..."
For me, as much as i love diving into arcane topics around linux ditros, etc., there is still something to be said for limits on patience. so for some things, i actually don't want to spend time and attention, and want the benefits of a declarative approach. Maybe its an age thing for me, but moreso a thing of patience...and Nix here is ticking the right boxes.
I’ve been addicted to using Linux for nearly a decade, so I relate to this. I’ve been using Nix off-and-on for 4-ish years, but have really invested/contributed to it this year.
Making Nix (particularly with flakes) my primary dependency this year has shifted my focus away from the operating system/Linux distribution I’m running on to making a totally portable env. I.E. I can be pretty comfortable on any OS, assuming I can use Nix.
It’s been really fun and useful for work and my homelabbing.
Ah, ok yeah I think it would actually be less work than how you have it setup which is why I asked. You could also just make them 'profiles' where including the files is the same as enabling the feature.
> Something that is genuinely terrifying to set up to me is GitLab. It just has too many moving parts. To deploy it with NixOS, I have this in my configuration:
To me, my first thought is to always deploy applications through containers (Podman + Systemd is my personal preference).
I see this as at least some isolation, I know containers are not a fool proof security system, but it at least some other layer, simple to deploy elsewhere, etc.
Is there a reason to preference running something "natively" in Nix over just using containers? Is Nix giving me anything if I mostly deploy containers? My servers are bootstrapped via pyinfra, and generally I just need to setup ssh, wg and a container runtime, so config drift isn't really much of an issue.
You can run containers on NixOS (also through systemd) and get the same benefits explained in this article. NixOS supports both Podman and Docker as a runtime. There is also the option to run native NixOS containers. See: https://nixos.wiki/wiki/NixOS_Containers.
This is what I'm using, I've written a module for Podman pods (which behave a bit like docker-compose stacks, though with a lit less powerful networking).
You can also run NixOS containers[1] - so you have a "real" NixOS configuration for your service that is still separated from the rest of the system via systemd-nspawn.
Regarding the linked page about containers: a major gripe of mine is when people make content which includes "don't do this part in production!" - but then never bother to explain what the production equivalent is. If it's out of scope for your guide then fine, but at least provide some keywords and links.
The big advantage of using nix modules to run a service is they usually work out of the box, with sane/secure/reasonably production ready defaults. Similar to why you might choose to use a Helm chart to deploy to a k8s cluster. Quality of modules varies greatly, though, depending on their popularity.
Common services usually have undergone some amount of hardening as well, so you probably aren't giving up much in terms of security vs. containers. Again, your mileage varies tremendously depending on the package, so I do strongly recommend reading the source of the nix module of security is a concern.
I also find that consulting the nix source for a service lets me quickly understand the different pieces that go into a deployment - this may not be an advantage to you if you aren't inclined to dive into nixpkgs source on the regular.
Finally, if you are at all bought in to Nix/NixOS then you will greatly appreciate being able to configure your services using module options that have already been created. You can also run docker containers in NixOS, but you'll experience quite a lot of friction if you want to expose service configuration as nix options. Using an existing NixOS module means someone else has done that work for you.
In addition to what was already said: systemd exposes (some of?) the same isolation features that are used by podman et al. to it's services as well. This can go as far as making the services software see only its state directory and nothing else, for example. DynamicUser is also a pretty cool feature in that realm. Some of the NixOS modules use these features to provide isolation for the services they define. In my experience these features are used much more extensively in NixOS than in other distributions.
As a somewhat avid user of Nix, I can say that this is by far the best piece of writing that serves as an introduction to Nix that I've read. It leaves nothing unsaid, whilst serving as a perfectly crystallized version of what I'd want to describe if someone asked me what Nix is.
I really like Nix, it seems like it has the potential to be the future of Linux, but it has enough issues that I'm not sure I'll be switching anytime soon. It still has the feel of something meant for Linux enthusiasts, not something that aspires to be the next Red Hat.
They do everything in a full programming language. That seems like an unnecessary layer of complexity. Other packages managers seem to mostly avoid the need to ever use the programming features, if they have any. Setup.py files are mostly just a static config, sometimes reading stuff from a file, only occasionally having any logic.
Nix seems like it's trying really hard to let you do whatever unusual stuff you want, rather than focusing on making common things easy.
I think a lot of the issue is that it's source-based, and they are aiming for purity and immutability throughout the whole process, above and beyond what other tools do.
They use unusual terminology and invent new concepts a lot. Like, a "derivation", which I understood at one point but forgot, but is like a Makefile equivalent, you make one with mkDerivation.
When you make a snap package, you pretty much just say "These versions of these .deb packages and these other snaps should be available in the environment of this package" and run it.
You can do "snap run --shell mysnap.mycommand" to mess around in the bash shell that your command would run in. It does pretty much everything I wanted Nix to do, and it's super easy.
I'm sure there are actual reasons why Nix isn't as easy as Snap though ,maybe the scope of the problem is just bigger, but it seems unnecessarily low level and hacker oriented rather than "just works" oriented, and the Nix website doesn't quite make it obvious whether this is a red had style production ready attempt, or am Arch style tinkerer OS.
> When I, like many other computer programmers, tell my friends who don’t write code that “it’s a miracle that modern technology even works”, Nix is one of those things that I’m referring to.
Interesting. Since I've started using Nix for managing my environment & projects, it's the only thing that I'm confident will work. I have no idea how the RPM & DEB ecosystems, among others, work when random dependency breakage can happen all the time. This is extra true when building Docker images whose Dockerfile's second line is almost always `apt-get update && apt-get install [list-of-packages]`. I've had so many problems with that process over the years that I've wondered how anything worked at all. Nix was like some heavenly realm where I could be 99.9% confident the build would work every time.
Great point which mirrors my experience as well, even during my foray into gentoo (ostensibly with the hope of fixing these inconsistencies) I ran into system-killing issues with similar root causes. The only thing I regret about nix is that I didn't find it sooner.
By the way, how production-ready is nixos-infect for installing NixOS on VPSes that don't support NixOS out of the box? I've been meaning to do some new deployments with NixOS, and nixos-infect seems to be the recommended way from several resources, but the warning on nixos-infect's README scares me a little.
I think things are pretty good with nixos-anywhere[1] now. I made a video last month walking through the process of provisioning a VPS on Hetzner Cloud[2] if you want to see what the process looks like in real-time.
Wow, it looks pretty nice! I see that it even provides a Terraform module: "All-in-One: This is a consolidated module that first installs NixOS using nixos-anywhere and then keeps it updated with nixos-rebuild. If you choose this, you won't need additional deployment tools like colmena." That would a great thing if its solid.
I've never heard of kexec before; not very familiar why installing NixOS using kexec is only supported on x86 but not aarch64.
I wonder what are the advantages and downsides of nixos-anywhere v.s. nixos-infect.
I have been running NixOS on my laptop and on my homelab.
I would not use it for a production system. Currently trying to migrate off of it to, I dunno, something else and Ansible. Nix has eaten hours of my life and upstream packages break all of the time. Couple of weekends ago the Mullvad module broke. Before that Virtualbox. Before that ZFS. You can have a perfectly fine configuration that you never touch but upstream instability will prevent a nixos rebuild from actually working.
You don't have to use nixpkgs-unstable though? You can always pin your nixpkgs, which I do, and I've as long as one tests that that version runs fine, then it always runs fine.
This seems like a bummer. I'm under the impression that one of the points of nix is to have pain free rebuilds. I wonder how to address this issue and what the source is -- is it a critical mass thing (more users/contributors solving problems)? or an issue arising from inexperienced module maintainers?
From what I understand it's moreso nixpkgs maintenance policies by default significantly reducing the use you can get from nix' pretty solid dependency resolution.
Basically nixpkgs' maintenance policy is that only one version of a package should exist in their repositories. This being intended to reduce maintenance overhead since the nix package manager can freely switch package versions based on the nix channel (read: git branch or commit that nixpkgs is on).
The problem is that unless you meticulously start version pinning all your dependencies, your tooling will always run on the absolute latest version available, regardless of major/minor updates on either stable or unstable. Which can obviously cause problems.
It requires an extra step of care, one not particularly helped by the fact that actual version pinning for an individual package is done by the nixpkgs channel, rather than specifying the desired semver spec so unless you know beforehand what version is at which commit, it probably won't be helpful when it ends up rendering the OS unbootable (not to mention the questionable use in manually typing over git commit hashes but I digress on that).
Pinning in nix is rather easy; either `nix registry pin nixpkgs` or explicitly `nix registry add nixpkgs github:NixOS/nixpkgs/925b70cd964ceaedee26fde9b19cc4c4f081196a` does the job. For specific packages, flakes and flake locks manage this fairly automatically as well. Do they not suffice for you?
For grabbing a package from another version of nixpkgs, its quite straightforward to only use that specific nixpkgs for one package and nothing else. NixOS also keeps old generations around in case the new configurations break anything, so I'm not very worried about the OS being rendered unbootable in any case (although I've never managed to break my NixOS, so I can't attest to having made use of this personally).
I'll also add that I run multiple services (pretty much every hosted project listed in the "projects" section of my homepage) on either a NixOS VM or a NixOS dedicated machine these days (I gradually migrated everything from Kubernetes).
Not only is this now way cheaper (which increases profit margins for paid services I run), but it's also significantly more reliable as a one-person developer and de facto SRE.
"convergent" is "attempts to reach a target state" (like a Terraform apply), whereas "congruent" is "forced to target state" (like destroying/launching a VM).
Yeah, agreed that declarative is probably the wrong word, nix is Turing complete after all.
Also, thanks for the interesting article! It coalesces some concepts that have been tenuously anchored in my mind for awhile. The lack of a story around convergent solutions in the IaC tooling is one of my biggest frustrations and is a primary driver behind my unreasonable love for k8s. Eventual consistency in platform/infra automation is a superpower.
> Take black box binaries, for example: tarballs of pre-built blobs, sans source code, that you “just” have to unpack and run. Nix inherently doesn’t play well with these, and that can make certain things harder to do (see: games).
Sounds like it might be by design, source-unavailable software being a ticking bomb among other issues ?
(Games get their source code lost too, and it also sucks when that happens.)
It may not play well with them ootb considering the lack of an fhs filesystem, but I think nix has shown itself to be adept at handling these cases well. Steam in particular is a great example, and I've even packaged user-hostile blobs like crowdstrike falcon with a flake (so I could run nixos for work).
It might be a huge dockerized stack but it does indeed run out of the box, gitlab-runner too. If I managed to run it without pain, it qualifies as "easy".
> You can’t follow README instructions verbatim anymore: if you want to use something, it has to be packaged by Nix. Or, at least, it should. If there happens to be a Nix package maintained by someone in nixpkgs, great! If not, you better be in a mood to write some Nix code.
That is all the overview that is necessary to understand.
Nix is source-based. If you write your own "package definitions" you can distribute those in all the same ways you would use for source code, since they are source code. nixpkgs for example is a monorepo of many of those package definitions, among other things. Flakes are a (still experimental) approach to (among other things) streamline the options you have outside of nixpkgs inclusion.
If you also want to distribute pre-built binaries you would use a cache. https://cache.nixos.org/ is exactly that for nixpkgs. You can host your own via http(s), ssh or s3. There is also cachix, which is basically a hosting provider for nix caches that is pretty widespread in the community, I think.
Channels are, AFAIU, a reference to some point-in-time/commit/version of nixpkgs. They are exposed in the form of branches in the nixpkgs repository and used in some other places as well. A channel has a set of conditions that need to be met for it to be advanced to a newer version of nixpkgs. For the nixpkgs-unstable channel, which follows the master branch, the packages it contains need to be built by hydra and be present in the global cache before it advances (I don't think this is true for all packages, but this is the general idea). This is to make sure that users who use packages from this channel will mostly find them in the cache (if you use master directly you can often run into packages that you need to compile yourself, because they are not yet present in the cache).
The stable channels are basically "just" branched of from master (or nixpkgs-unstable? not sure) when the given point release is due (there is more to it, for example there is an effort made to make all the packages contained in that release actually buildable, called "Zero Hydra Failures" or ZHF). They will then mostly stay that way apart from the odd backport for security reasons and the likes.
Basically, it is very similar to how a larger software project might be managed with a develop branch and older releases that still receive backports.
(This is my mental model of it anyway, as a user for a few years. There are probably details that might be a bit off or not exactly accurate.)
You can also configure the consumer to consume it over ssh without special setup on the host. But it opens a connection for every single request so it ends up killing the performance [1]
I've been trying to encourage people to give NixOS in VMs a try as a safe way to build up a nice working environment while still maintaining your daily driver until you're interested in/ready to switch. I have a starter template for NixOS on WSL[1] with an accompanying walkthrough[2] if you want to get an idea of how much effort this might require on your part.
My biggest issue with homebrew is you’re always at head and you always upgrade the world. This is not good when you want repeatable things to happen. My second biggest issue with nix is you can pin things, but it’s pinned to some obtuse hash of the world. I build embedded code and I just want my toolchains to work perfectly, including avoiding the weird cmake bug in >3.25 that keeps dropping —std=20 from my arguments and then failing to build my “modern” c++ code. The amount of work this takes is too damn high with both brew and nix.
Previously, I worked on and delivered a developer command line Swiss Army knife type tool, you have probably seen them. They do everything from build, docker, package, test, submit code, tail logs, etc. These have a ton of dependencies, and when you support Macs you end up using homebrew. Your install can then be broken by someone brew installing something mundane in their own machine. Not good. I tried to use nix for this, but packaging coverage wasn’t enough so I ended up wasting time packaging my own dependencies. Not good. I ended up putting a parallel brew repo up in my private git repo, then using a hidden brew that wasn’t on the user’s regular path to install the tools I needed at the right versions, and only upgrade them after I’d confirmed they worked. Still a waste of time and space but at least it didn’t suck up weeks of my time.
I’m back in a similar space. I’ve got a complex set of tooling I want to share the install with to my teammates so we don’t have random issues with it, and we’re on Mac, Linux, and Windows. I just want to pin to version numbers (and a bunch of it is python with version pinned requirements, installed by scripts that someone wrote who doesn’t use nix and expects a global imperative environment) so I’m back to hosting my own brew repo. This is really what nix should make trivial but to me it’s a leaky abstraction. I keep having to look under the hood and do my own work to make it function.