Another fascinating thing about NixOS is their system of automated integration tests for the whole distribution. You can quite easily set up declarative virtual machines with Nix expressions, and the test suites use this in very cool ways.
> The NixOS release process is novel in that package updates happen automatically as an entire cohesive set. We of course test that specific software compiles properly, but we are also able to perform fully automatic integration tests. We test that the operating system can boot, desktop environments work, and even server tests like validating that our MySQL server replication is still working. These tests happen automatically on every release and is a unique benefit to the Nix build system being applied to an operating system.
> These tests use QEMU and KVM, and spawn one or more virtual machines running NixOS.
He goes on to describe
(1) the KDE desktop test that boots a real graphical environment and uses scripted OCR to detect activity;
(2) the MySQL replication test that verifies correct replication between three separate MySQL servers on three separate virtual machines; and
(3) a BitTorrent test that starts four virtual machines—namely one tracker with Apache serving a torrent file, one NAT router with UPnP (!), two peers of which one is routed through the NAT—and verifies the full process of peer 1 downloading from the seeder and peer 2 downloading from peer 1.
Those tests can all run automatically on a build server for every commit to the package tree... it's quite something!
As you can run Nix on top of any Linux system you can use it wherever you want it.
Some time ago I was having a dependency-hell problem with a C++ program that used particular library versions which were available in Fedora (dev machine) but not Ubuntu (target) - plus there were no static versions of the library to compile the program as a static binary.
So I used nix to build the package for me. It also did not have the right version of one library, but I was able to "fork" the library for my build alone in the same configuration file. Then I was able to nix-copy-closure my program to a remote host, and it copied all dependencies over (and nothing more) and I was able to run it under Ubuntu.
Nix is great. I think it is the more proper solution to the problem people have been trying to fix with Docker.
I run Docker on my Nixos server. Nixos for easy reproducible operating system configuration, and Docker for application deployment so simple even application developers can do it ;)
I feel Docker is a better solution to application deployment than Nix is, but NixOS is the best OS configuration out there hands down.
You usually use Vagrant to provision a machine with a suitable image and then use some other tech to setup your environment on that image. Common choices are Chef, Puppet or Salt, last time I checked.
You can use NixOS to replace both the OS image and the configuration tech.
I've been having great success (and fun!) recently, using Nix to manage the build for an application that I wrote in D a while back. The app builds reliably on a particular DMD version (one that's not already bundled in NixPkgs), and with third-party dependencies pinned at certain revisions. I'd like to make it easy to re-build the whole app, from any machine, with exactly that configuration and with a minimum of fuss. Nix seemed like a great way of coordinating all of that.
After the initial learning curve of the Nix expression language, and how the parts of Nix interact, I was able to write ~25 lines of Nix script that will run the whole thing from scratch: find and deploy the right DMD version, download the right dependencies, and build and install the application.
After a build, I can "copy the closure" of the application to as many other machines as I want, with no rebuilding needed. This guarantees that all dependencies (down to libc) are identical on all machines -- even if the machines already have other versions of those libraries installed. I'm very pleased with the results.
So: there's a learning curve to Nix; but beyond that, things get better, and personally I'm finding the return on time-invested to be very good. If you're interested in building your own software with Nix, I strongly recommend reading the "Nix Pills" articles a few times and doing experiments as you go -- many things didn't click for me at first, but the articles helped things fall into place.
I've had a lot of problems using Nix in the past. Not least of which is that it's terribly hard to search for anything because of the name--not only is the name generic ("nix package manager" -> "Congress nixes aid package...", "nix" -> "*nix", etc), but search engines have a terribly hard time figuring out what nix project I'm inquiring about ("nix package manager", "nix expression language", "NixOS", specific Nix derivations, etc).
Another problem is that the documentation for individual functions is nonexistant. If you want to know how a particular function works, you have to go to the nixpkgs repo in github and try to guess about where the function's definition would live in the massive directory structure, and once you find the definition you care about, you have to read the source code (in Nix expression language--if you're not well acquainted, good luck: the docs are pretty terse and hard to Google for as previously mentioned) which likely references other functions, and since imports don't seem to correspond with the directory structure, you're back to guessing where the symbol you're looking for actually lives. The problem is exacerbated by the dynamically-typed nature of the expression language; other dynamically typed languages at least tell you what type of arguments they expect, but since these definitions aren't annotated at all, you're left guessing. Documentation would do wonders for the project, especially if it's autogenerated like other languages and put in an HTML format so we aren't left hunting around the nixpkgs repo.
What Nix strives to be seems promising, but it's really hard to even evaluate the core ideas for all of the usability issues.
I agree that documentation is a struggle, esp. in the nixpkgs repository. Ideally the maintainers would insist on a "pedantic" level of documentation to help newcomers to onboard themselves.
The Nix Pills series was very helpful, and I like the pedagogical approach he takes there. The Nix documentation itself feels overly dry and academic (which perhaps makes sense given the author's background). I think the world is ripe for a good "Real World Nix" book to lay out an easier path.
I've sometimes struggled with this problem with "go" (=> "golang")
naming big projects something common should be a no-no and it's a pet peeve for most of everyone. for something with a very limited scope it's probably less annoying
I'm a former Arch user who's been using NixOS as my daily-driver at home for about a year now, and I've really enjoyed it, even above Arch (which I'm also a big fan of).
The biggest gain over Arch for me is the safety net it gives you in making configuration changes. Once I've got the machine working, that state can be booted to for as long as I care to keep its pointer around. This means things like kernel upgrades are no longer the anxiety-inducing problem they used to be on Arch for me; if an upgrade fails, I can quickly reboot to the previous state without having to do anything special.
As a result of this, I no longer need to put off kernel upgrades when I'm trying to get something time-sensitive done! To be fair, I was running the ZFS-enabled Arch kernel, which was the cause of most of my problems. However, using the ZFS kernel on NixOS has been anything _but_ problematic; it just works, and keeps working!
Nix/NixOS also makes setting up a new machine just the way I like it a joy. If I move to another machine, need to build a new one, or need to re-image the current one for any reason, my configuration (stored in source control) follows me around without me having to remember all the one-off tweaks I did to make, say, hibernation work correctly.
I haven't yet mastered building packages since I don't find myself needing to compile things from source all that often, but I can certainly feel the pain of learning how to do so; it still feels like quite a mountain to climb before I'd really know what I was doing.
Learning to build your own packages is a bit of a chore, but once you get there, it could not be easier.
I suspect building your own packages for Debian or Redhat would be much more difficult.
Let alone getting that package accepted upstream, which is not that hard with Nixos.
Relatively new NixOS user here: I consider Nix{,OS} and Guix{,SD} the future approach to build a GNU/Linux distro and in general the way to manage a modern OS. Simple as that.
My main pain point with NixOS is it's obscure DSL, instead of the excellent Scheme, my pain point with GuixSD testing is absence of LVM support and less flexibility to create custom installer config than NixOS (for instance I'm a nilfs2 user for all fs and while it's extremely easy to have a root on nilfs2 with NixOS it's tricky on GuixSD).
For the rest it awesome:
- fully reproducible deploy (included NixOps and homeManager)
- safe update, being symlink-based we get poor man's Opensolaris BEs for both config changes, update and even major release upgrade
- can manage nearly all inside home via homeManager
- can deploy a series of hosts via NixOps, like a built-in Saltstack on steroid.
I have been using Nix for about 2 years now at our company and its pretty amazing. In my experience it doesn't make compiling things much easier than normal, but the difference is that once something builds, it just works. ALL THE TIME. In previous companies I spent a ton of time debugging other devs environments that worked fine on my machine, but cause linker errors, breakage due to transitive deps, slightly different compilation artifacts, etc. Furthermore most tools are language specific leaving you to fend for yourself to get the system dependencies or rely on a separate binary distribution package manager. Nix handles everything we throw at it, we have even configured our dev env to roll to the new nightly Rust every week without any effort from the developer. We also get the performance benefits of compiling everything with march=native. Additionally our dependencies and dev environments are cached, so most of our engineers never have to wait for things to compile. The issue we have ran into is due to the fact we are running a traditional distro instead of NixOS, which causes headaches when your code depends on a kernel module (Nvidia drivers for example) or you want to register systemd units. This has led us to have some hacks in our setup to make it work. Ultimately we are thinking of just making the transition to NixOS. Its unbelievable to me that I can deploy an entire mesos cluster with just two lines of code.
FWIW if you enjoy Nix and are interested in using it in production feel free to reach out to me.
brandon [at] standard [dot] ai
Do you have a super quick example that you can include in plugs like this? I am doing my own caching directly with Azure storage (though it works anywhere you can enumerate and upload blobs) [1], but I think Cachix and Hercules are a huge potential chance to excite more folks about Nix and NixOS.
I'm thinking something like a blog post where someone patches the Linux kernel, does a `nix-build | cachix push` and then show how another naive user might do `cachix use && nix-build` and get a magic no-op build. Maybe just a slightly more fleshed out example under the "What" section of the Cachix homepage.
For the non-Nix/NixOS folks, it's an impressive demo, and I suspect even for Nix-folks, it's impressive for hiding all of the signing/upload complexity.
For others reading this comment, Nix + Cachix means that no two people on your team should ever need to spend CPU cycles building two copies of the same binary, or the same docker image, or the same VM image, etc, assuming all of the inputs are the same. It's so cool, I'm so happy to keep seeing people finding out about Nix and NixOS. I feel like what Kubernetes did to application development, Nix does to operating system, toolchain, and application development, and configuration management.
EDIT: follow-up: It look like less than 2 minutes to start pushing to my new Cachix repo. Excellent job with this, it's going to be hard to motivate myself to keep maintaining my scripts with this being so easy.
Thanks! That's very good feedback. We are currently working hard to get all features done and streamlining use. Next phase is marketing to non-Nix devs, probably 2019 :)
One caution though -- being a GNU project, Guix takes a much stronger stance on non-free software than Nix does. In Nix, there's a flag you can enable that will let you build non-free software in the Nixpkgs repository. I don't believe the same flag exists in Guix.
Note that either system could be used to build your own non-free software, this is more a question of what is available in the shared repositories.
There used to be something on the front page, but the result was that too many people thought it was just a nix language replacement when it really was its own implementation of the packaging model that nix pioneered. The Guix documentation gives full credit to nix where it is due. The founder of the project spent years contributing to nix.
NixOS is difficult to start out with, and great after that :p.
Started using it recently for work and it has been reliable thus far. The only issue that I've had was when upgrade to Mojave where a patch was needed. But to do this you clone the nixpkgs repo (of package descriptions), cherry pick the fix, point the package manager to the repo, and it continues to work.
One can create docker images from the machine definitions, so the target need not be nixos, but anything that runs docker.
But if you're not the type of person to pull through difficulties then this distro is not for you (yet).
Another approach is not to patch nixpkgs at all, but just to "override" packages in Nixpkgs outside of the repository, and build them from your own project directories.
A simple example: I wanted to build FFMPEG, but with non-free libraries enabled (which is not the default). Rather than edit "ffmpeg-full" in the nixpkgs repo, I just overrode it:
# default.nix (in some project directory; call "nix build -f ." to run it)
with nixpkgs;
{
my-ffmpeg =
let overrides = {
nonfreeLicensing = true;
fdkaacExtlib = true;
ffplayProgram = false;
qtFaststartProgram = false;
};
attrs = {
name="my-ffmpeg-" + ffmpeg.version;
};
in
ffmpeg-full.override overrides // attrs;
}
I haven't ever tried this myself, but I was under the impression that the `--arg` and `--argstr` arguments to nix-env were supposed to let you change build options. So could you not say `nix-env -iA nixpkgs.ffmpeg-full --arg nonfreeLicensing true --arg fdkaacExtlib true --arg ffplayProgram false --arg qtFaststartProgram false`?
You may well be right. In my first attempts, I did try various combinations of `--arg` and `--argstr`, but I never succeeded in getting a proper build. That's why I ended up using the override approach instead. I understand a little bit more (very little!) about Nix now, so perhaps I should give it another try.
One last tip... If you build this way, it will write everything to /nix/store, but it won't install it into your local profile. However, the 'nix-build' process creates a 'result' symlink in your project directory which points at the package you just built. So Calling 'nix-env -i ./result' ought to install it into your profile as well.
Here's a funny module I wrote that lets you very easily configure your NixOS server to serve simple CGI scripts over HTTP, useful for simple web hooks and such.
It's an example of something Nix is great for: making reusable system modules that involve systemd units, program dependencies, auto-generated configuration files, and so on, all in a very reproducible and hygienic way.
I use NixOS on my desktop, laptop, and on all new server deployments. It's definitely a learning curve, especially considering the documentation is nowhere near as good as it should be yet, but absolutely worth it in the long run; and I rarely run into unavailability of software.
Security updates are indeed sometimes slow to arrive, for the reasons that aidenn0 already described. However! One thing that's particularly interesting about the security update process for NixOS, is that it's an open collaborative effort: https://github.com/NixOS/nixpkgs/issues/23334
More eyeballs are definitely needed for handling security issues, especially since LWN stopped doing their reports, but the process itself is definitely one that has significant potential.
First with the naive, as-described nix there are two options:
* For updates to propagate through all the build servers takes a day or two, so you can just be insecure for 48 hours
* Or, you can always build your packages yourself by setting your update channel to just be the head of the nix repository, but now you won't get binary caches for anything that has had its source updated since the last build. This actually works better than you might think, but is still a no-go for most people.
Both of those suck, so there is something called replaceDependency[1] that lets you define a package in terms of the already built-package and a new version of the library. This lets you quickly get a security update in a shared-library out to multiple slow-to-build packages quickly.
1: https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-supp...
The existing way is to use replaceDependency(), which takes an input as an already built package and as an output the same binary but with different RPATHs for the shared objects. This however is just stopgap, as other approaches are being discussed. Since I'm not running a server, I haven't followed it as closely as I perhaps should
This is patently false. There is a stable release every six months, and a team dedicated to patching the stable release. I'm surprised you missed this, as it's right there in the first sentence of the download page: https://nixos.org/nixos/download.html
You have the option of following the master nixpkgs branch, or one of several "channels" which are essentially tested checkouts of the main nixpkgs repo. There are both stable release channels (i.e. nixos-18.09) and unstable channels (i.e. nixos-unstable).
It's not immediately clear to me that there's an "lts" or "stable" that would allow easy security only updates, preferably in an automatic fashion? Or is this one issue a special case where a) the admin must see the announcement, and b) manually act to secure systems?
Updates are committed to the stable nixpkgs branch. These then propagate through the channel. NixOS has a configuration option to enable automatic upgrades[1], which periodically pulls the latest nixpkgs checkout from the channel you're following, rebuilds the system, and switches to that configuration.
Currently, there's no real support period for non-latest releases, as there's insufficient manpower to support them.
That's not for philosophical or ideological reasons, though; if somebody steps up and takes on the role of coordinating long-term support, it'd be welcomed.
After running some tests I was ready to deploy to a new production server, but unfortunately my hosting provider (OVH) doesn't offer a ready made image.
After trying to use Debian as a starting point, and several failed boots, I threw in the towel... Seems like OVH does funky things with their EFI which doesn't work without some black magic.
For now I'll continue using it on my laptop but it's not living up to its full potential, most of the configuration is for desktop stuff and NixOS doesn't really cover that.
When I get the chance I'd really like to try it on my servers, hopefully sooner rather than later. I can see it working really well with Saltstack for infrastructure management.
I got NixOS running smoothly on Hetzner a few months back without much issue. They even offer a nice ISO to select and do the initial install with without downloading it yourself.
Packet has one-click nixos setup on bare metal with hourly pricing, and nixops supports several (from memory EC2, GCE, Azure, and Hetzner). It also has local virtualization backends for getting yoru feet wet.
Just a note: NixOS is available as a community AMI on EC2, so you don’t have to use NixOps just to get a NixOS AWS box. I have a bunch of them set up just by choosing that AMI.
This is the way I actually started using Nix in deployment. You can run a full Nix installation from within a home directory without any special permissions. Frustrated with the inflexibility of a shared hosting provider running CentOS, we decided to usurp most all of the provided OS & libraries and ran everything thru Nix, giving us fully reproducible control over our dependencies.
My friend at work will never stop talking about Nix and NixOS. The entire world would be a better place if we all just stopped using Docker and Chef and switched to Nix. Every package you could want is in Nix, and it's all on Github. Anything your tool can do, Nix can do it better, except it's congruent instead of convergent.
My rebuttal has always been that NixOS has been out for awhile and we still haven't seen mainstream adoption despite how supposedly great it is. I don't know of any high profile companies using Nix. It appears to be as hobbyist as Haskell.
Is it though? I deploy applications (a mix of Python, node and C++) for which the vast majority of the dependencies are missing, misconfigured or out of date in nixpkg. Sure I could volunteer to maintain dozens of nix packages but the fact remains that forcing all of your deps through a single packaging system is risky. NixOS in particular encourages you to buy into the ecosystem 100% or not at all. That's a big ask, even for small organizations.
Compare that to Docker where your can build your app on top of any base image and use/develop/deploy that artifact from many different environments. I can distribute and build my dependencies in whatever way is appropriate for my application, my team and collaborators can hack on that image from their chosen OS, and the application can be built deterministically and deployed anywhere that can run a docker daemon. No lock-in required: the dev host OS, deployed host OS and container OS are managed independently. Docker is not as "pure" as NixOS but it works under the real-world constraints of heterogeneous environments.
Of course everything would be better if we could all agree on one package manager and one OS and one programming language for that matter. But we can't, so the clear winner is the technology that can leverage this homogeneity into a uniform build artifact (containers). Technologies that require package-level homogeneity across an organization in order to succeed are fighting an uphill battle.
The claimed advantages of Docker over Nix here don't line up with the reality I've seen after years of experience with Docker and Nix, particularly with how they relate to services that wind up deployed with Kubernetes.
I can do almost all the same things that you do with Docker, with Nix. Why would I ever need a different base image if I'm composing my image with Nix? Nix+Docker is the best of both worlds and there's really absolutely no reason to not pair them together.
Besides if for some reason, you can't package something in Nix, or refuse to buy into the `nixpkgs` ecosystem, you can still build a reproducible Docker image with whatever base layer you want (`alpine`, `redis`, whatever), with Nix, and benefit from all of the benefits mentioned in this post and others in this thread.
Deployment time, yeah, they're all opaque containers and they can be running whoknowswhat. But for day to day development, Nix absolutely beats Docker. I've spent way too much of my life rebuilding binaries and container images that already exist. Wasted too much of my life on containers and binaries of unknown, unreproducible origins. None of those things are concerns when you're building your container images with Nix. They're not even a thought I have any more.
It's slowly moving out of the hobbyist realm though - there has been a consistent output of "stable" releases with very little problems upgrading.
One reason it hasn't gotten much traction yet might be because the perceived benefit is too low to convert existing systems: You can upgrade in place, downgrade, manage a whole fleet easily etc, but the people with these problems already have (bad) solutions for those in place.
So there's a lot of inertia, and uptake has to come from new installations.
I too managed a Chef based infrastructure, and it was terrible. I don't like it when simple deploys break a production system because of a missing comma in some config file. With NixOS/NixOps, many issues that Chef has just don't exist.
Generally yes. Better technologically is usually not outweighed by having to learn and support something that has a smaller community base, and there's usually less people contributing to the ecosystem.
Python many not be the best language, but it has a package for everything and it's easy to find answers when you get stuck.
>My rebuttal has always been that NixOS has been out for awhile and we still haven't seen mainstream adoption despite how supposedly great it is. I don't know of any high profile companies using Nix. It appears to be as hobbyist as Haskell.
If it's anything like Haskell, then here's how this goes: there are going to be some problems for which NixOS is not only the best solution but the only reasonably viable solution. For all other problems, it will be inconvenient.
I think it's well known, that Nix has a lot of implementation related problems that prevent its mass adoption. They are fixable, but only by different people and in a different package manager. By those, who care about UX and can understand that learning some weird functional programming language and its ecosystem to do basic package management is not ok, but who can still appreciate Nix's ideas.
NixOS doesn't have directories like /usr/bin, /etc, etc. This defeats the traditional `make && make install` path; instead you have to build a nix package, and be familiar with stdenv.
My concern is that it would be more difficult to install software that doesn't have a nix package built (in particular, from git repo trunks), and also more difficult to have confidence that the software I write would work on other Linux distros. For this reason I chose Arch over NixOS.
In practice NixOS users find these to be significant issues? How do you deal with them?
It takes about 5 minutes to make a nix expression for a "./configure && make && make install", with most of that time being figuring out the dependencies and calculating the checksum of the tarball.
For any standard build tools (cmake, scons, whatever) it's just one extra line. Applying patches is one extra line. For a non-standard workflow, you can supply a shell script for each step (unpack, configure, build, install, &c.). so it's just however many lines that is.
I've had zero issue getting any source-based third party stuff running. Binaries are a pain, but that's always true if you aren't running the exact distro that the binary was built for.
> Binaries are a pain, but that's always true if you aren't running the exact distro that the binary was built for.
This is actually easier to deal with on NixOS than elsewhere, because the tools for handling that case (patchelf, buildFHSUserEnv, supporting multiple versions of a lib, etc) are so well developed.
Indeed, just two days ago I made a derivation for a large binary, which is built against an older Boost (1.58). On most other package managers, this would be painful, but with Nix I just made a derivation for that version of Boost by overriding Boost 1.59, which is in nixpkgs:
My concern is that it would be more difficult to install software that doesn't have a nix package built (in particular, from git repo trunks), and also more difficult to have confidence that the software I write would work on other Linux distros. For this reason I chose Arch over NixOS.
I have a set of overlays for various packages and try to submit updates and improvements upstream. Having some experience with making packages for other package managers, I found making Nix derivations easier than RPM or deb packages and about as difficult as Arch PKGBUILDs. stdenv automates a lot of work, and when a package uses a standardized configuration/build system (autotools, cmake, cargo), most of the work is in finding/adding the dependencies.
I did some Haskell programming ~6-7 years ago, so the learning curve for the Nix language was probably less steep.
The primary difficulty I found is that not everything is documented equally well and you often have to poke through nixpkgs to find examples and/or good practices.
also more difficult to have confidence that the software I write would work on other Linux distros
I guess I have been lucky that I write work-related code mostly in Go and Rust, so building on other systems is never really a problem. But for C/C++ projects I would just compile in an Ubuntu container as well.
One way to deal with them: if you have some software package you only want to build as quick as possible and don't want to bother with any nix stuff, spawn an Ubuntu (or any other distro if the package has instructions for a certain distro) docker container and build the app within that. (Or use steam-run from NixOS which is basically an Ubuntu container)
This also has the advantage that you can use the exact same distribution that the instructions for said package mention. (If the thing you want to compile has instructions for Ubuntu 16.04 no problem just use the docker image for that)
For anything that you want to use on a regular basis, creating a nix package is pretty easy most of the time. If you only want to build a newer version of a package directly from git, you can even copy the package for the stable version and just overwrite the src in many cases.
Regarding software that works on NixOS but not on other distros I think it's usually the other way around: by testing your software on NixOS you uncover issues that would have made your software less portable (hidden dependencies, hardcoded path assumptions, ...)
Making a Nix package is usually quite fun, but if the software has a really gnarly setup (various forked dependencies, strange configuration scripts that involve the internet, npm modules with native library dependencies, etc) it can be bad.
For such cases I like Docker because it lets you easily create little mini-Ubuntus and do whatever you want there.
But I think if you know nix well enough then making some strange setup work might end up a better long term solution that the ubuntu, no? I.e you put in the time upfront in researching how the thing wants it at build time, and then you have the infrastructure ready should the thing change its mind in future versions. Making it build in some less flexible context can become a lot more hackish down the line.
While true, the effort vs. payoff tradeoff doesn't always work out in favour of Nix there. That's largely due to lacking documentation, but I imagine there will always remain 'quick hacks' in use for things that don't need to work for an extended period of time, for example.
That having been said, for anything that needs to work for a longer period of time, it's 100% worth taking apart the install process.
When you're doing development, you can generally use `nix-shell` to install all the pre-requisite tools. If you wanted to compile GNU hello, for example, you'd do:
$ nix-shell -A hello
Which will populate a temporary shell with everything you need to compile GNU hello. Then you can just do `./configure` and `make` like usual, normally. If you know an explicit list of dependencies, you can do that too, e.g. `nix-shell -p libxml2 bison flex pkg-config`, for example. Then run `make` and run `./bin/foobar`, or whatever your application is.
This is OK for development cycles (`make && make test`, things like that) but not for the long term. Most of the time incremental development works just fine this way, though, especially for software you're writing or developing yourself. You just run your commands inside a `nix-shell` instead.
If you want to use a Git repository for something, most of the time you can just 'override' a packages source code input to a new version. In Nix, even source code is actually an 'input' (a parameter) to a build script (a function), so you can just provide a new copy of the source code (pass a different parameter to the function) and rebuild. Of course, someones Git repository may have changed in ways that make the old build script incompatible. You can similarly override/update the build script, and there are a number of mechanisms to do this.
> My concern is that it would be more difficult to install software that doesn't have a nix package built (in particular, from git repo trunks)
This is true and in reality most Nix users are package authors. You will probably find things that are not packaged and you will have to author them. Occasionally, a package may not have some feature enabled (maintainers generally decide these things on their own when it comes to optional features), so you may have to fix it.
Updating/custom source code versions isn't as much of a problem though. Once the Nix expression for some software is authored just pointing/updating the source repository is normally enough to do it.
> and also more difficult to have confidence that the software I write would work on other Linux distros
IME, if your software can work correctly on NixOS with little modification (or extremely perverted build scripts), it will almost certainly work fine anywhere else. In particular as long as your build system obeys "standard rules" like supporting PREFIX and respecting things like `pkg-config`, not hardcoding binary paths everywhere but detecting them, it will normally work just fine. cmake, autotools, pypi, etc all have pretty usable solutions for the extremely common cases. Nix can often quickly unveil assumptions in your build step that would previously be masked (for example, I cannot count the amount of READMEs that have 'apt-get' instructions for build prerequisites, which do not actually account for all the necessary dependencies.)
Realistically I have authored dozens of packages and the ones that cause the most pain are ones that have complex, bespoke build systems with a lot of assumptions built in, or that don't actually specify all the dependencies they think they do, and silently get things wrong. Think packages like OpenJDK, Mono, any web browser, etc etc. Other things occasionally require some bespoke patching to get cleaner integration, even if the build system itself is otherwise nice enough (such as PostgreSQL or Linux itself.)
> In practice NixOS users find these to be significant issues? How do you deal with them?
They can be annoying, and I would say writing build expressions is sometimes a bit more difficult than I'd like -- but not enough to outweigh the actual advantages, which are otherwise tremendous.
Normally I just author packages and submit them upstream for cases like this. In the case where a fix for some issue seems reasonable to upstream I just submit it upstream like a traditional maintainer. Not everything falls into this category, though, so, like any maintainer, you might be on the hook for it.
If you want to use a Git repository for something, most of the time you can just 'override' a packages source code input to a new version. In Nix, even source code is actually an 'input' (a parameter) to a build script (a function), so you can just provide a new copy of the source code (pass a different parameter to the function) and rebuild.
Though you have to be careful with overriding derivations with the same name on which a lot of other derivations are dependent, since it will rebuild every transitive dependency as well. Of course, if you are using a overridden derivation just for your own projects, you could give it a different name.
I'm not totally sure if those graphs are really representative of that, or at least they should be taken with a grain of salt. I've been using Nix for quite a while and I add packages I'm missing semi-regularly (although many of them are minor). The metapackage database on repology.org seems to indicate that it tracks every individual Haskell, PyPI package etc that we ship, so it's actually including thousands and thousands of auto-generated packages, and duplicate versions of everything in the top-level.
I'm not sure if that qualifies for "most packaged software" (since it skews the numbers very sharply in particular directions), but I guess you'd have to define it first. But Nix is different enough that it might be hard to have any kind of straight comparison.
Interesting. Typically Debian has been the distro where you can find anything packaged. Not always the latest version, though, unless you run unstable.
I wish this would become more popular in production, it's just so much better than RHEL and everything. IMHO Nix i the best for servers and enterprise desktop and Manjaro is the best for individual desktop/laptop.
I think it would be really cool if someone made a NixOS-based distro focused on individual users like Manjaro does. NixOS is pretty modular so it wouldn't be all that hard.
We use Nix to manage our entire machine learning pipeline. ML pipelines have crazy number of dependencies and compatibility ranges from Python to Tensorflow to CUDA to Nvidia driver version to hardware card... etc.
I'm so disappointingly late to this thread, but NixCon just happened and they were publishing the videos in real-time on YouTube (major props for doing it that way): https://www.youtube.com/channel/UCjqkNrQ8F3OhKSCfCgagWLg. Might be a bit out of context if unfamiliar with Nix, but it was exciting for me to watch. It feels like momentum is picking up and folks are interested in improving UX and evangelism!
As much as I like the -design- of Nix as a system overall, I would strongly discourage trusting it today for any production or security sensitive use case until some concept of VCS signing, and review/release integrity is in place.
nice concept. can't find information on if this is based on another distro, or if it's an all new one, but the underlying idea is solid as for me it tries to solve one of the fundamental issues with linux which makes people move to things like containers or complicated deployment tools and scripts. this seems a lot more straightforward.
I use it as my daily driver on my laptop and it's great. It's very simple to add new source-based programs.
It doesn't handle binary-only applications though (e.g. NoMachine, games, &c.). It has a couple of options to make those work, and it's almost always possible to get it to work natively, so if there's not already a nix expression for the app I want, I just take the expedient path of having a docker container with an ubuntu user-space, since pretty much all proprietary apps run on ubuntu.
If you don't want to bother with the container, `steam-run` will almost always work. That gives the same Ubuntu chroot (aka. buildFHSUserEnv) as is used by Steam.
Very neat concept, you should Show HN that. Reminds me of another similar project where instead he had everything running in RAM. Totally forget the name of that project now though.
Although the Mac version of the packages is far more likely to be broken or buggy in some way. Unfortunate, but an inevitable consequence of not many people using it.
But also a consequence of having to float on top of non-free/uncontrollable base libraries. The only reason Nix works so well on Linux is because it can be in control of everything down to the kernel. For (presumably) this reason, nixpkgs will tend to only work well for one macos release at a time. There were also no pre-built binary packages for macos last I checked, so, having to build everything, if one dependency has a slightly flaky build procedure, it spells trouble.
It's not based on another distro, although I guess its closest cousin would be Gentoo, which is another source-only distro. By default everything in NixOS is compiled from source, but they use some novel techniques to reduce duplication, and you can setup binary caches for your workplace if you wish.
Edit: apparently packages are distributed in binary by default, but if you customize options it’ll be compiled locally. My mistake.
NixOs is the amalgamation of 3 things: Nix, Nixpkgs, and the other bits it takes to make an os.
Nix is a programming language and a packaging tool. It allows packages to be programmed in a type safe way, then executed to install the actual libraries or binaries. This is the area of the system that I actually know the least about.
Nixpkgs is a large collection of packages for Nix. The community really has worked on this set of stuff like gangbusters, it's quite impressive how many things they've wired in from scratch. Each package specifies its dependencies, and the various options it accepts to tweak various options. A good example would be setting the data directory for the postgresql service.
NixOs combines the previous two items along with the other bits it takes to make an OS work. So they provide official installation instructions and media, configure how Nix is going to actually be run on NixOs (as compared to say, OSX), provide stable channels for various versions, etc. Basically all the regular things you expect out of a distro.
Personally, I'm not exactly sure if this is where Linux is going to go, but I think that this is how Linux would have been built if disk was much cheaper back in 1970ish. A lot of the regular problems of systems management are completely solved in NixOs.
Need two different versions of the same shared library? No worries, they can each have their own copy, and Nix will automatically make sure the correct one, specified by the hash of all the build flags, will be the one that's loaded into memory for said program.
Have a shared system? Great! Users can install userland programs without root privileges. Because items are inserted into the nix store idempotently and paths are not universal, I can install vim without any other users seeing that, and without root. The only thing that takes root privileges are services (e.g. Postgres), or installing programs for universal access.
Sadly this does come with some downsides. A lot of programs over the years have come to make very bad assumptions about program location, or the ability to modify the PATH variable. A good example of this issue is RVM, which for the life of me I could not get to work.
Just a note: with a normal installation, NixOS will basically never compile anything on your computer, unless you customize a package or use a very recent revision of the package tree that hasn't gone through the build server yet.
Customizing packages is something I do occasionally to debug FOSS. I don’t recall which package I wanted to debug, but I remember that I gave up after my laptop was compiling for many hours. Any changes to one NixOS package automatically invalidate all of its reverse-dependencies, which isn’t the case with e.g. Debian, where I can, when deeming it safe, rebuild a package and install it over the currently installed version.
Yeah, although you have a lot of flexibility regarding how much of your system will be rebuilt.
Like, if you add a little patch to libpng as an overlay over your entire system, then you'll recompile a lot of stuff, but you can also choose to only use that custom libpng with a specific package.
Even in the latter case, it's true that Nix will always recompile the package that depends on libpng, instead of using dynamic linking and only recompiling libpng. This is because Nix has no way of knowing how your libpng patch will affect its reverse dependency, and because link paths are always content-addressed in Nix for reproducibility.
It wasn’t apparent to me from the docs (back then?) how to use the custom libpng only with a specific package. Do you have a pointer to the documentation on that please?
it was pretty entirely new at the time it started, lots of similar things were done since (guixsd is a port of nixos using gnu guile dsl instead of the nix language, reproducible builds are also strongly related to nixos)
If you look at the ISO url, you'll notice something that looks like a short Git SHA hash. That's the exact git checkout of `nixpkgs` used to build the ISO. All system-level artifacts built with nixpkgs wind up with it as part of their name. Very handy.
If you check both, you'll see they're both commits from the upstream `nixpkgs` repository, just a couple weeks apart. So it looks like your torrent is likely an old official ISO that could be upgraded in place the same way the newer ISO would need. (Note, I haven't confirmed that, someone could've uploaded a malicious file with a nice filename, etc. Don't take my word for it, etc.)
(edit: ah, and if I had to guess, the 776 and 1023 as part of the filename is probably the number of commits since the original 18.09 tag was created.)
Nix points the way to what Docker could have been. I don't think it's there yet, but it's getting better. Guix is, to me, even more interesting (and even less "there yet"), in that it doesn't use a NIH language, and it's focused on software freedom.
Recommended. My NixOS workstations have all been very stable, after the correct configuration.nix has been achieved. Being able to use nix-shell for development environments is invaluable. Being able to serve my own binary cache is an efficiency gain for my network.
Just tried NixOS in the VM... the application launcher does not update after installing new packages... even after a log off or reboot. Running kbuildsycoca5 fixes it.
See, this is the kind of absolute shit that makes Linux hot garbage. This is a central and catastrophic bug for a new user. I can't give NixOS to anyone with this kind of a bug. They'll be calling me at all hours and I'll want to shoot myself.
Jesus Harold Christ. It has to fucking work. Get the basics down before you put out crap like this.
I want the old nasty Linus back and I want him to take over a whole distro and tell the people who make shit like this that they are goofs and they should feel bad.
Software without a benevolent dictator is trash. Think of the user experience. Can you imagine the kind of misery that this sort of bug creates for the user and the sysadmin alike?
NixOS is a great idea. First new distro I've tried in years. But Linux devs are still stupid crazy little shits who can't get a fucking package to show up in the menu after an install.
What the actual fuck?
Back to the hated MacOS. The ghost of Nasty Steve still lurks in its passageways, so it hasn't gone to complete shit yet... even though they're trying.
Graham Christensen mentions some of these tests here: https://grahamc.com/blog/an-epyc-nixos-build-farm
> The NixOS release process is novel in that package updates happen automatically as an entire cohesive set. We of course test that specific software compiles properly, but we are also able to perform fully automatic integration tests. We test that the operating system can boot, desktop environments work, and even server tests like validating that our MySQL server replication is still working. These tests happen automatically on every release and is a unique benefit to the Nix build system being applied to an operating system.
> These tests use QEMU and KVM, and spawn one or more virtual machines running NixOS.
He goes on to describe
(1) the KDE desktop test that boots a real graphical environment and uses scripted OCR to detect activity;
(2) the MySQL replication test that verifies correct replication between three separate MySQL servers on three separate virtual machines; and
(3) a BitTorrent test that starts four virtual machines—namely one tracker with Apache serving a torrent file, one NAT router with UPnP (!), two peers of which one is routed through the NAT—and verifies the full process of peer 1 downloading from the seeder and peer 2 downloading from peer 1.
Those tests can all run automatically on a build server for every commit to the package tree... it's quite something!
The whole BitTorrent test is a 116-line Nix expression that's eminently pleasant to read: https://github.com/NixOS/nixpkgs/blob/49a6964a4250d98644da61...