On: Spack – scientific software package manager for su...

[soraminazuki]

I'm not sure if I understand the first point. Doesn't Nix currently achieve the same thing feature-wise?

For example,

    spack install packageX %libA@2.0

would translate to

    nix-build -E 'with import <nixpkgs> {}; packageX.override { libA = libAv2_0; }'

For this to work, both Spack and Nix would have to package libA v2.0. If it isn't, the user would need to create their own. Assuming a different version of libA is already packaged in Nix, here's how it would look like without the "nix-build -E" part:

    let
      pkgs = import <nixpkgs> {};
      libAv2_0 = pkgs.libA.overrideAttrs (old: {
        src = pkgs.fetchzip {
          url = "...";
          sha256 = "...";
        };
      });
    in
    pkgs.packageX.override { libA = libAv2_0; }

I assume something similar would be required for Spack too.

So aside from Spack having a nicer shorthand syntax for customization, I don't get what Spack can do that Nix can't in terms of features. Or to be more specific, how a dependency resolver can eliminate human work.

[tgamblin → soraminazuki]

Spack packages are parameterized, so there is one `package.py` file per package, not several, and you can have many versions and options declared. See, e.g., `zstd`: https://github.com/spack/spack/blob/develop/var/spack/repos/...

This was a conscious decision we made to allow many versions to be built without requiring split implementations and without checking out different commits -- that saves some work already. It also forces the repo maintainers to consider the other use cases, which tends (IMO) to make the recipes more portable.

But the real question is not just work saving but correctness. You cannot always just swap in libA@2.0 as in your example. For:

    packageX ^libA@2.0 

Suppose that both packageX and libA depend on MPI (a versioned standard for which there are several implementations) and, further, libA requires MPI@3. Spack will ensure that:

    1. packageX and libA use the same MPI implementation (e.g., openmpi, mpich, mvapich)
    2. the MPI implementation chosen satisfies the provider requirement
    3. any version/option constraints that packageX has on MPI are satisfied along with those of libA.

You could also imagine that the two packages might have conflicts with certain implementations of MPI, e.g. say packageX conflicts with mpich and libA conflicts with mvapich. You'd have to choose mpich.

This isn't just an overlay; it's a constraint solve. The choice of libA@2.0 can have effects on other nodes in the graph, and different choices may need to be made elsewhere; maybe even things like disabling options or choosing different dependencies.