> Our most important finding is that the reproducibility rate in nixpkgs has increased steadily from 69% in 2017 to about 91% in April 2023. The high reproducibility rate in our most recent revision is quite impressive, given both the size of the package set and the absence of systematic monitoring in nixpkgs.
That's one way to read the statistic. Another way you could read the graph is that they still have about the same number (~5k) of non-reproducible builds, which has been pretty constant over the time period. Adding a bunch of easily reproducible additional builds maybe doesn't make me believe it's solving the original issues.
> We knew that it was possible to achieve very good reproducibility rate in smaller package sets like Debian, but this shows that achieving very high bitwise reproducibility is possible at scale, something that was believed impossible by practitioners.
Maybe I miss some nuance here, but why is Debian written off as being so much smaller scale? The top end of the graph here suggests a bit over 70k packages, Debian apparently also currently has 74k packages available (https://www.debian.org/doc/manuals/debian-reference/ch02.en....); I guess there's maybe a bit of time lag here but I'm not sure that is enough to claim Debian is somehow not "at scale".
It's a bit like asking what percentage of Nix-packaged programs have Hungarian translation -- if Nix packages some more stuff the rate might decrease, but it's not Nix's task to add that to the programs that lack it.
Nix does everything in its power to provide a sandboxed environment in which builds can happen. Given the way hardware works, there are still sources of non-determinism that are impossible to prevent, most importantly timing.
Most programs depend on it, even compilers, and extra care should be taken by them to change that. The only way to prevent it would be to go full-on CPU and OS emulation, but that would be prohibitively expensive.
A compiler invoked twice on the same source file is not mandated to produce the same binary, but it should produce a binary with the same functionality.
There are infinite number of binaries that do the same thing (e.g. just padding random zeros in certain places wouldn't cause a functional problem).
Nix is very good at doing functionally reproducible builds, that's its whole thing. But there are build steps which are simply not deterministic, and they might produce correct, but not always the same outputs.
OS scheduling is non-deterministic, and there are quite a few things that are sensitive to the order of operations (simplest example: floating point addition). Is you want to guarantee determinism, not just provide it on a best effort basis for things that are willing to cooperate, the only way do that is to put everything into a fully deterministic emulator, which is terribly slow.
is SCHED_FIFO or SCHED_RR or the newer incarnations viable? Is it possible to make QEMU deterministic? a quick glance at the realtime schedulers state source code needs to take that into consideration.
I know it seems like i don't know what i am talking about, often, on here. The reason is i don't live on HN so i type a comment and rarely remember the words i want to use before the edit/delete window closes.
The idea that i can't control when things occur during a compilation seems suspect. Is there a certain "code size" or other bellwether where this non-determinism starts to crop up? I ask because i get the feeling if i start compiling trivial stuff it will all be bit-perfect and if i say so i'll get lambasted for "well obviously trivial stuff can be reproducible," so i am heading that off at the pass, first.
I don't think there are many cases where it's used "correctly" but it does get constructed in some cases - e.g. in Harry Potter some of Ron's brothers call him "ickle Ronniekins", which is slightly nonsense but we recognise it immediately as a maximally diminutive form of his name.
I can't think of an augmentative suffix, only prefixes (super- and things like that).
We (via French, I guess) have a mildly productive suffix "-ette" or "-et", as in cigar-ette, kitchen-ette (floret, bassinet, owlet). It doesn't imply animateness the way "-kin" does; e.g. "hotelette" or "showerette" seem like plausible coinages in a way "hotelkin" and "showerkin" don't (but I sense some cross-pollination from Japanese "-kun" messing with my interpretation of the latter). But "-ette" certainly connotes femininity: "little Ronniekins" says he's a baby, but "little Ronnette" says he's a girl.
I'm sure they do have a way of dealing with it in Bing Maps - I assume they can display borders differently depending on the country of origin (I think that's the common approach), but that has zero technological overlap with the presence or absence of Unicode flag codepoints in a system-level font.
Google didn't "vacuum up" Pebble. Fitbit bought them (after they were in financial trouble), Google bought Fitbit later so ended up owning the Pebble source code after that.
As the top voted comment on the article says, Google didn't have to do this. It's probably driven by only a few people internally there, and if everyone's cynical and nasty about it, they're less likely to try doing the nice thing again next time. That isn't a good outcome.
I agree with the general point - there are a bunch of things I don't like about Docker much. (Podman inherits the same issues but it is just copying the interface).
Definitely agree on the licensing thing. It's quite a trap if you have some copyleft surprise and they could do more - like just require an SPDX identifier on each repository / image.
I've used systemd-nspawn before. I didn't find it notably simpler and did find lots of weird edge cases where things didn't work (most recently something between ~249 and ~253 giving 'permission denied' errors on mounting /proc into new sub-namespaces within it, boy was that not a fun or easy time to try to work out). Maybe that makes their final point a fair one, that VMs avoid a lot of this without so many awkward subtleties.
That's a ridiculous way of thinking about it. Anyone who says they don't want illegal immigrants isn't objecting simply because of the legal status, they think the process should be upheld which prevented those people from immigrating legally.
It's the same reason we don't seek to improve crime rates by legalising theft. Sure, there'd be fewer people labelled as "criminal", but the original problem would remain (and in all probability would become worse).
And why the process of immigrating legally is more than buying a ticket and a security check at the border?
Obviously, it is the reason that matters and the reason is not benign.
People want other people to like people in certain way they and then castrate Alan Turing for illegal love. People want to have slaves then they have illegally free slaves problem.
315,000 repositories + 10,000 per day? They were obviously correctly concerned this wouldn't be able to go on forever, hence the pre-emptive email, and of course they got the response saying it was okay, but I really feel like this is the kind of thing that's too dangerous to leave your company sitting on because sooner or later they were going to be told "no". It feels too much like it's found a point of arbitrage in Github's ToS, and indeed ended up causing problems.
I suppose they did respond pretty fast, but if I were them I'd have liked to have had the S3 option in my back pocket earlier. Maybe I'm just being too risk-averse here...
If nothing else, I was a little surprised they didn't (or at least didn't mention) having a fail over plan in place already. Seems like "Prepare for the worst, hope for the best" would have been the logical game plan.
It was sold as supporting a much more concurrent rendering engine, which they felt was basically too hard to write correctly/safely in C++, but presumably would be considerably faster.
My interest in a rust web engine is absolutely memory safety, because it's a fun game at work to post the "RCE o' the week" seemingly caused from exposing literally millions of lines of C++ to the wild Internet https://chromereleases.googleblog.com/search/label/Stable%20...
> For whatever reason (maybe post 2to3 PTSD), Python community seems not extremely eager to jump on latest versions of Python
I don't think it's the 2 -> 3 thing any more, that was a while ago. Honestly there are just a lot of things that don't work well in 3.x.0 Python releases. For example, 3.12.0 had the per-interpreter GIL thing; I tried that in 3.12.0 and ran into a completely breaking issue almost immediately. They were responsive & helpful and did fix the first issue in 3.12.1, but we still had more issues with parts of the C API which seemed to work in 3.11 and better again in 3.13, but it really felt like that needed another release to solidify. (Also you can't import datetime in that setup in 3.12, which is also a pretty big deal-breaker).
I can only imagine the free-threading thing will need at least the same kind of time to work the kinks out, although it is nice to see them moving in that direction.
One that I ran into at previous job: 3.10 removes the ability to implicit cast floats to ints in a bunch of places. That was very much a breaking change for a bunch of code out there.
Comparing ints and floats sounds like a pretty high nono in basically any language ever. Why more python projects should use MyPy and not freely switch float to int to decimal. They aren’t the same thing.
That's one way to read the statistic. Another way you could read the graph is that they still have about the same number (~5k) of non-reproducible builds, which has been pretty constant over the time period. Adding a bunch of easily reproducible additional builds maybe doesn't make me believe it's solving the original issues.
> We knew that it was possible to achieve very good reproducibility rate in smaller package sets like Debian, but this shows that achieving very high bitwise reproducibility is possible at scale, something that was believed impossible by practitioners.
Maybe I miss some nuance here, but why is Debian written off as being so much smaller scale? The top end of the graph here suggests a bit over 70k packages, Debian apparently also currently has 74k packages available (https://www.debian.org/doc/manuals/debian-reference/ch02.en....); I guess there's maybe a bit of time lag here but I'm not sure that is enough to claim Debian is somehow not "at scale".
reply