I would rather something not be packaged at all than packaged badly; this whole experience reads like a lesson in what not to do.
What the author left out in his blog post is that I specifically explained why this was going to be an issue and what was going to happen when he and I first talked about Debian packaging; and why for a package that needs to work correctly for the system to boot (fsck, mount helper, encryption unlock), playing these kinds of games with swapping dependencies out was not a good idea.
But with Debian and Fedora people alike it's been like talking to a brick wall. "No, your concerns don't matter, this is our policy and we're going to do what we want with your code". They've made it very clear that I have no voice or say in how they package it.
So then he breaks the build by switching to the packaged version of bindgen, an old unsupported version.
And he makes no attempt to debug, doesn't even send me the build error until months later.
As a result, Debian users were stuck on a broken version of -tools that wasn't passing mount options correctly, which meant that when a drive died and they needed to mount in degraded mode, they weren't able to. This happened to multiple people - that bothered to report it. This was quickly fixed upstream, but as far as I know it's still broken in Debian; and Jonathan Carter wasn't fielding any of those bug reports, I was.
I'm tired, frustrated and demoralized by this drama and tired of constantly having to respond to it. This has taken up an enormous amount of my time.
I really wish this guy hadn't packaged -tools for Debian in the first place. I never asked for it. I have enough on my plate to do without a bunch of distro drama to deal with as well.
> I would rather something not be packaged at all than packaged badly; this whole experience reads like a lesson in what not to do.
You might consider adding a big warning on your official documentation about unsupported distribution packages.
Add links to relevant issue tracker/bug reports or mailing list discussions saying until So and so issue are resolved, the official statement is that distribution package is unsupported, not recommended and deemed dangerous to use.
This is as much leverage as you can have with the distribution community. Then wait for them to upstream patches and attempt to fix the issues. Accept fixes as you consider appropriate or not.
It's also important to at least respect the distribution's opinions in regards to having only one version of a library's major release . Just respect and understanding, you don't have to agree.
Also, all the problematic libs cited by the packager are 0.x.x numbered which sounds like a very young and immature ecosystem of dependencies. Of course, this is bound to cause pain to packagers. I think this speaks volumes about the high level of interest for bcachefs, that they actually tried to make it work instead of not bothering.
0.2 and 0.4 are different "major releases" of rust crates as you say. The major release is determined by the first non-0 component in the version number. The issue is that debian appears to only allow one version even if there are multiple major versions.
If debian is fine with packaging versions 2.0 and 4.0 but not 0.2 and 0.4, then debian does not understand rust version numbers.
> So then he breaks the build by switching to the packaged version of bindgen, an old unsupported version.
Could bcachefs-tools solve this problem upstream of distros by releasing a source tarball with the bindings already generated, so that you can use whatever version of bindgen you want, and bindgen is not needed at all when building from this release tarball?
This would be similar in nature to traditional autotools approach where some amount of pre-compilation is done to produce a release tarball from what actually lives in the source repository.
It's also what I do for some of my ffi crates. I have a script in the root of the repo which regenerates the bindings, but when published to crates.io, the bindings are already generated and bindgen is not part of the build downstream of my crate.
I already provide tarballs with vendored dependencies - I did this for Fedora, so that package builds wouldn't have to download anything.
I'm not going to special case the bindgen bindings because this really isn't a bindgen specific issue - this could just as easily been any other dependency.
Speaking from practical experience, bindgen is a particularly heavyweight dependency. It requires libclang to be available and also adds a considerable amount of time to the compile. If it makes compiling the software easier - and end users compiling the software themselves would benefit here as well as distro maintainers - then it would seem like a pragmatic move to me.
Any time a dependency is changed, the package version should be bumped and retested. This is a massive improvement in reproducibility, bisectability and our ability to QA.
Before this model, bisecting breakage that was a result of a change in library 'a' breaking something in consumer 'b' was effectively impossible; now it is.
What Debian and Fedora have been doing is a big step backwards, this whole 'unbundling of dependencies' (that were going to be statically linked anyways) needs to die.
> What Debian and Fedora have been doing is a big step backwards, this whole 'unbundling of dependencies' (that were going to be statically linked anyways) needs to die.
They, and most others, have been doing that since their respectively beginnings, and for good reasons: everybody expects distros to fix security issues, which is greatly aided by ensuring an-as-small-as-necesary dep tree and thus a single version of any particular dep.
Changing that isn't impossible, but I am not aware of any distro that's managing this and be widely considered 'stable'.
How can distros change while keeping up their end which is stability and robustness in the face of bugs and security issues?
> They, and most others, have been doing that since their respectively beginnings
As I explained on another comment of mine (https://news.ycombinator.com/item?id=41409199), it's not since their beginning, but since a particular zlib security incident which showed them the risks of vendored libraries. Any discussion of vendoring policy which does not consider that incident is necessarily incomplete.
That's very nearly a solved problem at this point; I get notified by bots (github, mainly) if a dependency has as a security vulnerability, and then it's very nearly a one click action to do a cargo update and commit the new lockfile.
The distro people could've been working on the tooling for automating this at the distro level (and some people in debian are doing work that would enable this); we don't need to go this insane "unbundle everything" route.
Before broadband, and before forges like github with free CI and bots, which is all fairly recent, distros packaged everything was a godsend, not insane.
> everybody expects distros to fix security issues
IMO this is a huge mistake in the Linux distro world. It doesn't scale. It's similar to the weird idea that Debian should contain every program in the world and if the authors want their programs to be installable on Debian they must make a Debian package for it. And Fedora and Arch and Gentoo and...
Debian does not prevent you from installing stuff from elsewhere, it's just that their packages come with certain availability and stability guarantees. You may not care for them, but clearly bonkers it is clearly not.
The only "junk" I'm seeing here is your comment. If you don't find bcachefs interesting, please move along and keep your bitter thoughts to yourself. Last thing we need is more assholes harassing Kent into throwing in the towel on the best GPL-compatible alternative to ZFS we're likely to have in decades.
What the author left out in his blog post is that I specifically explained why this was going to be an issue and what was going to happen when he and I first talked about Debian packaging; and why for a package that needs to work correctly for the system to boot (fsck, mount helper, encryption unlock), playing these kinds of games with swapping dependencies out was not a good idea.
But with Debian and Fedora people alike it's been like talking to a brick wall. "No, your concerns don't matter, this is our policy and we're going to do what we want with your code". They've made it very clear that I have no voice or say in how they package it.
So then he breaks the build by switching to the packaged version of bindgen, an old unsupported version.
And he makes no attempt to debug, doesn't even send me the build error until months later.
As a result, Debian users were stuck on a broken version of -tools that wasn't passing mount options correctly, which meant that when a drive died and they needed to mount in degraded mode, they weren't able to. This happened to multiple people - that bothered to report it. This was quickly fixed upstream, but as far as I know it's still broken in Debian; and Jonathan Carter wasn't fielding any of those bug reports, I was.
I'm tired, frustrated and demoralized by this drama and tired of constantly having to respond to it. This has taken up an enormous amount of my time.
I really wish this guy hadn't packaged -tools for Debian in the first place. I never asked for it. I have enough on my plate to do without a bunch of distro drama to deal with as well.