The performance difference between H1/H2 and H3 in this test doesn't really surprise me. The obvious part is the highly optimised TCP stack. But I fear that the benchmark setup itself might be a bit flawed.
The biggest factor is the caddy version used for the benchmark. The quic-go library in caddy v2.6.2 lacks GSO support, which is crucial to avoid high syscall overhead.
The quic-go version in caddy v2.6.2 also doesn't adjust UDP buffer sizes.
The other thing that's not clear from the blog post is the network path used. Running the benchmark over loopback only would give TCP-based protocols an advantage if the QUIC library doesn't support MTU discovery.
I don't think taking shots at the Caddy version being not the latest is a fair criticism to be honest. Version 2.6.2 was released roughly three months ago, so it's not like we're talking about anything severely outdated, most servers you run into in the wild will be running something older than that.
I think you mixed up what year we're now :). Caddy 2.6.2 October 13, 2022 so it's been not 3 but 15 months since release.
Even more relevantly, HTTP/3 was first supported out of the box in 2.6.0 - released Sep 20, 2022. Even if 2.6.2 had been just 3 months old that it's from the first 22 days of having HTTP/3 support out of the box instead of the versions from the following 3 months would definitely be relevant criticism to note.
This is why I'm not a fan of debian. (I assume OP got that version from debian because I can't think of any other reason they wouldn't have used latest.) They packaged Caddy, but they never update at the rate we would reasonably expect. So users who don't pay attention to the version number have a significantly worse product than is currently available.
Stable/tested but not latest version. Or unstable/untested but latest version. Chose one.
The distribution you chose, also makes you make that choice. If you're using Debian Stable, it's because your prefer stable in favor of latest. If you use Debian Testing/Unstable, you favor latest versions before stable ones.
Can't really blame Debian as they even have two different versions, for the folks who want to make the explicit decision.
I don't call an old version with known bugs to be "stable/tested". No actual fixes from upstream are being applied to the debian version. There are known CVEs that are unpatched in that version, and it performs worse. There's really no reason at all to use such an old version. The only patches debian applied are the removal of features they decided they don't like and don't want packaged. That's it.
By that definition, almost no software in Debian could be called "stable", as most software has at least one known bug.
When people talk about "stableness" and distributions, we're usually referring to the stableness of interfaces offered by the distribution together with the package.
> There's really no reason at all to use such an old version
Sometimes though, there is. And for those people, they can use the distribution they wish. If you want to get the latest versions via a package repository, use a package repository that fits with what you want.
But you cannot dictate what others need or want. That's why there is a choice in the first place.
Stableness of interfaces is supposed to imply the software version is still maintained though. E.g. how stable kernel versions get backports of fixes from newer versions without introducing major changes from the newer versions. It's not meant to mean you e.g. get an old version of the kernel which accumulates known bugs and security issues. If you want the latter you can get that on any distro, just disable updates.
But you're right people are free to choose. Every version is still available on the Caddy GitHub releases page for example. What's being talked about here is the default behavior not aligning with the promise of being a maintained release, instead being full of security holes and known major bugs. It's unrelated to whether Debian is a stable or rolling distro rather about the lack of patches they carry for their version.
> Stableness of interfaces is supposed to imply the software version is still maintained though. E.g. how stable kernel versions get backports of fixes from newer versions without introducing major changes from the newer versions. It's not meant to mean you e.g. get an old version of the kernel which accumulates known bugs and security issues. If you want the latter you can get that on any distro, just disable updates.
I'm sure the volunteers working on this stuff is doing the best they can, but stuff like this isn't usually "sexy" enough to attract a ton of attention and care, compared to other "fun" FOSS work.
Do you have an example of a CVE affecting Caddy that's not patched in Debian? In my experience they've been pretty responsive to security reports, including in the "long tail" of obscure / buggy packages.
The biggest factor is the caddy version used for the benchmark. The quic-go library in caddy v2.6.2 lacks GSO support, which is crucial to avoid high syscall overhead.
The quic-go version in caddy v2.6.2 also doesn't adjust UDP buffer sizes.
The other thing that's not clear from the blog post is the network path used. Running the benchmark over loopback only would give TCP-based protocols an advantage if the QUIC library doesn't support MTU discovery.