This version has got to be the worst kernel released in a while in terms of regression, from AMDGPU null pointer dereference crash[0] to f2fs data corruption bug[1] and now this. Fixes for these are on their way as far as I can tell but since the stable team are probably on Christmas vacation it might take a while.
These are some attitude goals for me. It's so easy to take things personally. Being able to take things constructively even when they might be personal is a great skill.
5.10 is a Long Term Support release that is going to get used by many distros for a long time. Maintainers might have tried (unsurprisingly) to get some interesting features merged.
On the bright side updating to 5.10 fixed a regression of a 5.4 to 5.8 kernel upgrade to me. The fix might have been in 5.9 but I only got the idea of upgrading after the 5.10 release.
Anyways, Linux needs some more CI so that such bugs can be found during the RC phase.
Where in the current CI that we have today is lacking that needs to be improved? We always want more testing and testers, what is preventing everyone from helping with this?
I'm a software engineer who's not involved in Linux Kernel Dev... but I've got a stack of old laptops that I'd be happy to set up to run automated CI if that'd be helpful.
Is there a webpage or doc somewhere I can look at?
(I'm not trying to snark - the fact that you're you and you're here asking for help is making me want to dip my toe in).
Simplest thing to do, just run Linus's latest releases (the -rc releases), or from his git tree, on your machine and report any problem.
Second-simplest thing to do is to run the linux-next branch/tree on your machines and report any build warnings and runtime issues you find. That's what will be the "next" kernel releases and is where all of the developer/maintainer trees are merged together before they are sent to Linus.
Both of those should be very easy to do, and any problems found there should be easy to fix and resolve before they get to a "real" release.
I haven't been following kernel dev for years; what does the CI setup look like? Did the Phoronix Test Suite ever find its way into widespread use?
Back when I was building kernels for embedded hardware (Sheevaplug) in the 2.6.33 timeframe, I found a USB audio regression between 2.6.33.7 and later versions. If there were a semi-turnkey way to set up a testbench that could automatically reboot hardware in every new kernel, run through some basic tests, and report any deviation, I probably would have been more likely to do so. At the time I was working solo trying to release a polished consumer product (sadly though the product was released the business didn't work out) and didn't have time to dig into and report bugs.
We have so many different CI systems running on the kernel on a hourly basis.
We have the 0-day bot from Intel that runs so many things on all developer trees. We have kernelci running on many many different hardware platforms, and we have Linaro test systems also running on many different branches and hardware platforms.
If you want to tie your own hardware into the system, kernelci is the best place to start, I recommend looking into that.
> How is the kernel tested ? There weren’t any tests covering any of this ?
Despite appearances, "the kernel" is not a single monolithic thing. There is a about a 100 kloc core (but I haven't looked up that number in years). The rest, hardware drivers, network protocols, file systems, crypto, raid ... bolt on as modules.
Those modules are maintained separate teams. They are as related to the kernel as the phone dialler app is related to Android. The quality of each module is the responsibility of that team, not "the kernel" team. And that applies to testing the module as well.
In a sense, "the kernel" team is more like debian or redhat than developers. What they have done is develop a framework that lets them take bits created and maintained by a cast of thousands, and bolt it together into what appears to be a single coherent thing from the outside. So the answer to "how is the kernel tested" is "it's complex, and not centrally planned".
The other answer is what you are seeing is in fact part of the testing process. Most people use kernels packaged by their distribution. kernel.org releases are more like Microsoft's pre-releases of Windows. Most Debian users for example won't see it until it gets to Debian testing. To get there it must pass through Debian experimental (which is where 5.10 sits now) then sit in Debian unstable without bug reports for a while. Those release names should give you a hint about the anticipated stability of the kernel version. I personally won't use it until it takes another step, which is from Debian testing to Debian backports (which is when it because available to Debian stable users who are willing to risk compatibility issues).
This means that for for most users, 5.10 it's done yet as it has barely begun it's testing regime.
[0] https://bbs.archlinux.org/viewtopic.php?pid=1943906#p1943906
[1] https://bugzilla.kernel.org/show_bug.cgi?id=210765