I'm not going to defend the article and their test procedure, but I'll give a bit of insight anyway:
CentOS, as you may or may not know, is RHEL[1] recompiled from the source RPMs.
For RHEL [and I work at Red Hat], we:
* Sell support for 7-10 years from release date. For RHEL 6, that's support/updates/security fixes until Dec 2017 or Dec 2020 depending on how much you pay.
* Spend about a year before release intensively QA-ing all the packages. We have a huge team of people just doing this.
* QA each 6-monthly point release to the same standards.
* Have lots of developers working on optimizing RHEL specifically (as well as working upstream).
* Have a separate team that works on performance and tuning.
* Work with hardware suppliers to ensure best performance on specific hardware, and make sure we have support for hardware before it is released. [Typing this on a pre-release Intel SVP ...]
* Get world-beating benchmark results on things such as SPECvirt by working with manufacturers and other large companies, and going back to the original developers to work out bottlenecks on very high end hardware. [Last week my colleague was logged into a PC with 4 TB of RAM].
The above costs a bunch of money. I think we spend in the region of $100 million on R&D each year.
Note of course we only test the binaries we release. We don't test CentOS's binaries ...
* Still make HUGE kernel updates every few days on RHEL 6.x
which, while you're reading made me wonder. we bump into kernel issues from time to time due to update, and I always wondered why the policy was what it is.
basically it seems like backports while retaining the same kernel major/minor version numbers... because a lot of it aren't just bug fixes
Note that as I have been maintaining a Debian spinoff for years I highly value RH's packages quality, and have generally been picking from RH packages for patches and stuff "that just works" (incorporating that in the debs)
We backport new features, bug fixes, hardware "enablement" etc into the same kernel. We keep the kernel structures and kernel symbols compatible (actually there's a big whitelist of approved symbols that your kernel module is allowed to call and we guarantee those will stay the same).
The basic reason for this is so software vendors can write kernel modules for RHEL 6.0 which will be binary compatible with all future RHEL 6.x releases. (The kernel modules will need to be recompiled when RHEL 7 comes out).
This involves lots of careful review of backports to make sure (eg) that all kernel structures like task_struct stay compatible. I'm not involved with this directly, but all I can say is it's a pretty amazing achievement over a ~4-5 year development cycle and ~7-10 year update cycle.
I assume that we keep the kernel major.minor version the same to reflect this. 'Course the release number changes each time ...
Benchmarks? Facts? Test procedure? At least some basic explanation?