Unsurprisingly, they benchmarked against the latest that was available at the time. It's not some nefarious trick to make the benchmarks look good. We can see that the heyday of OSv development work was about 7 years ago:
Hopefully, OSv is a stable product that continues to work well, so heavy investment in development hasn't been needed on an ongoing basis, primarily just maintenance work. Or, it could be abandonware... but the README gives the impression that it is still being maintained.
I would still be hesitant to deploy such a deeply invasive solution, but the idea is interesting.
> We can see that the heyday of OSv development work was about 7 years ago
Yeah, the startup who developed this ending up pivoting to a different product instead, ScyllaDB.
I think they found that while OSv is cool technology, there wasn't enough market demand to make a successful business out of it.
BEA used to have a product called JRockit Virtual Edition (JRVE). It was a thin operating system designed to only run one process, the JVM. So conceptually quite similar to this, but closed source. Then Oracle bought BEA, and then after a while (not straight away after the acquisition), Oracle killed it. Rumour has it that the Linux and Solaris teams were opposed to the idea of Oracle providing a third operating system with far less features than theirs did, which is part of why it was killed. I don't think it was ever hugely popular with customers either.
(Disclaimer: Used to work for Oracle, but not directly on the products mentioned.)
Here's a post from one of the OSv developers on performance compared to Linux in 2020 [1]. It paints quite a different picture compared to those benchmarks from 2013.
>While ideally a unikernel like OSv could provide better performance than traditional kernels because of things like lower system call and context switch overhead, less locking, and other things, there are many things working against this ideal, and resulting in disappointing performance comparisons:
1. Most modern high-performance software has evolved on Linux, and evolved with its limitations in mind.
So for example, if Linux's context switches are slow, application developers start writing software which lowers the number of context switches - even to the point of just one thread per core. If Linux's system calls are slow, application developers start to batch many operations in one system call, starting with epoll() some 20 years ago, and culminating with io_uring recently introduced to Linux. With these applications, it is pointless to speed up system calls or context switches, because these take a tiny percentage of the runtime.
2. They say a chain is as weak as its weakest link.
This is even more true in many-core performance (due to Amdahl's law). Complex software uses many many OS features. If OSv speeds up context switches and system calls and networking (say) by 10%, but then some other thing the software does is 2 times slower than in Linux, it is very possible that OSv's overall performance will be lower than Linux. Unfortunately, this is exactly what we saw a few years ago when ScyllaDB (then "Cloudius Systems") was actively benchmarking and developing OSv: Many benchmarks we tried were initially slower in OSv than in Linux. When we profiled what happened, we discovered that although many things in OSv were better than Linux, one (or a few) specific things in OSv which were significantly efficient in OSv than in Linux. This could be some silly filesystem feature we never thought was very important but was very frequently used in this application, it could be that OSv's scheduler wasn't as clever as Linux's to handle this specific use case. It could be that some specific algorithm was lock-free in Linux but uses locks in OSv, so becomes increasingly worse on OSv with the more CPUs you have. The main point is that if an application uses 100 different OS features - Linux had hundreds of developers optimizing each of these 100 features for years. The handful OSv developers focused on specific features and made clever improvements to them - but the rest are probably less optimized than Linux.
3. Many-core development is hot
When the OSv project started 7 years ago, it was already becoming clear that many-core machines were the future, but it wasn't as obvious as it is today. So OSv could get some early wins by developing some clever lock-reducing improvements to its networking stack and other places. But the Linux developers are not idiots, and spent the last 7 years improving Linux's scalability on many-core systems. And they went further than OSv ever got - they support NUMA configurations, multi-queue network cards, and a plethora of new ideas for improving scalability on many core systems. On modern many-core, multi-socket, multi-queue-network-card systems, there is a high chance that Linux will be faster than OSv.
4. Posix API is slow
This is related to the first issue (of software having evolved on Linux), but this time for more "traditional" software and not really state-of-the-art applications using the latest fads like io_uring. This traditional software is using the Posix API - filesystem, networking, memory handling etc. that was designed decades ago, and make various problematic guarantees. Just as one example, the possibility to poll the same file descriptor from many threads requires a lock every time this file descriptor is used. This slows down both Linux and OSv, but not giving OSv any advantage over Linux, because both need to correctly support the same slow API. Or even the contrary - Linux and its hundreds of developers continue to come up with clever tricks for each of these details (e.g., use RCU instead of locks for file descriptors) while OSv's few (today, very few) developers only had time to optimize a few specific cases.
So it's no longer clear that if raw performance is your goal, OSv is the right direction. OSv can still be valuable for other reasons - smaller self-contained images, smaller code base, faster boot, etc. For raw performance, our company (ScyllaDB) went on a different direction: The Seastar library (http://nadav.harel.org.il/seastar/, https://github.com/scylladb/seastar) allows writing high-performance applications on regular Linux, by avoiding or minimizing all the features that makes Linux slow (like context switches) or cause scalability problems on modern many-core machines. Initially Seastar ran on both Linux and OSv (with identical performance, because it avoided the heavy parts of both), but unfortunately today it is using too many new Linux features which don't work on OSv - so it no longer runs on OSv.
