It's less mature, which shows up in lots of places, such as sometimes having less than ideal defaults (as in buffer sizes shown here), and bugs if you start using more fancy features (which improve over time of course).
This is approximately the case for any alternative IP stack you might pick though, a mature IP stack is a huge undertaking with all the many flavors of enhancements to IP and particularly TCP over the years, the high variance in platform behaviors and configurations and so on.
In general you should only take on a dependency of a lesser-used IP stack if you're willing to retain or train IP experts in house over the long haul, because as is demonstrated here, taking on such a dependency means eventually you'll find a business need for that expertise. If that's way outside of your budget or wheelhouse, it might be worth skipping.
gVisor's netstack is still much slower than the kernel's (and likely always will be). The goal of this userspace netstack is not to compete with the kernel on performance, but offer an alternative that is more portable and secure.
How is it more portable or secure than an API that's been stable for decades, and getting constant security fixes?
I see an explanation in their blog about avoiding TUN devices since they require elevated permissions, but why would you need a TUN device to send data to/from an application? I can't understand what their product does from the marketing material but it doesn't look like it would require constructing raw IP packets instead of TCP/UDP packets and letting the OS wrap them in the other layers.
You can have multiple layers of security boundary on most of the customer-exposed surface area, and avoid more risky surface areas in the kernel.
Portable is a bit of a weird word here because for many of us with gray beards the word means architectures, kernels and systems, but I think in this context it tends to more mean "can run just as easily on my macbook as in a cloud container", but in practice the software isn't that portable, as Go isn't that portable - at least not in the context of vs. a niche C "portable network stack" that would build roughly anywhere that there's a working C toolchain, which is almost everywhere.
Constant security fixes for the kernel are a real pain in deployments unless you follow upstream kernels closely. If your business is in shipping Linux runtimes with a high packing density, you really need to find ways to minimize the exposed Linux surface area, or organize to be able to ship kernel upstream updates at an extremely high frequency (relative to normal infrastructure upgrade rates for kernels / mandatory reboots) (and I would not consider kexec safe in this kind of context, at all).
An alternative approach might be firecracker / microvms and so on, but those have their own tradeoffs too. The core point is that you want more than one layer between the host machines and the user code that wants to interact with Linux features.
> You can have multiple layers of security boundary on most of the customer-exposed surface area, and avoid more risky surface areas in the kernel.
I fail to see what "risky surface areas" in the kernel you're avoiding. You have more packets going through the kernel network stack(since you're wrapping a TCP connecting in a UDP connection that goes through the kernel) than just using the TCP stack in the kernel. Are you saying that the TCP stack in the kernel cannot be trusted, but a userspace kernel you maintain can(that's a bit ridiculous...)
> can run just as easily on my macbook as in a cloud container
Any POSIX C code that listens on non-privileged ports will run on machines with the correct glibc version(and you can statically compile the glibc or not need it like go does). This includes linux and macOS(and if you're using a library that's on multiple OSes you get even more support without having to implement TCP in userspace).
> Constant security fixes for the kernel are a real pain in deployments unless you follow upstream kernels closely.
I don't think you understand. You're still at the mercy of the kernel for security patches to the UDP stack, you're just now also having to maintain a TCP stack in parallel.
> An alternative approach might be firecracker / microvms and so on
Wouldn't an alternative approach just be to use cross-platform libraries and non-privileged ports?
> The core point is that you want more than one layer between the host machines and the user code that wants to interact with Linux features.
You just said the opposite... how can more things requiring security fixes be a bad thing, while you arbitrarily want more layers between you and the most security tested code for networking available to you.
> Are you saying that the TCP stack in the kernel cannot be trusted, but a userspace kernel you maintain can(that's a bit ridiculous...
Yes that’s exactly right. It’s not ridiculous. Netstack is written in a GC’d language which alone eliminates several categories of vulnerabilities that exist in the kernel. But more important than that is that it’s in USERSPACE. So even if you do compromise gVisor netstack the best you have is the capabilities that any other normal process has. Compare that to the kernel vulnerabilities where you potentially have cracked root.
> You're still at the mercy of the kernel for security patches to the UDP stack, you're just now also having to maintain a TCP stack in parallel.
The TCP stack is at least an order of magnitude more complex than UDP and has a correspondingly much higher number of bugs filed against it. Only relying on UDP is a security win.
> I fail to see what "risky surface areas" in the kernel you're avoiding. You have more packets going through the kernel network stack(since you're wrapping a TCP connecting in a UDP connection that goes through the kernel) than just using the TCP stack in the kernel. Are you saying that the TCP stack in the kernel cannot be trusted, but a userspace kernel you maintain can(that's a bit ridiculous...)
There's a constant stream of bugs in kernel network and IO interfaces, many of which require direct local interaction for exploitation, and aren't remotely attackable. Don't assume, spend a few hours and have a read through some.
> Any POSIX C code that listens on non-privileged ports will run on machines with the correct glibc version(and you can statically compile the glibc or not need it like go does). This includes linux and macOS(and if you're using a library that's on multiple OSes you get even more support without having to implement TCP in userspace).
That doesn't get anywhere near the use case here which is: run third party user supplied code unmodified.
> I don't think you understand. You're still at the mercy of the kernel for security patches to the UDP stack, you're just now also having to maintain a TCP stack in parallel.
The surface is not "UDP" and "TCP", this view is a huge distortion. As I suggested above, have a read through some of the relevant bugs over the last two years, and consider their implications in the relevant use case: running unmodified third party user code on a system.
> Wouldn't an alternative approach just be to use cross-platform libraries and non-privileged ports?
No, again, that doesn't meet the use case: run unmodified third party user code on the system.
> You just said the opposite... how can more things requiring security fixes be a bad thing, while you arbitrarily want more layers between you and the most security tested code for networking available to you.
Your characterization of Linux further suggests the exercise above would be a great experience.
