This answer is malpractice on at least two levels:
1. It entertains a performance argument without interrogating the task and environment the appliance is engaged in. You can always in theory make a typical program faster by rewriting it in carefully scheduled assembly. We don't often do that because the performance win would be irrelevant to the problems we're trying to solve.
2. It pretends that the only reliability benefit from moving to u from k is in virtual memory. But, no: the major benefit is from moving from C to something more expressive and safer. The notion of "defect density" captures this argument: with it held constant, a solution that can be expressed in fewer lines of code will have fewer faults. But, of course, defect density between C and a high-level language won't be constant, and it won't favor C, where you can't even count safely with the basic types provided by the language.
Once the driver doesn't exist in the kernel (vfio/iommu), and you have processes and threads and efence/mprotect, this all doesn't make sense for practical purposes. The author's reasoning just doesn't hold up in modern networking and storage appliances when it comes to state of the art performance in Linux userspace.
There is no reason HLLs can't be used in kernel space, is there? Excepting shell-like languages (which require fork, which you typically can't do in a kernel), most embedded kernels provide everything a HLL might expect in kernelspace.
Malpractice? Strong words, considering you're mischaracterizing the article.
"Virtual memory—which gives us the user/kernel space split and the process model of programming whereby programs are protected from each other—is just the most pervasive form of isolation."
That's not claiming virtual memory is the only reliability benefit, and you can't expect a short article to be a treatise on all the differences between kernel space and userspace.
Furthermore, the article isn't claiming that kernel programming is _better_ than userspace programming, it's just pointing out that there's a tension between performance and safety - which you acknowledge!
A TCP/IP stack isn't fundamentally faster for running in user- or kernelspace, but avoiding user/kernel transitions helps. So maximum-performance implementations that don't need to care about playing nice with others might be best off choosing one or the other and staying there; running in userland is then more convenient (but e.g. the TUX in-kernel webserver exists/existed.)
Mind you, most high-performance software doesn't need to go this far, and stuff like PF_RING / Linux' packet_mmap or "batched" systems calls like recvmmsg() (or even read()) further reduces the number userland/kernel transitions.
The vast majority of the time though you need to let multiple processes use the same TCP/IP stack. As soon as you have to do that, moving the TCP/IP stack to userspace means you have to do context switches instead of syscalls.
That is not a good thing if you care about performance.
The kernel implementation does context switches and syscalls. It depends on your usersoace architecture if you have to do context switches, you get to decide.
No. If I use netcat on my Linux machine to connect to a remote machine and send and receive packets, no context switches are involved.
If I want to move the TCP/IP stack to userspace on my Linux machine, how do all the programs I use (firefox, ssh, IRC client) send and receive packets without context switches?
If your answer is to change the userspace architecture and shove everything into one process, you've made isolation _worse_. If you don't, how do you avoid the context switches to talk to the network stack?
The key point is that the network stack has to own shared state, and sending and receiving packets _must_ touch that shared state. If it's accessible to every process (shared memory), you've tossed out isolation and security. If it's not, you have to context switch.
> If your answer is to change the userspace architecture and shove everything into one process, you've made isolation _worse_.
Processes are one way of doing isolation but they're a very crude one (give each thing that needs to be isolated its own address space, swap the whole stack every time you switch tasks). A userspace scheduler (which, sure, would be running within a single process at the UNIX level, but that's the wrong level to be thinking about it) that knows about the details of what it's running can achieve the same level of isolation in a more lightweight way.
netcat does switch into and out of the kernel; you're otherwise correct that userland networking pulls towards a one-process appliance (hence my original "... that don't need to care about playing nice with others").
No, if OpenBSD was actually better for those use cases I guess companies would have noticed it already, instead of paying for microkernels with slower performance.
"The tension in any of these systems is between performance and isolation. "
Also caught my attention. In my mind it is a bit analogues to discussion about 'microservices'.
Although micro services, are probably at the peak of the Hype Cycle.
Where isolation is projected as 'productivity enabler', across teams.
Very difficult to challenge this, especially if you work for a company where revenue is declining/not growing because of lack of ideation/innovation/market conditions.
So, when company fails to ideate and grow revenue, the product management/CIO will be turning onto the productivity.
I suspect the next move for the company, where the above is discussed, -- is moving Development to a lower cost area.
Especially if the 'hiring problem HR is noting' is not just with requirements, but also salary ceilings.
THIS : " If your problem must be decomposed into a set of cooperating processes, then programming in user space is the exact same problem as programming in the kernel"
Corollary (in my experience): many systems evolve to have many cooperating processes.
and THIS: "The tension in any of these systems is between performance and isolation."
To solve for the root issue described - hiring(1) - they should be looking for people who have an aptitudes for these those types of issues ..more so than say kernel or user space skills & experience.
(1) "HR complained that they could not find sufficient, qualified engineers to meet the demands of management for more hands to make more features"
Minor unrelated complaint, but when you write "hiring(1)" like that, it immediately makes me think there's an unknown man page I need to read rather than a footnote reference. In my experience, citations and footnotes generally use brackets, e.g. "hiring[1]".
1. It entertains a performance argument without interrogating the task and environment the appliance is engaged in. You can always in theory make a typical program faster by rewriting it in carefully scheduled assembly. We don't often do that because the performance win would be irrelevant to the problems we're trying to solve.
2. It pretends that the only reliability benefit from moving to u from k is in virtual memory. But, no: the major benefit is from moving from C to something more expressive and safer. The notion of "defect density" captures this argument: with it held constant, a solution that can be expressed in fewer lines of code will have fewer faults. But, of course, defect density between C and a high-level language won't be constant, and it won't favor C, where you can't even count safely with the basic types provided by the language.