> build a extremely minimal kernel (i.e. ditch Linux entirely) and link your application against necessary bits of code
It would be nice, but this is really hard to do when modern software has so many layers of crud. Good luck getting say, a PyTorch app, to work doing this without some serious time investment.
But you don't need to write against all the layers of crud. You only have to write against the bottom layer, the kernel API. This sort of software would have no need to specifically support "libxml" or "TLS", because that is multiple layers above what this sort of software does.
The flip side is that if you want something like low-level access to your specific graphics card you may need to implement a lot of additional support. But of course nothing says you have to use this everywhere at the exclusion of everything else. There's plenty of systems in the world that from the kernel point of view are basically "I need TCP" and a whole bunch of compute and nothing else terribly special.
[Author of the paper here] You hit the nail on the head, this is precisely what we do (kernel API compatibility) with the LF Unikraft project (the evolution of the 2017 paper) at www.unikraft.org, and kraft.cloud, a cloud platform that leverages Unikraft.
Most of that effort should be sharable. if you know you will only have one python process you can get rid of a lot of cruft. If you know you will be running in a VM then you only need the driver for the network interface the VM provides not every network interface every designed (often including ones that your hardware doesn't even physically support). So while there is serious time investment it isn't nearly as much as it would be to write a competitor to linux.
I'm not sure if I missed a bit here, but I have some colleagues doing research on unikernels for HPC and the point is that this unikernel is running directly on the hardware or hypervisor and not inside another VM. The unikernel is effectively a minimal VM and the network stack is one of the things they struggle the most with due to sheer effort.
[One of the authors of the paper] I wouldn't recommend writing a network stack from scratch, that is a lot of effort. Instead, with the Unikraft LF project (www.unikraft.org) we took the lwip network stack and turned it into a Unikraft lib/module. At KraftCloud we also have a port of the FreeBSD stack.
It would be nice, but this is really hard to do when modern software has so many layers of crud. Good luck getting say, a PyTorch app, to work doing this without some serious time investment.