FWIW, Linux itself has plenty of support for TCP offload engines. I don't think Firecracker uses that at the moment, but there's no reason why it has to be that way if that's a true bottleneck in the system.
I think the Firecracker team has stated that PCIe bypass wasn't something they wanted to do, so I don't see how they'd open up to other accelerators' bypass method. But seems like building a vmm from the rust-vmm toolbox is 'not that hard' and there are some PCIe bypass crates already, so... Have fun?
I'm not saying an actual NIC with TCP offload, but instead something like adding TCP offload to the virtio nic if for some reason initialization of the network stack was a latency problem. If the VM is only running at layer 3/4, most init latency problems disappear.
This is exactly what I'm referring to. You have a pool of virtual NICs on the host in user-space created by the VM runtime that get assigned to the guest on provision-time, which just passes through any commands/syscalls (bind/connect/etc.) via a funky driver interface. You'd have to mangle the guest kernel syscalls or libc though, it might be really ugly.
> You'd have to mangle the guest kernel syscalls or libc though, it might be really ugly.
You wouldn't have to. There's patches for hardware TCP offload using the normal socket syscall ABI. The kernel net stack maintainer is pretty ideologically against them so they're not mainlined, but plenty of people have been running them in production for decades and they're quite mature.
> Linux itself has plenty of support for TCP offload engines
Could you link to any specific Linux kernel source that implements support for TCP offload? AFAIK networking subsystem maintainers were always opposed to accommodate TCP offload because it is A Bad Idea.
At its simplest, TCP offload can be just letting the hardware chunk a large packet into smaller ones on the wire. I don't think anything trying to offload more than that has really seen much daylight outside of very proprietary "smart NICs".
