On: Execute Docker Containers as QEMU MicroVMs

[ashishbijlani]

> Can we somehow combine the advantages of the docker ecosystem with VMs?

Shameless plug: this is exactly what our goal is with https://kwarantine.xyz We are creating a new hypervisor (from scratch) that can run strongly isolated Docker/LXC containers.

[amscanne → ashishbijlani]

The "fork" sounds like you blue pill the OS for each container? I'm assuming the concept is like Cappsule [1] or Bromium [2]?

[1] https://cappsule.github.io/ [2] https://en.wikipedia.org/wiki/Bromium#/media/File:Bromium-en...

[ashishbijlani → amscanne]

fork here is COW on the host kernel (i.e., copying EPT entries). We will post detailed technical documentation soon.

[mikepurvis → ashishbijlani]

Is this what gvisor is? https://github.com/google/gvisor

[ashishbijlani → mikepurvis]

No, gVisor is from Google. They emulate system calls in user-space and use VMs, which increases runtime performance overhead. We use hardware virtualization to directly run containers -- no I/O emulation, no expensive VM exits, scale as needed. Initial comparison with FC/GVisor/Xen here: https://github.com/ashishbijlani/kwarantine

[tptacek → ashishbijlani]

It sounds like you just said "yes, but what we're building is faster". The userland Linux emulation is a security benefit, not a liability.

[monocasa → ashishbijlani]

I'm not sure gvisor requires vm exits. Their first backend used ptrace very similarly to how user mode Linux worked.

Minor quip though since ptrace might even be slower than vm exits; your core point stands.

[rkeene2 → monocasa]

User Mode Linux is still around and works well. I use it when I need a "fakeroot" without any special privileges on the host.

https://rkeene.org/viewer/tmp/fakeroot.sh.htm

[mikepurvis → rkeene2]

Thanks for the poke on this. I had looked briefly and become frustrated that many of the instructions I found assumed you were a kernel dev and started with compiling everything from source— the Debian-supplied UML binaries seem to work well for my needs though, and do indeed allow doing basic stuff like mounting a disk image so you can run install-grub on it.

[rkeene2 → mikepurvis]

Hmm, building UML from source is really easy. Here [0] is my process for doing it, as a Makefile. The actual compile step is just one line (line 32):

    $(MAKE) -C linux-$(KERNEL_VERSION) ARCH=um linux

The rest of it sets up the configuration how I want and compiles other dependencies (like slirp) or is for maintenance, like cleaning up, or downloading.

This is a rather old version -- newer versions check the checksum and use my HashCache system.

[0] https://rkeene.org/viewer/tmp/uml.Makefile.htm