VM is also useful to handle fragmentation and to map parts of a large "array" to different NUMA nodes (so that it can be accessed by several threads without overloading a single memory channel). In a multi-process system, if you get rid of VM, you'll be pretty much obliged to have kernel-level relocating GC or to make array indexing involve software table lookup. I keep my TLB, thank you.
