From what i have understood, Ginseng sets sensitive memory regions to only be accessed by the TEE then unmap these regions from the kernel memory space. If your kernel module try to read/write/map these regions, a memory violation exception is raised and then handled by the TEE.
That literally means that every time you take an interrupt, you also take an extra fault into the hypervisor (since your CPU cannot read the vector, because it has been protected). In that case, forget any ideas of speed. The whole point of hardware assisted virtualization was to prevent that situation. These guys suggest going decades back in terms of performance. No thanks.