This seems far more complicated than necessary. This design assumes that a trusted execution environment is available, and it also assumes that a trusted hypervisor is available. If you have both of those, then why can’t you just run your software in an enclave directly protected by the trusted hypervisor?
I read it more carefully. Egads! They invented their own hypervisor that essentially poorly imitates Xen’s PV mode. I wouldn’t trust this thing at all.
Here are some likely holes:
They prevent the kernel from mapping “sensitive” code into its own address space, but they don’t seem to prevent the kernel from mapping it into a user address space with write permission, which is just as bad. (Also, the kernel already maps most memory writably in the direct map, and they don’t mention what they do about this, so I would guess that they have a bug.)
They don’t mention protecting sensitive code from DMA.
The kernel can corrupt use code execution in many ways, such as corrupting non-“sensitive” registers. They don’t seem to have a rigorous model to defend against this.
They protect the IDT, but I don’t see anything about protecting the SYSCALL MSRs. The kernel could redirect SYSCALL to skip the magic hook. This might not matter if there are no syscalls in sensitive regions.