One thing that complicates the construction of a new compatible virtual machine is that the beam files are actually pre-processed at load time into another internal format that is then executed by the vm. Nevertheless see for example http://erlangonxen.org/ they have build a new vm running on top of xen.
