QEMU looks interesting. I have a few questions about its intended use case.
1. My understanding is that the primary use case of QEMU is for server virtualization on Linux hosts leveraging KVM. Is there a use-case for QEMU on non-Linux hosts?
2. I have read that QEMU excels in server virtualization but lags behind VirtualBox in desktop virtualization. Is this still the case? [1]
3. Is it considered "slow" for cross-architecture virtualization? [1]
Whether you think it's "slow" for cross-architecture use depends on your standards. It's always going to be slower than using the native CPU, because we have to emulate everything, and even with a JIT that's not going to be fast. Empirically not much effort is being put into trying to make the emulation/JIT faster (and there are serious technical obstacles as well as mere lack of effort/time), but if your target being emulated is a 200MHz embedded dev board then we can probably manage to go as fast as that in software... In any case the QEMU target is often more convenient for debugging or simply more easily available than the real hardware.
It is sometimes a bit disheartening to see so much development effort go toward the virtualization/KVM/Xen side rather than seeing improvements in TCG. I realize that Redhat (and maybe others?) funds a lot of development in that area, so it's natural that that's where most of the improvements are. I mostly use QEMU as a CPU emulator, though, and it's frustrating to see so little activity on the TCG side, especially when it comes to things like bugfixes (Win7 x64 bluescreens under TCG, mipsel loadvm broken, to name two issues that have bitten me).
Still, kudos to the developers for making some things better (TCG actually does seem noticeably faster in 2.0.0) and congratulations on the 2.0.0 release!
Yes, I think Richard Henderson put in some effort in optimising TCG, which has helped.
My take on the KVM vs TCG split is simply that in the end open source projects are driven by the people who put in the work, and inevitably if there's a set of people whose day job is to work on the project then it's going to tend to improve in the areas those people and companies need. And in general the people working on the emulation side of things seem to be happy enough with the performance levels we currently have: I see plenty of patches to add new ARM boards or fix issues with devices, or to fix bugs in the linux-user emulation code. In comparison, there doesn't really seem to be much interest in TCG performance (and serious improvements in performance, though possible, would be a six month or longer project to achieve).
The particular bugs you note are even further out in the cold since x86 guest TCG in particular is pretty much orphaned and MIPS is not a great deal better (though I have been heartened to see recent contributions from Imagination).
Thanks. That makes sense. I think I should have phrased that question (#3) differently, since there's a huge difference between relative and absolute speed & performance.
What I had read as implied in the SO answer I linked to was that there was some other means of implementing cross-architecture emulation that was faster than QEMU.
In other words, there's a difference between being "slow" in absolute terms vs. being "slow" relative to other virtualization solutions.
From your answer, I gather that QEMU isn't under-performant relative to other emulation options.
QEMU is used by pretty much all open-source emulation or virtualization systems. Bits of it are even used by VirtualBox. QEMU on its own is not a virtualization system, just an emulator. KVM is the hypervisor most closely associated with QEMU, but it also works out of the box with Xen. Using QEMU with either of those hypervisors works just as well as VirtualBox for desktop and workstation virtualization needs, with the only significant downside being that QEMU isn't tightly coupled to a single standard GUI for management. Using QEMU for cross-architecture emulation (not virtualization) is slow, because that's an inherently high-overhead task. I don't think QEMU sets any records for being particularly fast or slow at cross-architecture emulation.
Running two x86 hardware-assisted hypervisors at once is madness. However, running one inside another is possible.
You seem to be laboring under some serious misapprehensions here. QEMU is very much a virtualization solution. It calls itself an emulator because it's older than the name 'virtualization.' KVM is merely the kernel infrastructure that enables ring-0 access for faster hardware access. For many years it was called qemu-kmod; to this day the actual process that you run is called 'kvm-qemu.'
You are overstating the difference between 'virtualization' and 'emulation.' There really isn't one. What used to be called 'emulation' is nowadays called 'full-virtualization' as opposed to 'paravirtualization.' QEMU, KVM, and XenKVM are full-virt, while traditional Xen is paravirt.
KVM is not a hypervisor; libvirt is. Xen blurs the line between virtualization platform and hypervisor because both pieces ship within the same project. Either libvirt or Xen can act as hypervisor for QEMU or KVM.
QEMU's lack of hard-dependency on a specific gui is in fact its strength, not its weakness. This makes it much easier to modularize and deploy at scale, which is what OpenStack does, for instance. Every QEMU VM is capable of listening on a given port for VNC connections, which means you don't even particularly need a GUI for basic use. More featureful guis can be found in the libvirt project (e.g. virt-manager).
QEMU is not particularly fast at cross-architecture stuff because that was not its original focus. However, it was one of the first packages available for linux that even made that possible at all, which is why it is still known for this functionality.
* QEMU calls itself an emulator because emulation was its original intent; that it was possible to relatively easily modify it to use hardware virtualization rather than emulating the CPU was a happy accident some time later
* for the CPU the "virtualization"/"emulation" distinction is huge -- if you can use the CPU's hardware assist to directly run guest code things will be fast; if you're emulating the guest CPU things will be very much slower
* the process being run isn't "kvm-qemu" unless your distro
is providing back-compatibility wrappers (which in turn are only there because the changes to QEMU to make it work with KVM were for some years maintained out of tree)
* KVM is absolutely a hypervisor; libvirt is not, it is a management layer that can configure and control a hypervisor
* you can't use Xen with KVM, because they're both hypervisors
Bingo. In fact I remember using QEMU briefly some years before virtualization or "VMWare" was even a widely-known term, and that was only because of my interest in the emulation scene at the time.
> You seem to be laboring under some serious misapprehensions here. QEMU is very much a virtualization solution. It calls itself an emulator because it's older than the name 'virtualization.'
QEMU is an emulator, it isn't by itself capable of doing virtualization. It can allow a hypervisor to do CPU (and some I/O) virtualization in lieu of QEMU doing binary translation/emulation. In that case QEMUs purpose is mainly I/O emulation. 'Virtualization' as a name and concept predates QEMU by several decades.
> KVM is merely the kernel infrastructure that enables ring-0 access for faster hardware access. For many years it was called qemu-kmod; to this day the actual process that you run is called 'kvm-qemu.'
kqemu and KVM are technologically unrelated.
> You are overstating the difference between 'virtualization' and 'emulation.' There really isn't one. What used to be called 'emulation' is nowadays called 'full-virtualization' as opposed to 'paravirtualization.' QEMU, KVM, and XenKVM are full-virt, while traditional Xen is paravirt.
Para-Virtualization is the concept where the guest OS is aware that it is being run on a hypervisor. For example using hypervisor hypercalls to create new address spaces rather than setting up hardware page tables. "Full" or hardware level virtualization happens when the guest OS does not need to be aware of this fact. Neither is related to emulation.
> KVM is not a hypervisor;
It is.
> libvirt is.
This one isn't.
> Xen blurs the line between virtualization platform and hypervisor because both pieces ship within the same project. Either libvirt or Xen can act as hypervisor for QEMU or KVM.
"hypervisor":"virtualization platform" ~ "kernel":"operating system". KVM and Xen can act as hypervisor for Qemu managed by libvirt.
> QEMU on its own is not a virtualization system, just an emulator.
Thanks. That's an important distinction I had gotten wrong.
The QEMU wikipedia page describes it as "a free and open-source hosted hypervisor that performs hardware virtualization." [1]. In the context of the entire article, that description is slightly misleading.
It was certainly not created for server virtualization, though that may be the primary use with present-day Qemu.
I mostly use Qemu for cross-architecture emulation, where the convenience to simulate certain embedded architectures and devices makes it very interesting to use. The integrated GDB server is also very neat, especially for kernel debugging.
Flashing and testing embedded devices is cumbersome, so using an emulator is preferable to speed up the code-build-test-debug loop. Runtime speed is not always that important.
I have read that QEMU excels in server virtualization but lags behind VirtualBox in desktop virtualization. Is this still the case?
I am not an expert in this area, but I find it to be the case for me. Running a desktop guest under kvm is a pain - I'm about to move to a new workstation and am considering moving to virtualbox. My desktop guest under kvm doesn't accept the mouse scroll wheel, doesn't support clipboard copying, doesn't support dynamic desktop resizing, and virtualbox does. These things are theoretically fixable with kvm, but there's no clear and easy path to fix them (or wasn't, last I checked. I tried a few things and failed). Virtualbox just has 'install guest additions' and you're done.
I've not tried virtualbox for servers - the perception I had was that the management tools were really focused around desktops.
If you are talking about running Windows specifically under KVM, then switch your display from VNC to Spice, change the video adapter to QXL and inside the Windows VM install spice-tools from http://www.spice-space.org/download/binaries/spice-guest-too...
This alleviates some of the problems you are referring to.
Anyone know if there are good tools for migrating a VMware .vmx and accompanying files to a QEMU machine? Or is this more complicated than I imagine? I'd like to ditch the VMware crap modules that break with every kernel upgrade in favor of KVM.
I did just that when I switched my server from ESXi (which used the same formats). For me, the vmware2libvirt command from Ubuntu's virt-goodies package did the trick.
Thanks, this was very helpful. Here's what I did (converting VMware Workstation VM):
1) Followed rwmj's guide to setting up a br0 device for libvirt in Fedora.
2) Deleted all vmware snapshots. sync. sync.
3) Grabbed vmware2libvirt from Ubuntu (it's just a standalone python script) and
used it with --bridge br0 -f foo.vmx > foo.xml.
4) Converted my vmdk to raw with qemu-img, moved it to /var/lib/libvirtd per your
instructions
5) Edited my xml to point to the right disk, use the qemu/raw driver, use the right
path for kvm (/urs/bin/qemu-kvm on Fedora).
6) Import into libvirt with virsh define foo.xml
7) virsh start foo
It just works! Great. And virt-viewer over qemu+ssh is great; I run this VM on a headless box on the LAN from my desktop.
Helpful, thanks. But I'm reminded of my brother's email about how simple it was to create a Linux kernel for a tv-controller - 3 pages of build instructions, ending with "So easy!" What is it about Linux folks ...
Ovfconverter should be able to go both ways and translate the configuration to and from what you want. Naturally there are some limitations, as not all hypervisors support what other hypervisors support, visa versa.
KVM can work with VMware vmdk-flat files natively without conversion. Just make sure you have the drivers and kernel support for KVM installed beforehand and you don't even need to convert them.
The VirtualBox virtualization drivers have serious issues. They're written so poorly that they're notorious for causing kernel panics on linux. The kernel devs consider your kernel tainted if you load the virtualbox modules.
I've crashed so hard using virtualbox that I didn't even get a kernel panic, just a box that wouldn't even respond to alt+sysrq.
On top of this, I personally prefer QEMU because it doesn't come with a clunky GUI.
Incorrect. The kernel devs specifically cited the quality of the drivers when adding it to the taint list. Check the mailing list if you don't believe me.
The number of bug reports we get from people with virtualbox loaded are
truly astonishing. It's GPL, but sadly that doesn't mean it's good.
Nearly all of these bugs look like random corruption. (corrupt linked lists,
corrupt page tables, and just plain 'weird' crashes).
And the no-GUI, geared-towards-commandline, doesn't-have-directories-with-spaces type stuff is just my personal preference, as I've already said.
This is also bad information. Kernel taint is a bitmap that
represents a number of things:
- Proprietary software (non-gpl-compatible, as you say. ex: vmware)
- Crap (drivers/staging)
- Out-of-tree modules (virtualbox)
- whether the kernel has hit a BUG already
- ...
- lots of stuff
VirtualBox taints the kernel because it is out-of-tree, not because it is GPL-incompatible. Why is a GPL compatible kernel module(s) not merged yet?
My installs are headless, no 3D accel or Unity needed.
As originally mentioned, the VBox kernel modules are of poor quality. You may or may not trigger an edge case in your daily use, with or without a desktop environment.
I personally had a terrible time using VirtualBox on a Lenovo T420 running Windows 7 x64 about 2 years ago. I was having frequent blue screens, so Lenovo sent out a tech to swap the motherboard. The issue persisted, and after some time, I noticed that it seemed to always occur when I had multiple VMs running. The problem went away when I migrated to VMware Workstation.
1. Qemu supports more architectures, such as running ARM or MIPS code on x86 hardware.
2. Server use. It works with KVM, Xen, Virt-manager, etc; it plays nice with the ecosystem on GNU/Linux. I know my VPS is using Qemu as part of their stack.
3. That last point also means that improvements to KVM and whatnot also improve Qemu.
4. If you are a "freetard", these days VirtualBox uses non-free code in the BIOS. Qemu can be used totally free.
5. As petty as it sounds: It was around first, and people stick with what they know (this is the principle reason I use Qemu).
Actually, VirtualBox's BIOS code is 100% freely redistributable and modifiable. Unfortunately, the only compiler capable of building it is non-Free (OpenWatcom). It turns out there's not a whole lot of interest in writing compilers targeting 8086 real mode these days. :(
I think this is closest to the correct answer, so I will try to expand on it. Qemu, is a processor emulator[1] and can emulate processors like x86, x86_64, ppc, mips, arm etc. Qemu is usually run with kvm[2].
Virt-manager uses QEMU and I think it is pretty good, and much faster than Virtualbox. If you wish to use a desktop though, you should configure the video card of VM manually, its performance is not that good, but if you really want to use desktop VM, I suggest VMWare, I found the interaction much faster, but Virtualbox is pretty good to.
In short, QEMU with KVM is way faster, for servers, non-gui interactions.
In addition to KVM use couradical points out, it's also used in the Bitblaze BAP tool's TEMU component[1], which does dynamic analysis (including taint tracking) over x86 programs and kernels.
QEMU/KVM are the default virtualization used in OpenStack [1], for one. I also use it on my laptop to host guests because of how much Virtual Box sucks under Linux [2], as others have mentioned.
The best reason, IMO, is that the tools around KVM (using QEMU) and virt-manager are becoming much more pleasant to use than their VirtualBox counterparts. The GUI could still use some cleaning up, IMO, but overall I have found the experience much more pleasant in recent virt-manager releases than the VirtualBox way of doing things.
What is QEMU and why do I care? It would be helpful if that had been in the first paragraph. I am going to dig deeper, but I'm sure a huge number of people didn't.
I know where you're coming from, and I've had the same reaction to a few other similar posts on HN, however before I posted that reaction I realized two things:
1. If it's on the front page of HN then a lot of other folks around here probably know what it is already.
2. If it got voted up fairly quickly, it's probably also a big or important technology that I should familiarize myself with, at least superficially.
QEMU (short for "Quick EMUlator") is a free and open-source hosted hypervisor that performs hardware virtualization.
QEMU is a hosted virtual machine monitor: It emulates central processing units through dynamic binary translation and provides a set of device models, enabling it to run a variety of unmodified guest operating systems. It also provides an accelerated mode for supporting a mixture of binary translation (for kernel code) and native execution (for user code), in the same fashion as VMware Workstation and VirtualBox do. QEMU can also be used purely for CPU emulation for user-level processes, allowing applications compiled for one architecture to be run on another.
You could have found out quicker than typing the comment. The honourable thing to do in this situation is to find out and then post a helpful link for others.
Yeah, looking at it again, you're right, I was lazy and obnoxious. My comment should have been "For those who don't know what QEMU is (I didn't), it is ...". Won't do it again.
I get and share this complaint for random new startups that nobody has ever heard of before, and where the link is to the page that should be telling me what it is.
But this is an old, well-known project. If you put "QEMU" into Google, literally every single hit in the first page of results is about it.
1. My understanding is that the primary use case of QEMU is for server virtualization on Linux hosts leveraging KVM. Is there a use-case for QEMU on non-Linux hosts?
2. I have read that QEMU excels in server virtualization but lags behind VirtualBox in desktop virtualization. Is this still the case? [1]
3. Is it considered "slow" for cross-architecture virtualization? [1]
[1] http://superuser.com/questions/447293/does-qemus-performance...