Completely unrelated, but what do you think about vips (http://www.vips.ecs.soton.ac.uk/)? I recently discovered the difference in performance between im and gm and that let me to investigate vips, which is presumably (a lot) faster than both.
In principle, it looks promising (in particular, using ORC to compile image manipulation kernels to SIMD), but it is a different thing to im or gm. It's a library and a GUI. The library is slightly too low-level for most things I'd use gm for, but a GUI is too laborious.
Perhaps it could be integrated into GM or something similar, and used as a back end for certain operations.
(GraphicsMagick is the ImageMagick of pictures, for me. IM has a few more features, but GM is more stable and usually much faster.)