Patch author here. bzbarsky's comment covered most of it. It hasn't turned out to be that difficult. On linux, it was mostly playing whack-a-mole of avoiding calls in to gtk/x11 and then creating headless implementations of the platform specific code.
One slight complication of headless has been wiring up the headless "widgets" (in Firefox, we refer to most of the platform specific code as widgets e.g. there's gtk widgets, cocoa widgets, windows widgets). Usually the widget type is defined statically at build time, but in the case of headless we wanted Firefox to either use the headless widgets or the platform specific widgets at runtime. Luckily, some of the work on multi-process Firefox work also added another type of widget and made it much easier to support multiple types of widgets.
Overall, the hardest part has been trying to replicate all the events that would normally be triggered by the platform gui code. I've also found that these events and order can vary per platform which makes it hard to do in a non-platform specific way. We're still working out some issues here.
One slight complication of headless has been wiring up the headless "widgets" (in Firefox, we refer to most of the platform specific code as widgets e.g. there's gtk widgets, cocoa widgets, windows widgets). Usually the widget type is defined statically at build time, but in the case of headless we wanted Firefox to either use the headless widgets or the platform specific widgets at runtime. Luckily, some of the work on multi-process Firefox work also added another type of widget and made it much easier to support multiple types of widgets.
Overall, the hardest part has been trying to replicate all the events that would normally be triggered by the platform gui code. I've also found that these events and order can vary per platform which makes it hard to do in a non-platform specific way. We're still working out some issues here.