Did any other contemporary CPU have such arcane operating modes? I have a general memory that the Motorola 68K had user and supervisor modes but the concept seemed much more straigtforward. Why did developers tolerate this mess?
This is all for backwards compatibility. Unreal mode is kind of an "accidental" mode that was not created intentionally by Intel. Intel's idea was that the processor would start in the backwards-compatible real mode, and if you switched to protected mode you would never look back.
But real mode was implemented on top of logic designed for protected mode, because that is a simpler way to do things (think of it like protected mode, but you're not protecting anything).
So you go into protected mode, mess with things, switch back to real mode (all unintended by Intel) and you're in “unreal mode”.
> Why did developers tolerate this mess?
The hardware was cheap, performance was decent, and there were plenty of customers.
Less accidental; more like undocumented. BIOSes have used Unreal Mode, EMMs/XMSs can or temporarily use Unreal mode, Intel's SMM use it. There's even some evidence modern processors still support & use it.
One downside to Unreal Mode is that leaves the processor in in a state where naughty or non-standard TSRs, Interrupts, BIOS ROM code, and programs can cause hard lock ups.