X11 is an old display protocol predating modern concepts of application sandboxing. It gives gives windows lots of control, like allowing them to move the mouse, enter keystrokes, etc. Basically, anyone who has an X11 handle has the same level of access as the user. In Firefox, the process that's supposed to be sanboxed (the content process) has precisely such a handle.
No per site process isolation, Firefox on X11/Linux is baaaad, and that's just the beginning.