Your comment implies it will break on anything that's not "Linux on Intel/AMD", but that's not the case. Most architectures are either little-endian or support both little- and big-endian (although not all operating systems support little-endian on a bi-endian architectures, e.g. I believe AIX doesn't).
I'm willing to bet that the number of big-endian systems amounts to less than 1% of all systems running X, and the number of those that use X forwarding is a subsection of that. I'm big on maintaining compatibility, but it's a fairly minor break which benefits almost everyone and affects very few people.
PowerPC is big endian, but it seems like the world has been standardizing/converging on little endian for a while now. It's not just Intel and AMD, it's also all ARM CPUs and RISC-V.
To me, this seems a bit similar to the 8-bit byte. We take that for granted nowadays, of course memory is divided into bytes, but it wasn't always so. There used to be machines with memory that was addressable only in 36-bit or 40-bit words. There also used to be machines that uses one's complement to encode negative integers, before everything became two's complement.
I hope that eventually, all CPUs are little endian. It's easier to interoperate without these arbitrary distinctions.
I'm willing to bet that the number of big-endian systems amounts to less than 1% of all systems running X, and the number of those that use X forwarding is a subsection of that. I'm big on maintaining compatibility, but it's a fairly minor break which benefits almost everyone and affects very few people.