It's really a matter of experience (and sometimes lots of brute force); it's not easy to explain but I am able to make a reasonable guess at identifying ARM, MIPS, and x86, and Z80 by looking at the "texture" of the bytes as displayed in a hexdump or ASCII. Basically, stare at enough binaries for long enough and it tends to come naturally, just like people are usually able to tell apart human languages even if they don't actually know them.
