I wonder if this misconception has more to do with the fact that programming languages define left/right bitshift operators and since left shift means making a number bigger, that implies the most significant bits are on the left (eg first in a LTR language).
