When someone says a consideration is orthogonal to another one, they are not literally embedded in an Euclidean space. In this context, first order is used loosely to mean that the deviations come from effects which are only secondary in magnitude and relevance, not to indicate that we're looking at the asymptotic behavior of a formula when a parameter is taken to be infinitesimal.
The trouble is that EMH is a formula, so the "to first order" thing has a literal meaning that is not what the author meant. It's kind of like saying that "1-x^2/2 is cos(x) to first order," moreso than saying "my business plan is a good idea for every involved party, to first order."
The EMH is accurate to way better than the tangent line, which is what a first order approximation is.