My understanding, though I've never personally made it explicit, is that you view "contra/covariance" as arising not as a property of vector spaces exactly (concretely) but instead from tensors-as-multilinear-maps and you talk about them as functors between transformation groups on vector spaces which either preserve (co-) or reverse (contra-) transformations.
The inner product lets you view vector spaces as spaces of transformations already so the same sort of idea comes out now viewing vector spaces as maps in their own right.
The inner product lets you view vector spaces as spaces of transformations already so the same sort of idea comes out now viewing vector spaces as maps in their own right.