Equivariant machine learning is a thing that people have tried... Tends to be expensive and slow, though, and imposes invariances that our model (a universal function approximator, recall) should just learn anyway: If you don't have enough pictures of upside down cows, just train a normal model with augmentations.
