Fair enough, if you fear karmic retribution that makes sense. I don't, so for me the shoe remains on the same foot. Other people are more valuable to me than animals, and also more dangerous. The stakes are higher and therefore the moral considerations are more prudent (or karmic, if that's your bag).
Karma has nothing to do with revenge, karma is just another word for consequences. Every action leads to reactions; if they didn't the universe would have descended into chaos in no time at all. And thats how we learn, by acting and studying reactions until we get it. One of those lessons is recognizing the value of all living beings.