That is a really good point and something I hadn't really considered. I could also see a lot of my feelings of empathy being instead turned into anger if things had been different so I might have generalized a personal experience.
I'm well out of my pay grade, but with the various anti-social / psychopathic disorders, it's not an empathy/anger axis so much as just not caring about consequences for others. It's a total disregard. More troubling: there's very, very little by way of treatment for many of these disorders, whether you're looking at pharmaceuticals or talk therapy. The DSM can make for interesting late-night reading.