Not going out of your way to help people doesn't make you an asshole.
> I buy things I don’t need knowing the same money could literally save lives. I worry about my own security even though I could find a new job within a month.
These things are normal, I wouldn't worry too much about it.
But you are right that going through life thinking you are some lonely, tortured genius is not healthy and is a recipe for being a true asshole.
Not helping others when you are able doesn't make me an asshole; it makes me self centered. Yes, it is quite normal, but I disagree with your next clause: we should think harder about it and not dismiss it so easily.
I've pitched this one before, as it is a short book (pdf, or audiobook) and is one of the most impactful things I've read in a long time. It was Peter Singer's book, The Life You Can Save [1]
The book methodically and non-judgmentally addresses many objections and false beliefs people have about charity [2] and effective altruism. In a nutshell: ideally we'd have enough resources that everyone had everything they need, but we don't. There is a strong instinct to care more about people who are similar to you or live near you than for others, but we shouldn't. That is, the suffering of a person you will never meet is just as real as the suffering of someone you are friends with. Because we have limited resources to help others, the resources we do have should be focused on those who are worst off.
[2] For instance, many people moan about how much of the federal budget is spent/wasted on foreign aid. When asked to guess how much of the budget is spent, people (in the US) commonly guess 10-20%. When asked what they thing would be more appropriate, they commonly say 5-10%. The reality is it is well under 1%.
To play a devil’s advocate (I don’t think I’m right, it’s just the next obvious thought), the life you save produces another two, and after few steps you still can’t. Animal charities, and I feel sorry for this comparison, in areas without controlled population tend to overwhelm themselves to the point where the charity dilutes into conditions indistinguishable from natural.
You may say, well, then help fixing the root cause. That’s right, but you have to find true one and fix it, and both is hard. Much easier to calm yourself by a donation to people who you prefer to believe to understand more.
One point is that when people have secure lives, the birth rate plummets. When your only capital is children, you make more children. By meeting the needs of the worst off, they have fewer children, not more.
I'm sure there were other points made, but I don't want to attempt to restate those from my shaky memory of what I read more than a year ago.
> I buy things I don’t need knowing the same money could literally save lives. I worry about my own security even though I could find a new job within a month.
These things are normal, I wouldn't worry too much about it.
But you are right that going through life thinking you are some lonely, tortured genius is not healthy and is a recipe for being a true asshole.