Men care because women care. I don't think it's that far-fetched that women prefer men with hair, and, as someone who had hair, then didn't, and now does again, I can definitely confirm it.
Women are free with their opinions on men's looks. They want to see height, chest hair, facial hair... but they seem pretty consistent in stating that they don't really care about baldness.