No, its because 10-20 years after you graduate, no one asks you for your GPA at Harvard but they certainly think you are a genius just because you went there (you could graduate with a 2.0 GPA). As opposed to average state school guy with a 4.0 GPA. Many years after you graduate, you regularly get asked where you went to school but rarely asked "what was your GPA". This isn't even mentioning the type of whiz kids or socially powerful people you can befriend at these type of institutions. Steve Ballmer is smart but if he didn't meet Bill Gates, i doubt he would ever have become a multi billionare.
Harvard and Yale have 10x horrible negative character connotations compared to MIT or UChicago, and it gets worse all the time.
McNamara (Vietnam), Kissinger (illegal bombing of Laos), Clinton (impeached), Bush (WMD lies), Ted Cruz and other senators who literally can't pass 7th grade science. I can continue for an hour.
For even the "elite", character connotations take a back seat to "ability and drive", which neither of those people mentioned are lacking. To the average person, most of them probably don't even know too much about who Henry Kissinger was or what he did, the see Harvard and think "ohhh smart"