I think the egotistical aspect is the most powerful: many researchers have built an identity based on the fact that they “know” something, so to propose better alternatives to their pet theories is tantamount to proposing their life is a lie. To change their mind they need to admit they didn’t “know”.
The better the alternatives, the more fierce the passion with which they will be rejected by the mainstream.
I now think it’s best explained by simple economics. Academia and academics are the product of economic forces by and large. It’s not quirky personalities or uniquely talented minds that make up academia today. It’s droves of conscientious (big five sense) conformists, with either high iq or mere socio-economic privilege, who have been trained by our society to feel that financial security means college, and even more financial security means even more college. Credentials are like alpha .05, they solve a scale problem in a way that alters the quality/quantity ratio. If you want more researchers/research/science output, credentials and alpha .05 cargo cult stats are your levers to get more quantity at lower quality.
The better the alternatives, the more fierce the passion with which they will be rejected by the mainstream.