Because that is myopic or selfish. You didn't or couldn't buy a house when it was smart, now you want to screw over people who bought homes recently because prices are unaffordable. That is either selfish because you don't care about their property values (when they probably much like you had to work hard for it), or myopic because you didn't even consider it.
What exactly are you even proposing? You want the government to mandate cheaper land, construction material, and builder wages? To lower taxes on all of the above? People who think there is some simple solution "duh just build more houses!" are extremely myopic. Again it's not my problem, I'm not against building more houses and I also get NIMBY because "affordable" or government subsidized housing has a direct correlation to crime rates, lowering property values, and lowering the quality of schools.
I'm simply saying those who were stuck buying at inflated prices might be opposed to that.
My home is paid off and I got it for half of it's market value. I have no skin in the game either way as I'd never sell.