There are private high schools that cost tens of thousands of dollars too. The OP was talking as if this was normal. The average student is going going to a state school and paying somewhere near $10k a year for tuition.
If you can't get a scholarship and it's not a top 10 school $50k per year probably isn't worth paying.
And then with 10-15k for overpriced student housing since most colleges make mandatory for Freshmen and sometimes even Sophomores. Then you might have to pay 1k a year to pay for a parking spot. 2-5k for cafeteria access. Add it all up and you've got 80k-100k(at 4%-9% interest) for four years at a state school. Its not impossible to get out from under but it represents a large delay, especially for degrees with low ROI
Random googling turns up 87 colleges that have residency requirements[1]. So you're right that it isn't "most" but it is also not super rare. I'm just saying its a sneaky way for colleges to bump up costs. Most college students won't go through the work to get a waiver and will just go with the flow. Its like the difference between opt-in and opt-out.
If you can't get a scholarship and it's not a top 10 school $50k per year probably isn't worth paying.