First, NYC is also kind of a clusterfuck; rents there are terrible. But while NYC needs to continue building more, it at least has the excuse that it's built up a lot already, and its mayor is actively encouraging more development.
People don't cram into arbitrarily small spaces, so one thing to remember is the cause and effect: way more than 840K people want to live in SF proper, but there's no room. So 376K housing units effectively sets the population cap. But because a larger pool of people want to live there, the prices all go up. Rent control fixes that for incumbent residents but worsens the problem for everyone else. The only way to alleviate that is to create more houses so that a larger percentage of the people who want to live there actually can.
People don't cram into arbitrarily small spaces, so one thing to remember is the cause and effect: way more than 840K people want to live in SF proper, but there's no room. So 376K housing units effectively sets the population cap. But because a larger pool of people want to live there, the prices all go up. Rent control fixes that for incumbent residents but worsens the problem for everyone else. The only way to alleviate that is to create more houses so that a larger percentage of the people who want to live there actually can.