Space partially, the US House of Reps is already pretty packed so to add enough to get to >2 members per district you'd have to add a lot. You also get into potential issues diluting power that much, with so many members things like committee memberships become the only people with real power on an individual level.
Increasing it up to the state level maybe any smaller and you'll still have room for gerrymandering. Could even have districts still for people to have a person to directly go to but who represents which district is down to the statewide percentages. I'd love to see something like that combined with a ranked choice system. First past the post really heavily incentivizes a 2 party system.