What really happens: if everyone tries to acquire a particular income producing asset, the asset gets so expensive that the return is not worth the risk plus the time value of the money it could generate. Stocks in the 1920's, Tokyo real estate in the 1990s, US housing the mid 2000's...
This is also true of being a doctor, practicing law, growing fruit, programming computers, driving a cab, creating art, making music, running a car wash, and crab fishing.
Passive income usually requires a burst of highly-productive time at the beginning.
If you invent a machine which does farming with no human labor, and I invent a factory which makes your machines with no human labor, and so on down the line:
1) Human productivity jumps to infinity
2) No one needs to work
3) We all collect passive income, growing exponentially*
Most passive income is some subset of that. If you write a book read by millions, or invent something saving lives, that's continuing to generate real value for the world. Even if you work very, very hard, while scrimping and saving, so you don't have to work later, everything works out okay (so long as there are kids willing to work hard).
* With different exponents, until we have a class of ultrarich and ultrapoor, until the revolution, with either the poor masses being killed by high tech, or the rich by the human masses