This line of reasoning still works even if it's only a few people buying things (think excessively rich people buying ever increasing numbers of luxury goods).
In reality the marginal propensity to consume is greater the less money you make, so there's an argument to be made that increases in wealth disparity might in and of themselves cause a decrease in consumption. It gets complicated though, as when people don't consume with dollars they invest them, and invested dollars are ultimately spent on something too.
Invested money is spent on something but only if someone wants to borrow the money. Now the financial system has got very good at recycling the savings of the rich into more borrowing of the poor (subprime loans), but it is hard to see this continuing at that rate. The rich largely want risk free savings (in aggregate), so the banks turned high risk loans into risk free savings, with the government backstopping them. This has not been working too well. The vast majority of wealth is looking for safe assets (except the Elon Musks) which effectively means people need to have jobs to pay the rentiers.
That, and invested money is not cyclical in the economy. Yes, the invested spend it on something, but only on the pretense that future returns of theirs will go back to the original investor with interest. The only exchange of resources there is the time between having money now and giving more money back later.
Compared to more equitable economics where the baker gives the mechanist bread to fix the milkmans car who gives the baker milk, and each of them is using cash received from the other to provide goods for their wants and needs, is much more tantamount of a functioning economy than loans, especially when investment money seeks the safest of avenues for returns, thus the vast majority of potential entrepreneurs are out of scope of getting the money anyway.
In reality the marginal propensity to consume is greater the less money you make, so there's an argument to be made that increases in wealth disparity might in and of themselves cause a decrease in consumption. It gets complicated though, as when people don't consume with dollars they invest them, and invested dollars are ultimately spent on something too.