Saying society is going to collapse due to population decline is about as absurd as saying society will collapse due to overpopulation.
What will end up happening is that some groups will maintain replacement fertility levels or higher, and some groups won't, but we'll trend towards an equilibrium, or flop between too little fertility and too much fertility. Either way we won't go extinct unless we explicitly kill each other.
It depends on what you mean by society. If you mean the entire world, then of course I agree with you. But cultures and societies do go extinct, quite regularly. The Roman Empire, most notably, also had a fertility collapse in their final years. And by the end Rome itself had just become a facsimile of what was once the greatest city of the greatest empire in the world.
So, for instance, many people don't realize that right now is still the good times for Japan. Here [1] is their population pyramid. It currently still looks kinda-sorta pyramidish. But in ~20 years, their population pyramid will be completely and wholly upside down. And their population, economy, and everything else will fall into complete collapse. They'll be losing about half of their population each 20 years at that point. No culture can survive this. This will be made even more true by the fact that it will likely trigger a vicious cycle where the massively skewed age ratios, collapsing economy, and other issues will further reduce fertility.
What will end up happening is that some groups will maintain replacement fertility levels or higher, and some groups won't, but we'll trend towards an equilibrium, or flop between too little fertility and too much fertility. Either way we won't go extinct unless we explicitly kill each other.