It's oversimplifying the problem. "Government has problems, so get rid of it." ignores problems in the private sector. The private sector is great at solving problems where there is a market demand that can be solved without externalities. When there are externalities -- e.g. pollution -- you need a competent regulator. You're back to square one: you always need a government. At the end of the day, someone has to do the hard work of advocating for competent government, and to ignore that is to ignore the problem.
Are you trying to say "what if there are some cases where government oversight results in a net loss?" or what you actually said: what if the net of all action against externalities by government is cheaper than having no government?
If you actually do then decide to have no government, then some other entity will take over and they will be your new government. At the end of the day you have to be part of some group, and that group needs rules. Democracy happens to be my preferred way of deciding rules for the group, but maybe you have some other preference? Maybe the richest should be able to do whatever they want as long as it benefits them and those that they transact with. Who am I to judge.
If we are only talking about the solution to some externalities, and that overall government is a net win, then the solution is to make sure that you are electing people that care about governing wisely. That means that they know when to regulate and when to remove regulations. Too much of either is disastrous.
