Yes, it costs money to get a patent invalidated based on prior art. But: an amount that even a single person who stands to actually gain from having a patent overturned should have no problem with. You're asking the USPO to spend time redoing work, literally halting any other patent work they could be doing instead. So it's not a trivial amount, but it's also hardly a prohibitive amount if you actually want a specific patent revoked.
The AIPLA Report of the Economic Survey for 2017 notes that the typical patent infringement suit with less than $1 million at stake costs on average costs more than $600,000 dollars, while the typical patent infringement suit with between $1-10 million at stake costs on average nearly $1.5 million to litigate.
Yes, it costs money to get a patent invalidated based on prior art. But: an amount that even a single person who stands to actually gain from having a patent overturned should have no problem with. You're asking the USPO to spend time redoing work, literally halting any other patent work they could be doing instead. So it's not a trivial amount, but it's also hardly a prohibitive amount if you actually want a specific patent revoked.