In solutions that don't require unobtanium, current cells are arguably only about 1/4th of the way there. In solutions that have been constructed regardless of cost, it's still not 1/2 the way.
I think seeing another 100% gain in affordable solutions is conservative, since 400% is what's possible.
If the vast majority of buildings used "negative" power (that is that they generated more back into the grid then they took out) due to small scale power generation on premises, then the net effect may not dictate a need for a grandiose centralized solution.
When people compare say a million homes power needs, which is about 1000Gwh/month, in the non-distributed systems you need a 1000Gwh/month solution (think nuclear). However, this hypothetical 4-fold increase in efficiency would need only 7 solar panel (1.6m x 1m) per house for a net zero need distributed system.
If the right market incentives were there, and in a future market anticipating the swanson curve, someone may install say 25 panels (18 in excess of their need) to generate monthly passive income ("fans" of solar have 30+ panels)
Under this model, someone would be powering their house and two of their neighbors.
You can see how this can add up with the right market structure.
My argument is that in a world of a free-market entrepreneurial energy, you won't need a centralized power generator capable of powering 1 million homes. In fact, you may not need centralized power generation at all.
The future problems aren't going to be in power generation, but in storage.
There's a pretty solid upper-bound on that.