The primary limiting factor at the moment is obviously the insane price of rocketing stuff into space.
The people who are opposed to nuclear power because of the military proliferation risk might have something to say about the capability to transmit large amounts of power from space to the surface of this planet.
And you'd have to add to that objection the people who oppose nuclear power because of the costs. Those people would definitely be opposed to these ideas barring some new technological development that made space-based infrastructure several orders of magnitude less expensive.
The people who are opposed to nuclear power because of the military proliferation risk might have something to say about the capability to transmit large amounts of power from space to the surface of this planet.