That depends on how cheap space flight is in the future, in the past (say 1800) the thought that it would be substantially cheaper to manufacture something in China and ship it across the pacific ocean than it would be to manufacture it locally was not only laughable but utterly insane.
That depends on how cheap space flight is in the future
Your argument works better to support the statement, "That depends on how cheap space manufacturing is in the future." We can fairly easily procure massive amounts of aluminum and silicon on the Moon. If we can invent industrial processes and perfect tele-operation, then we could create massive solar farms on the Moon. However, it might be even better to launch the material from the lunar surface with electromagnetic drivers and construct the solar farms in orbits, where they would receive sunlight 100% of the time.
Japan's space agency has already conducted tele-operation tests in conditions of massive lag. It turns out that highly motivated people can tele-operate manipulators even with 1 second of lag. This puts the Moon within reach of such tele-operation.
Sure, solar panels operate at a significantly higher efficiency on the moon due to no atmosphere to diffuse the sun's energy... but then you have to pay the atmosphere cost to get it to Earth.
Your suggestion presupposes that we can transfer energy through the atmosphere more efficiently than the energy of the sun is transferred through it.
We certainly could do so if we had a space elevator (but at that point just put the solar panels in geostationary orbit with the elevator, not on the moon).
I remember reading about this concept (geez!) 20 years ago, though the sources were not as far as the moon [1]. I believe Isaac Asimov ranked civilizations based on the proportion of energy gathered from its main star.
(Just thinking out loud here)