>> Do you know of any open source projects with the approximately $5 million it takes to train GPT3?
Two things. 1) research into smaller more efficient models and 2) hardware prices will come down for a bit longer. So longer term this kind of thing should be free.
It's common for companies to be first at complex things because the coordinated effort or cost involved, and later that becomes more feasible and cheap or free options become possible. I'm all for inventors/companies getting paid for new developments, but I'm also not terribly excited for them to keep innovation locked up and charge rent on it for eternity. This works out in most cases at varying pace, but not always.
Two things. 1) research into smaller more efficient models and 2) hardware prices will come down for a bit longer. So longer term this kind of thing should be free.
It's common for companies to be first at complex things because the coordinated effort or cost involved, and later that becomes more feasible and cheap or free options become possible. I'm all for inventors/companies getting paid for new developments, but I'm also not terribly excited for them to keep innovation locked up and charge rent on it for eternity. This works out in most cases at varying pace, but not always.