On: The Curse of Recursion: Training on Generated Data...

[beezlewax]

Is that not what open ai did to train these models originally though?

[r00fus → beezlewax]

Possibly, but in this case, what may be looked at as a "bootstrap" sounds like ongoing maintenance cost.

With "free data" from reddit gone, now the cost of symbiotic gen+human data will be even more expensive.