On: Phi-4: Microsoft's Newest Small Language Model Specializing in Complex Reasoning

[simonw]

It shows that if you deliberately train LLMs against their own output in a loop you get problems. That's not what synthetic data training does.

[belter → simonw]

I understand and appreciate your clarification. However would it not be the case some synthetic data strategies, if misapplied, can resemble the feedback loop scenario and thus risk model collapse?