Model Collapse Is Avoidable Even as Real Data Vanishes: A Sharp Criterion for Iterative Gaussian Estimation
arXiv stat.ML11h4 min read
arXiv:2611.00259v1 Announce Type: new Abstract: Large generative models are commonly trained on data collected from the open web, which may contain data produced by previous generations of the same model. Previous work has shown that recursively training on model-generated data can progressively degrade model performance, a phenomenon known as model collapse. Yet, generative models continue to improve dramatically over time. Existing theory suggests that this is enabled by the availability of new real data at each iteration, kept separate from generated data. However, exact separation is not r