Recursive learning without collapse: a weighting-based stabilization framework
Journal of the Royal Statistical Society Series B: Statistical Methodology · 01.07.2026
DOI: 10.1093/jrsssb/qkag099 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the Royal Statistical Society Series B: Statistical Methodology, online erschienen am 01.07.2026. Verfasst von Hengzhi He und 2 weiteren Autor:innen. Die Arbeit nennt 28 Literaturangaben. Gefördert von: NSF, Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
Recent studies identified an intriguing phenomenon in recursive generative model training known as model collapse, where models trained on data generated by previous models exhibit severe performance degradation. Addressing this issue and developing more effective training strategies have become central challenges in generative model research. In this paper, we investigate this phenomenon within a novel framework, where generative models are iteratively trained on a combination of newly collected real data and synthetic data from the previous training step. To develop an optimal training strategy for integrating real and synthetic data, we evaluate the performance of a weighted training scheme in various scenarios, including Gaussian distribution estimation, generalized linear models, and non-parametric estimation. We theoretically characterize the impact of the mixing proportion and weighting scheme of synthetic data on the final model’s performance. Our key finding is that, across different settings, the optimal weighting scheme under different proportions of synthetic data asymptotically follows a unified expression, revealing a fundamental trade-off between leveraging synthetic data and model performance. In some cases, the optimal weight assigned to real data corresponds to the reciprocal of the golden ratio. Finally, we validate our theoretical results on extensive simulated datasets and a real tabular dataset.
Förderung
- NSF
- Fundamental Research Funds for the Central Universities
Zitieren
APA
He, H., Xu, S., & Cheng, G. (2026). Recursive learning without collapse: a weighting-based stabilization framework. Journal of the Royal Statistical Society Series B: Statistical Methodology. https://doi.org/10.1093/jrsssb/qkag099
Vancouver
He H, Xu S, Cheng G. Recursive learning without collapse: a weighting-based stabilization framework. Journal of the Royal Statistical Society Series B: Statistical Methodology. 2026. doi:10.1093/jrsssb/qkag099
BibTeX
@article{He2026,
author = {He, Hengzhi and Xu, Shirong and Cheng, Guang},
title = {Recursive learning without collapse: a weighting-based stabilization framework},
journal = {Journal of the Royal Statistical Society Series B: Statistical Methodology},
year = {2026},
doi = {10.1093/jrsssb/qkag099}
}
EUVIMED