Diffusion models in text generation: a survey
PeerJ Computer Science · 23.02.2024 · 10 · e1905
DOI: 10.7717/peerj-cs.1905 ↗Kurz gefasst
Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 23.02.2024. Verfasst von Qiuhua Yi und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Computer Science and Technology, Zhejiang University of Technology, HangZhou, China. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Diffusion models are a kind of math-based model that were first applied to image generation. Recently, they have drawn wide interest in natural language generation (NLG), a sub-field of natural language processing (NLP), due to their capability to generate varied and high-quality text outputs. In this article, we conduct a comprehensive survey on the application of diffusion models in text generation. We divide text generation into three parts (conditional, unconstrained, and multi-mode text generation, respectively) and provide a detailed introduction. In addition, considering that autoregressive-based pre-training models (PLMs) have recently dominated text generation, we conduct a detailed comparison between diffusion models and PLMs in multiple dimensions, highlighting their respective advantages and limitations. We believe that integrating PLMs into diffusion is a valuable research avenue. We also discuss current challenges faced by diffusion models in text generation and propose potential future research directions, such as improving sampling speed to address scalability issues and exploring multi-modal text generation. By providing a comprehensive analysis and outlook, this survey will serve as a valuable reference for researchers and practitioners interested in utilizing diffusion models for text generation tasks.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Computer Science and Technology, Zhejiang University of Technology, HangZhou, China.
- School of Faculty of Education, University of Hong Kong, Hong Kong, China.
ORCID-Profile: Xiangjie Kong
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Yi, Q., Chen, X., Zhang, C., Zhou, Z., Zhu, L., & Kong, X. (2024). Diffusion models in text generation: a survey. PeerJ Computer Science, 10, e1905. https://doi.org/10.7717/peerj-cs.1905
Vancouver
Yi Q, Chen X, Zhang C, Zhou Z, Zhu L, Kong X. Diffusion models in text generation: a survey. PeerJ Computer Science. 2024;10:e1905. doi:10.7717/peerj-cs.1905
BibTeX
@article{Yi2024,
author = {Yi, Qiuhua and Chen, Xiangfan and Zhang, Chenwei and Zhou, Zehai and Zhu, Linan and Kong, Xiangjie},
title = {Diffusion models in text generation: a survey},
journal = {PeerJ Computer Science},
year = {2024},
volume = {10},
pages = {e1905},
doi = {10.7717/peerj-cs.1905}
}
EUVIMED