EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Diffusion models in text generation: a survey

Qiuhua Yi, Xiangfan Chen, Chenwei Zhang, Zehai Zhou, Linan Zhu, Xiangjie Kong iD

PeerJ Computer Science · 23.02.2024 · 10 · e1905

DOI: 10.7717/peerj-cs.1905 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 23.02.2024. Verfasst von Qiuhua Yi und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Computer Science and Technology, Zhejiang University of Technology, HangZhou, China. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 86 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Diffusion models are a kind of math-based model that were first applied to image generation. Recently, they have drawn wide interest in natural language generation (NLG), a sub-field of natural language processing (NLP), due to their capability to generate varied and high-quality text outputs. In this article, we conduct a comprehensive survey on the application of diffusion models in text generation. We divide text generation into three parts (conditional, unconstrained, and multi-mode text generation, respectively) and provide a detailed introduction. In addition, considering that autoregressive-based pre-training models (PLMs) have recently dominated text generation, we conduct a detailed comparison between diffusion models and PLMs in multiple dimensions, highlighting their respective advantages and limitations. We believe that integrating PLMs into diffusion is a valuable research avenue. We also discuss current challenges faced by diffusion models in text generation and propose potential future research directions, such as improving sampling speed to address scalability issues and exploring multi-modal text generation. By providing a comprehensive analysis and outlook, this survey will serve as a valuable reference for researchers and practitioners interested in utilizing diffusion models for text generation tasks.

Schlagwörter

Stichwörter der Autor:innen

Diffusion modelsNatural language generationText generation

Autor:innen und Institutionen

  • College of Computer Science and Technology, Zhejiang University of Technology, HangZhou, China.
  • School of Faculty of Education, University of Hong Kong, Hong Kong, China.

ORCID-Profile: Xiangjie Kong

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Yi, Q., Chen, X., Zhang, C., Zhou, Z., Zhu, L., & Kong, X. (2024). Diffusion models in text generation: a survey. PeerJ Computer Science, 10, e1905. https://doi.org/10.7717/peerj-cs.1905

Vancouver

Yi Q, Chen X, Zhang C, Zhou Z, Zhu L, Kong X. Diffusion models in text generation: a survey. PeerJ Computer Science. 2024;10:e1905. doi:10.7717/peerj-cs.1905

BibTeX
@article{Yi2024,
  author = {Yi, Qiuhua and Chen, Xiangfan and Zhang, Chenwei and Zhou, Zehai and Zhu, Linan and Kong, Xiangjie},
  title = {Diffusion models in text generation: a survey},
  journal = {PeerJ Computer Science},
  year = {2024},
  volume = {10},
  pages = {e1905},
  doi = {10.7717/peerj-cs.1905}
}
Kostenlos starten →