Text Diffusion with Reinforced Conditioning
Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (12) · 14069-14077
DOI: 10.1609/aaai.v38i12.29316 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yuxuan Liu und 8 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref).
Abstract (Originalsprache)
Diffusion models have demonstrated exceptional capability in generating high-quality images, videos, and audio. Due to their adaptiveness in iterative refinement, they provide a strong potential for achieving better non-autoregressive sequence generation. However, existing text diffusion models still fall short in their performance due to a challenge in handling the discreteness of language. This paper thoroughly analyzes text diffusion models and uncovers two significant limitations: degradation of self-conditioning during training and misalignment between training and sampling. Motivated by our findings, we propose a novel Text Diffusion model called TReC, which mitigates the degradation with Reinforced Conditioning and the misalignment by Time-Aware Variance Scaling. Our extensive experiments demonstrate the competitiveness of TReC against autoregressive, non-autoregressive, and diffusion baselines. Moreover, qualitative analysis shows its advanced ability to fully utilize the diffusion process in refining samples.
Zitieren
APA
Liu, Y., Yang, T., Huang, S., Zhang, Z., Huang, H., Wei, F., Deng, W., Sun, F., & Zhang, Q. (2024). Text Diffusion with Reinforced Conditioning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(12), 14069-14077. https://doi.org/10.1609/aaai.v38i12.29316
Vancouver
Liu Y, Yang T, Huang S, Zhang Z, Huang H, Wei F, et al. Text Diffusion with Reinforced Conditioning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(12):14069-14077. doi:10.1609/aaai.v38i12.29316
BibTeX
@article{Liu2024,
author = {Liu, Yuxuan and Yang, Tianchi and Huang, Shaohan and Zhang, Zihan and Huang, Haizhen and Wei, Furu and Deng, Weiwei and Sun, Feng and Zhang, Qi},
title = {Text Diffusion with Reinforced Conditioning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2024},
volume = {38},
number = {12},
pages = {14069--14077},
doi = {10.1609/aaai.v38i12.29316}
}
EUVIMED