EUVIMEDEUVIMED entdecken
Crossref

A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond

Yisheng Xiao, Lijun Wu, Junliang Guo, Juntao Li, Min Zhang, Tao Qin, Tie-Yan Liu

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2023-10 · 45 (10) · 11407-11427

DOI: 10.1109/tpami.2023.3277122 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023-10. Verfasst von Yisheng Xiao und 6 weiteren Autor:innen. Die Arbeit wurde bisher 83-mal zitiert (Crossref) und nennt 242 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Beijing Academy of Artificial Intelligence.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Non-autoregressive (NAR) generation, which is first proposed in neural machine translation (NMT) to speed up inference, has attracted much attention in both machine learning and natural language processing communities. While NAR generation can significantly accelerate inference speed for machine translation, the speedup comes at the cost of sacrificed translation accuracy compared to its counterpart, autoregressive (AR) generation. In recent years, many new models and algorithms have been designed/proposed to bridge the accuracy gap between NAR generation and AR generation. In this paper, we conduct a systematic survey with comparisons and discussions of various non-autoregressive translation (NAT) models from different aspects. Specifically, we categorize the efforts of NAT into several groups, including data manipulation, modeling methods, training criterion, decoding algorithms, and the benefit from pre-trained models. Furthermore, we briefly review other applications of NAR models beyond machine translation, such as grammatical error correction, text summarization, text style transfer, dialogue, semantic parsing, automatic speech recognition, and so on. In addition, we also discuss potential directions for future exploration, including releasing the dependency of KD, reasonable training objectives, pre-training for NAR, and wider applications, etc. We hope this survey can help researchers capture the latest progress in NAR generation, inspire the design of advanced NAR models and algorithms, and enable industry practitioners to choose appropriate solutions for their applications.

Förderung

  • National Natural Science Foundation of China
  • Beijing Academy of Artificial Intelligence

Zitieren

APA

Xiao, Y., Wu, L., Guo, J., Li, J., Zhang, M., Qin, T., & Liu, T. Y. (2023). A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10), 11407-11427. https://doi.org/10.1109/tpami.2023.3277122

Vancouver

Xiao Y, Wu L, Guo J, Li J, Zhang M, Qin T, et al. A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(10):11407-11427. doi:10.1109/tpami.2023.3277122

BibTeX
@article{Xiao2023,
  author = {Xiao, Yisheng and Wu, Lijun and Guo, Junliang and Li, Juntao and Zhang, Min and Qin, Tao and Liu, Tie-Yan},
  title = {A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2023},
  volume = {45},
  number = {10},
  pages = {11407--11427},
  doi = {10.1109/tpami.2023.3277122}
}
Kostenlos starten →