EUVIMEDDescubrir EUVIMED
Open access · CC BY 4.0

Autoencoders in Natural Language Processing: A Comprehensive Review

Moussa Redah, Wasfi G. Al-Khatib

Computers · 2026-04-08 · 15 (4) · 232

DOI: 10.3390/computers15040232 ↗

At a glance

Journal article in Computers, published online on 2026-04-08. Written by Moussa Redah and 1 co-authors. 44 references. The full text is freely available (open access, CC BY 4.0). Funded by: Interdisciplinary Research Center for Intelligent Secure Systems (IRC-ISS) at King Fahd University of Petroleum & Minerals.

ResumenDatos bibliográficosCiteRelated studies

Abstract (original language)

Autoencoder-based models have become a fundamental component of unsupervised and self-supervised learning in natural language processing (NLP), enabling models to learn compact latent representations through input reconstruction. From early denoising autoencoders to probabilistic variational autoencoders (VAEs) and transformer-based masked autoencoding, reconstruction-driven objectives have played a significant role in shaping modern approaches to text representation and generation. This review provides a comprehensive analysis of the evolution of autoencoder architectures and training objectives in NLP, and synthesizes applications of VAEs across language modeling, controllable text generation, machine translation, sentiment modeling, and multilingual representation learning. Although previous surveys have examined deep generative models or representation learning in NLP, there remains a lack of a unified review that systematically connects classical autoencoder variants, variational formulations, and modern transformer-based masked autoencoders within a single conceptual framework. To address this gap, this work consolidates architectural developments, training objectives, and major application domains under a reconstruction-based learning perspective, offering a structured comparison of modeling choices, datasets, and evaluation practices. Our analysis highlights the strengths and limitations of existing approaches, discusses the ongoing influence of autoencoder-style learning in NLP, and outlines future research directions focused on improving training stability, designing more structured latent spaces, and enhancing multilingual representation learning.

Funding

  • Interdisciplinary Research Center for Intelligent Secure Systems (IRC-ISS) at King Fahd University of Petroleum & Minerals

Cite

APA

Redah, M., & Al-Khatib, W. G. (2026). Autoencoders in Natural Language Processing: A Comprehensive Review. Computers, 15(4), 232. https://doi.org/10.3390/computers15040232

Vancouver

Redah M, Al-Khatib WG. Autoencoders in Natural Language Processing: A Comprehensive Review. Computers. 2026;15(4):232. doi:10.3390/computers15040232

BibTeX
@article{Redah2026,
  author = {Redah, Moussa and Al-Khatib, Wasfi G.},
  title = {Autoencoders in Natural Language Processing: A Comprehensive Review},
  journal = {Computers},
  year = {2026},
  volume = {15},
  number = {4},
  pages = {232},
  doi = {10.3390/computers15040232}
}
Start for free →