Dependency-aware self-attention for robust neural machine translation
PLOS One · 12.02.2026 · 21 (2) · e0342772
DOI: 10.1371/journal.pone.0342772 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, online erschienen am 12.02.2026. Verfasst von Chuncheng Chi und 4 weiteren Autor:innen. Beteiligte Einrichtung: College of Electrical Engineering, Yingkou Institute of Technology, Yingkou, China. Hauptthemen laut MEDLINE: Neural Networks, Computer, Attention, Translating. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Liaoning Province, Foundation of Liaoning Province Education Administration, Project of Yingkou Institute of Technology.
Abstract (Originalsprache)
Neural machine translation (NMT) has significantly benefited from integrating various forms of contextual information. However, conventional Transformer-based translation models primarily rely on self-attention mechanisms that are inherently position-invariant, making them inadequate for effectively capturing explicit syntactic dependencies, especially in low-resource scenarios or morphologically rich languages. To address this limitation, we propose a Dependency-Aware Self-Attention (DASA) mechanism that explicitly incorporates syntactic dependency structures into the attention computation. Our method first leverages a dependency parser to derive syntactic trees from source sentences, generating a dependency distance matrix representing pairwise syntactic proximity. This matrix is transformed into a normalized syntactic bias, which is seamlessly integrated into the attention mechanism through element-wise modulation of attention logits. By doing so, DASA guides attention weights towards syntactically relevant tokens, enhancing the Transformer encoder’s structural awareness and representation quality. Experimental results demonstrate that our approach substantially improves the translation performance, particularly in settings with limited training data. Experiments show that DASA enhances syntactic awareness and robustness, especially under data scarcity.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- College of Electrical Engineering, Yingkou Institute of Technology, Yingkou, China.
- Yingkou Automation Industrial Control Technology Innovation Center, Yingkou Institute of Technology, Yingkou, China.
- School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan, China.
ORCID-Profile: Chuncheng Chi, Hong Yan
Förderung
- Natural Science Foundation of Liaoning Province
- Foundation of Liaoning Province Education Administration
- Project of Yingkou Institute of Technology
- Foundation of Yingkou Automation Industrial Control Professional Technology Innovation Center
Zitieren
APA
Chi, C., Li, F., Liu, Y., Xie, P., & Yan, H. (2026). Dependency-aware self-attention for robust neural machine translation. PLOS One, 21(2), e0342772. https://doi.org/10.1371/journal.pone.0342772
Vancouver
Chi C, Li F, Liu Y, Xie P, Yan H. Dependency-aware self-attention for robust neural machine translation. PLOS One. 2026;21(2):e0342772. doi:10.1371/journal.pone.0342772
BibTeX
@article{Chi2026,
author = {Chi, Chuncheng and Li, Fuxue and Liu, Yichen and Xie, Peijun and Yan, Hong},
title = {Dependency-aware self-attention for robust neural machine translation},
journal = {PLOS One},
year = {2026},
volume = {21},
number = {2},
pages = {e0342772},
doi = {10.1371/journal.pone.0342772}
}
EUVIMED