EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Dependency-aware self-attention for robust neural machine translation

Chuncheng Chi iD, Fuxue Li, Yichen Liu, Peijun Xie, Hong Yan iD

PLOS One · 12.02.2026 · 21 (2) · e0342772

DOI: 10.1371/journal.pone.0342772 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 12.02.2026. Verfasst von Chuncheng Chi und 4 weiteren Autor:innen. Beteiligte Einrichtung: College of Electrical Engineering, Yingkou Institute of Technology, Yingkou, China. Hauptthemen laut MEDLINE: Neural Networks, Computer, Attention, Translating. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Science Foundation of Liaoning Province, Foundation of Liaoning Province Education Administration, Project of Yingkou Institute of Technology.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Neural machine translation (NMT) has significantly benefited from integrating various forms of contextual information. However, conventional Transformer-based translation models primarily rely on self-attention mechanisms that are inherently position-invariant, making them inadequate for effectively capturing explicit syntactic dependencies, especially in low-resource scenarios or morphologically rich languages. To address this limitation, we propose a Dependency-Aware Self-Attention (DASA) mechanism that explicitly incorporates syntactic dependency structures into the attention computation. Our method first leverages a dependency parser to derive syntactic trees from source sentences, generating a dependency distance matrix representing pairwise syntactic proximity. This matrix is transformed into a normalized syntactic bias, which is seamlessly integrated into the attention mechanism through element-wise modulation of attention logits. By doing so, DASA guides attention weights towards syntactically relevant tokens, enhancing the Transformer encoder’s structural awareness and representation quality. Experimental results demonstrate that our approach substantially improves the translation performance, particularly in settings with limited training data. Experiments show that DASA enhances syntactic awareness and robustness, especially under data scarcity.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Neural Networks, ComputerAttentionTranslatingHumansLanguage

Autor:innen und Institutionen

  • College of Electrical Engineering, Yingkou Institute of Technology, Yingkou, China.
  • Yingkou Automation Industrial Control Technology Innovation Center, Yingkou Institute of Technology, Yingkou, China.
  • School of Computer Science and Software Engineering, University of Science and Technology Liaoning, Anshan, China.

ORCID-Profile: Chuncheng Chi, Hong Yan

Förderung

  • Natural Science Foundation of Liaoning Province
  • Foundation of Liaoning Province Education Administration
  • Project of Yingkou Institute of Technology
  • Foundation of Yingkou Automation Industrial Control Professional Technology Innovation Center

Zitieren

APA

Chi, C., Li, F., Liu, Y., Xie, P., & Yan, H. (2026). Dependency-aware self-attention for robust neural machine translation. PLOS One, 21(2), e0342772. https://doi.org/10.1371/journal.pone.0342772

Vancouver

Chi C, Li F, Liu Y, Xie P, Yan H. Dependency-aware self-attention for robust neural machine translation. PLOS One. 2026;21(2):e0342772. doi:10.1371/journal.pone.0342772

BibTeX
@article{Chi2026,
  author = {Chi, Chuncheng and Li, Fuxue and Liu, Yichen and Xie, Peijun and Yan, Hong},
  title = {Dependency-aware self-attention for robust neural machine translation},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {2},
  pages = {e0342772},
  doi = {10.1371/journal.pone.0342772}
}
Kostenlos starten →