EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Statistical machine translation for Indic languages

Sudhansu Bala Das, Divyajyoti Panda, Tapas Kumar Mishra, Bidyut Kr. Patra

Natural Language Processing · 2025-03 · 31 (2) · 328-345

DOI: 10.1017/nlp.2024.26 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Natural Language Processing, online erschienen am 03.06.2024. Verfasst von Sudhansu Bala Das und 3 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Statistical Machine Translation (SMT) systems use various probabilistic and statistical Natural Language Processing (NLP) methods to automatically translate from one language to another language while retaining the originality of the context. This paper aims to discuss the development of bilingual SMT models for translating English into fifteen low-resource Indic languages (ILs) and vice versa. The process to build the SMT model is described and explained using a workflow diagram. Samanantar and OPUS corpus are utilized for training, and Flores200 corpus is used for fine-tuning and testing purposes. The paper also highlights various preprocessing methods used to deal with corpus noise. The Moses open-source SMT toolkit is being investigated for the system’s development. The impact of distance-based reordering and Morpho-syntactic Descriptor Bidirectional Finite-State Encoder (msd-bidirectional-fe) reordering on ILs is compared in the paper. This paper provides a comparison of SMT models with Neural Machine Translation (NMT) for ILs. All the experiments assess the translation quality using standard metrics such as BiLingual Evaluation Understudy, Rank-based Intuitive Bilingual Evaluation Score, Translation Edit Rate, and Metric for Evaluation of Translation with Explicit Ordering. From the result, it is observed that msd-bidirectional-fe reordering performs better than the distance-based reordering model for ILs. It is also noticed that even though the IL-English and English-IL systems are trained using the same corpus, the former performs better for all the evaluation metrics. The comparison between SMT and NMT shows that across various languages, SMT performs better in some cases, while NMT outperforms in others.

Zitieren

APA

Das, S. B., Panda, D., Mishra, T. K., & Patra, B. K. (2025). Statistical machine translation for Indic languages. Natural Language Processing, 31(2), 328-345. https://doi.org/10.1017/nlp.2024.26

Vancouver

Das SB, Panda D, Mishra TK, Patra BK. Statistical machine translation for Indic languages. Natural Language Processing. 2025;31(2):328-345. doi:10.1017/nlp.2024.26

BibTeX
@article{Das2025,
  author = {Das, Sudhansu Bala and Panda, Divyajyoti and Mishra, Tapas Kumar and Patra, Bidyut Kr.},
  title = {Statistical machine translation for Indic languages},
  journal = {Natural Language Processing},
  year = {2025},
  volume = {31},
  number = {2},
  pages = {328--345},
  doi = {10.1017/nlp.2024.26}
}
Kostenlos starten →