EUVIMEDEUVIMED entdecken
Crossref

A DQF-MQM Evaluation of Machine Translation in English–Arabic Customs Law

Karam Damseh, Mozhgan Ghassemiazghandi

Journal of Language Teaching and Research · 01.05.2026 · 17 (3) · 1019-1027

DOI: 10.17507/jltr.1703.26 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Language Teaching and Research, online erschienen am 01.05.2026. Verfasst von Karam Damseh und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study presents a corpus-based diagnostic evaluation of Google Translate’s performance in translating the Revised Kyoto Convention (RKC) from English into Arabic. Building on the "Adequacy-Fluency Tradeoff" observed in recent literature, researchers employ an explanatory sequential triangulation design to assess whether high-resource NMT models can satisfy the strict compliance requirements of Customs law. The methodology applies the COMET metric to a stratified test suite of the RKC corpus (N = 1,000 segments) to establish a quantitative baseline. To diagnose potential metric discordance, researchers conducted a targeted human evaluation: a Likert-scale assessment of the highest-scoring segments (n = 100) to verify the "performance ceiling," followed by a granular DQF-MQM error analysis of the lowest-scoring segments (n = 200) conducted by a panel of Experts. The analysis identifies a profound discordance within the RKC corpus: while the system achieved a high mean COMET score of 87.70, the detailed analysis of the "performance floor" revealed 1,579 discrete errors. The resulting penalty score (1,292.5) exceeded the professional acceptance threshold for this text type by a factor of 5.7. These findings indicate that, within this specific legal corpus, Google Translate exhibits "specification-blindness," systematically failing to disambiguate polysemous "Terms of Art" required for international trade compliance. The research concludes that for the RKC, automated metrics do not yet serve as a reliable proxy for legal review, and that a Human-in-the-Loop framework remains an absolute prerequisite for the deployment of Google Translate in Customs administration.

Zitieren

APA

Damseh, K., & Ghassemiazghandi, M. (2026). A DQF-MQM Evaluation of Machine Translation in English–Arabic Customs Law. Journal of Language Teaching and Research, 17(3), 1019-1027. https://doi.org/10.17507/jltr.1703.26

Vancouver

Damseh K, Ghassemiazghandi M. A DQF-MQM Evaluation of Machine Translation in English–Arabic Customs Law. Journal of Language Teaching and Research. 2026;17(3):1019-1027. doi:10.17507/jltr.1703.26

BibTeX
@article{Damseh2026,
  author = {Damseh, Karam and Ghassemiazghandi, Mozhgan},
  title = {A DQF-MQM Evaluation of Machine Translation in English–Arabic Customs Law},
  journal = {Journal of Language Teaching and Research},
  year = {2026},
  volume = {17},
  number = {3},
  pages = {1019--1027},
  doi = {10.17507/jltr.1703.26}
}
Kostenlos starten →