An exploratory case study on translating performance: ChatGPT vs. NMT vs. Human Translators
Journal of Modern Languages · 30.06.2026 · 36 (1) · 146-170
DOI: 10.22452/jml.vol36no1.8 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Modern Languages, online erschienen am 30.06.2026. Verfasst von Kayo Tsuji und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
This exploratory case study compares the translation output of a Japanese-language academic paper to English from four sources: the original author, Large Language Model (LLM, namely ChatGPT), Neural Machine Translation (NMT) and a professional, focusing on the differences in the output of ChatGPT and NMT. The recent development of these systems as a means of generating human-like translations has received increased scholarly attention comparing their strengths and weaknesses. However, there is a lack of research with respect to their performance with the Japanese-English language pair in the context of academic research publications. Two researchers in the field of Applied Linguistics were provided four samples to quantitatively rate in the categories of Accuracy, Fluency, Terminology and Style, and gave justifications for each score. The results showed that the author’s own translation rated highest, followed by NMT, then ChatGPT and, finally, the professional translation. Raters felt the author’s translation captured the intent behind the original text whilst the latter samples took a direct approach that more strictly adhered to the structure and literal meaning of the original text, which hurt their style and terminology but gave them relatively high accuracy scores. Moreover, raters considered the ChatGPT and NMT samples ill-fitting for the intended context: an academic publication. Regarding ChatGPT and NMT, specifically, both raters considered ChatGPT to provide a stricter, word-by-word translation of the text than that generated by NMT. However, given the influence of prompt design in ChatGPT’s output, it may be possible to provide a more successful translation with a revised input.
Zitieren
APA
Tsuji, K., Smith, B. N., & Oshima, H. (2026). An exploratory case study on translating performance: ChatGPT vs. NMT vs. Human Translators. Journal of Modern Languages, 36(1), 146-170. https://doi.org/10.22452/jml.vol36no1.8
Vancouver
Tsuji K, Smith BN, Oshima H. An exploratory case study on translating performance: ChatGPT vs. NMT vs. Human Translators. Journal of Modern Languages. 2026;36(1):146-170. doi:10.22452/jml.vol36no1.8
BibTeX
@article{Tsuji2026,
author = {Tsuji, Kayo and Smith, Benjamin Neil and Oshima, Hideki},
title = {An exploratory case study on translating performance: ChatGPT vs. NMT vs. Human Translators},
journal = {Journal of Modern Languages},
year = {2026},
volume = {36},
number = {1},
pages = {146--170},
doi = {10.22452/jml.vol36no1.8}
}
EUVIMED