EUVIMEDEUVIMED entdecken
Vergleichsstudie

Comparative Evaluation of Machine Translation Accuracy of Emergency Department Discharge Instructions: A Non‐Inferiority Study

Giovanni Rodriguez, Patricia Hernández, Christopher Kirwan, Lisette Dunham, Sayon Dutta

Academic Emergency Medicine · 2026-04 · 33 (4)

DOI: 10.1111/acem.70289 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Academic Emergency Medicine, online erschienen am 16.04.2026. Verfasst von Giovanni Rodriguez und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Emergency Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA. Hauptthemen laut MEDLINE: Emergency Service, Hospital, Limited English Proficiency. Die Arbeit nennt 43 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Patients with limited English proficiency (LEP) face disproportionate risks at emergency department (ED) discharge. Professional interpretation improves outcomes, but real‐time written translations remain difficult to provide in many EDs. Modern transformer‐based large language models (LLMs) may offer improved translation quality compared with older systems, yet their performance on ad hoc provider‐written ED discharge instructions is not well established.

Methods

We conducted a blinded cross‐sectional non‐inferiority study of English‐language ED discharge instructions translated into Spanish, Brazilian Portuguese, and Simplified Chinese comparing Google Translate and ChatGPT‐4o versus professional medical interpreters. Fifty‐three randomly selected provider‐written instructions (100–500 words, preserving spelling/grammar errors) were translated, yielding 477 unique translations. Professional medical interpreters, blinded to translation method, independently scored each translation on fluency, adequacy, meaning, and severity on a five‐point Likert scale. Inter‐rater reliability between the professional interpreter evaluations was calculated. A 0.5‐point non‐inferiority margin was pre‐specified, and adjusted mean Likert rating differences generated by mixed effects models for each accuracy dimension were compared between translation methods for each language. The proportion of clinically significant translation errors was compared between methods, as was the ability of evaluators to guess the translation method.

Results

Inter‐rater reliability was high across languages. Both machine translation methods were non‐inferior to professional interpreters for adequacy, meaning, and severity in Spanish and Portuguese, and for all four domains in Chinese. For fluency, Google Translate and ChatGPT‐4o were inferior in Spanish and Portuguese but non‐inferior in Chinese. The frequency of clinically significant errors did not differ significantly by translation method. Evaluators, blinded to method, frequently misidentified machine translations as professional.

Conclusions

In this multi‐language evaluation of real‐world ED discharge instructions, Google Translate and ChatGPT‐4o were non‐inferior to professional interpreters for most domains of translation accuracy.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Emergency Service, HospitalLimited English ProficiencyHumansLanguagePatient DischargeTranslations

Autor:innen und Institutionen

  • Department of Emergency Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA.
  • Department of Emergency Medicine, Brigham and Women's Hospital, Boston, Massachusetts, USA.
  • Harvard Medical School, Boston, Massachusetts, USA.
  • The Disparities Solutions Center, Massachusetts General Hospital, Boston, Massachusetts, USA.
  • Mass General Brigham Digital, Boston, Massachusetts, USA.

Zitieren

APA

Rodriguez, G., Hernández, P., Kirwan, C., Dunham, L., & Dutta, S. (2026). Comparative Evaluation of Machine Translation Accuracy of Emergency Department Discharge Instructions: A Non‐Inferiority Study. Academic Emergency Medicine, 33(4). https://doi.org/10.1111/acem.70289

Vancouver

Rodriguez G, Hernández P, Kirwan C, Dunham L, Dutta S. Comparative Evaluation of Machine Translation Accuracy of Emergency Department Discharge Instructions: A Non‐Inferiority Study. Academic Emergency Medicine. 2026;33(4). doi:10.1111/acem.70289

BibTeX
@article{Rodriguez2026,
  author = {Rodriguez, Giovanni and Hernández, Patricia and Kirwan, Christopher and Dunham, Lisette and Dutta, Sayon},
  title = {Comparative Evaluation of Machine Translation Accuracy of Emergency Department Discharge Instructions: A Non‐Inferiority Study},
  journal = {Academic Emergency Medicine},
  year = {2026},
  volume = {33},
  number = {4},
  doi = {10.1111/acem.70289}
}
Kostenlos starten →