Accuracy of Artificial Intelligence vs Professionally Translated Discharge Instructions
JAMA Network Open · 17.09.2025 · 8 (9) · e2532312
DOI: 10.1001/jamanetworkopen.2025.32312 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift JAMA Network Open, online erschienen am 17.09.2025. Verfasst von Melissa Martos und 10 weiteren Autor:innen. Beteiligte Einrichtung: University of Washington, Seattle. Hauptthemen laut MEDLINE: Artificial Intelligence, Patient Discharge, Translating. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 64 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NCI NIH HHS.
Abstract (Originalsprache)
Importance Patients using languages other than English are a group at risk of poor health outcomes and encounter barriers to access of translated written materials. Although artificial intelligence (AI) may offer an opportunity to improve access, few studies have evaluated the accuracy and safety of AI translation for clinical care under routine practice conditions.
Objective
To investigate the accuracy of AI translation compared with professional human translation of patient-specific issued pediatric inpatient discharge instructions. Design, Setting, and Participants This comparative effectiveness analysis compared translations by a neural machine translation model vs professional translators using patient-specific pediatric inpatient discharge instructions received by families between May 18, 2023, and May 18, 2024, at a single center academic pediatric hospital. Instructions were translated to Simplified Chinese, Somali, Spanish, and Vietnamese by professional translators and the Azure AI system and then broken into scoring sections. Two professional translators per language evaluated translations (blinded to source) on an established 5-point scale for fluency, adequacy, meaning, and error severity, with 1 indicating worst performance and 5 indicating best performance. Exposure AI vs professional translation. Main Outcome and Measure Quality of discharge instruction translation, including fluency, adequacy, meaning, and severity of errors.
Results
A total of 148 sections from 34 discharge instructions were analyzed. When considering all 4 languages together, average fluency, adequacy, and meaning were lower among AI compared with professional human translations. Among all tested languages, mean (SD) fluency for AI translations was 2.98 (1.12) compared with 3.90 (0.96) for professional translations (difference, 0.92; 95% CI, 0.83-1.01; P < .001), adequacy was 3.81 (1.14) compared with 4.56 (0.70) (difference, 0.74; 95% CI, 0.65-0.83; P < .001), meaning was 3.38 (1.15) compared with 4.28 (0.84) (difference, 0.90; 95% CI, 0.80-0.99; P < .001), and error severity was 3.53 (1.28) compared with 4.48 (0.88) (difference, 0.95; 95% CI, 0.85-1.06; P < .001). Compared with professional translations, the Spanish AI translations were noninferior in adequacy (difference, 0.08; 95% CI, −0.02 to 0.19) and error severity (difference, 0.03; 95% CI, −0.09 to 0.14) but inferior in fluency (difference, 0.38; 95% CI, 0.23-0.53) and just crossed the inferiority threshold in meaning (difference, 0.08; 95% CI, −0.04 to 0.20). The Chinese, Vietnamese, and Somali AI translations were inferior to the professional translations across all metrics, with the greatest differences for Somali. Conclusions and Relevance In this comparative effectiveness analysis of AI- vs professionally translated issued discharge instructions, AI-translated instructions performed similarly for Spanish but worse for other languages tested. Validation and clinical implementation of AI-based translation will require special attention to languages of lesser diffusion to prevent creating new inequities.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- University of Washington, Seattle.
- Seattle Children's Hospital, Seattle, Washington.
- Center for Child Health, Behavior, and Development, Seattle Children's Research Institute, Seattle, Washington.
Förderung
- NCI NIH HHS
Zitieren
APA
Martos, M., Fields, B., Finlayson, S. G., Hartell, N., Kim, T., Larimer, E., Lau, J. J., Lin, Y. H., Salaguinto, T., Tran, N., & Lion, K. C. (2025). Accuracy of Artificial Intelligence vs Professionally Translated Discharge Instructions. JAMA Network Open, 8(9), e2532312. https://doi.org/10.1001/jamanetworkopen.2025.32312
Vancouver
Martos M, Fields B, Finlayson SG, Hartell N, Kim T, Larimer E, et al. Accuracy of Artificial Intelligence vs Professionally Translated Discharge Instructions. JAMA Network Open. 2025;8(9):e2532312. doi:10.1001/jamanetworkopen.2025.32312
BibTeX
@article{Martos2025,
author = {Martos, Melissa and Fields, Blanca and Finlayson, Samuel G and Hartell, Nigel and Kim, Theresa and Larimer, Emily and Lau, Jason J and Lin, Yu-Hsiang and Salaguinto, Taylor and Tran, Nguyen and Lion, K Casey},
title = {Accuracy of Artificial Intelligence vs Professionally Translated Discharge Instructions},
journal = {JAMA Network Open},
year = {2025},
volume = {8},
number = {9},
pages = {e2532312},
doi = {10.1001/jamanetworkopen.2025.32312}
}
EUVIMED