Evaluation of ChatGPT's performance in providing treatment recommendations for pediatric diseases
Pediatric Discovery · 2023-12 · 1 (3)
DOI: 10.1002/pdi3.42 ↗Kurz gefasst
Fachartikel in der Zeitschrift Pediatric Discovery, online erschienen am 20.11.2023. Verfasst von Qiuhong Wei und 6 weiteren Autor:innen. Beteiligte Einrichtung: Children Nutrition Research Center Children's Hospital of Chongqing Medical University National Clinical Research Center for Child Health and Disorde…. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
With the advance of artificial intelligence technology, large language models such as ChatGPT are drawing substantial interest in the healthcare field. A growing body of research has evaluated ChatGPT's performance in various medical departments, yet its potential in pediatrics remains under‐studied. In this study, we presented ChatGPT with a total of 4160 clinical consultation questions in both English and Chinese, covering 104 pediatric conditions, and repeated each question independently 10 times to assess the accuracy of its responses in pediatric disease treatment recommendations. ChatGPT achieved an overall accuracy of 82.2% (95% CI: 81.0%–83.4%), with superior performance in addressing common diseases (84.4%, 95% CI: 83.2%–85.7%), offering general treatment advice (83.5%, 95% CI: 81.9%–85.1%), and responding in English (93.0%, 95% CI: 91.9%–94.1%). However, it was prone to errors in disease definitions, medications, and surgical treatment. In conclusion, while ChatGPT shows promise in pediatric treatment recommendations with notable accuracy, cautious optimism is warranted regarding the potential application of large language models in enhancing patient care.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Children Nutrition Research Center Children's Hospital of Chongqing Medical University National Clinical Research Center for Child Health and Disorders Ministry of Education Key Laboratory of Child Development and Disor…
- College of Medical Informatics Medical Data Science Academy Chongqing Engineering Research Center for Clinical Big-Data and Drug Evaluation Chongqing Medical University Chongqing China.
- Department of Nephrology Children's Hospital of Chongqing Medical University Chongqing China.
- Department of Neurology Children's Hospital of Chongqing Medical University Chongqing China.
- Department of Biomedical Data Science Stanford University School of Medicine Stanford California USA.
- Department of Global Statistics and Data Science BeiGene USA Inc. San Mateo California USA.
ORCID-Profile: Qiuhong Wei
Zitieren
APA
Wei, Q., Wang, Y., Yao, Z., Cui, Y., Wei, B., Li, T., & Xu, X. (2023). Evaluation of ChatGPT's performance in providing treatment recommendations for pediatric diseases. Pediatric Discovery, 1(3). https://doi.org/10.1002/pdi3.42
Vancouver
Wei Q, Wang Y, Yao Z, Cui Y, Wei B, Li T, et al. Evaluation of ChatGPT's performance in providing treatment recommendations for pediatric diseases. Pediatric Discovery. 2023;1(3). doi:10.1002/pdi3.42
BibTeX
@article{Wei2023,
author = {Wei, Qiuhong and Wang, Yanqin and Yao, Zhengxiong and Cui, Ying and Wei, Bo and Li, Tingyu and Xu, Ximing},
title = {Evaluation of ChatGPT's performance in providing treatment recommendations for pediatric diseases},
journal = {Pediatric Discovery},
year = {2023},
volume = {1},
number = {3},
doi = {10.1002/pdi3.42}
}
EUVIMED