GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?
Journal of Integrative and Complementary Medicine · 02.07.2025
DOI: 10.1089/jicm.2024.0902 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Journal of Integrative and Complementary Medicine, online erschienen am 02.07.2025. Verfasst von Erfan Ghanad und 8 weiteren Autor:innen. Beteiligte Einrichtung: Medical Faculty Mannheim, Department of Surgery, University Medical Centre Mannheim, University of Heidelberg, Mannheim, Germany. Hauptthemen laut MEDLINE: Medicine, Chinese Traditional, Educational Measurement, Licensure, Medical. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 14 Literaturangaben.
Abstract (Originalsprache)
Purpose
This study evaluates the performance of large language models (LLMs) in the context of the Chinese National Traditional Chinese Medicine Licensing Examination (TCMLE). Materials and Methods: We compared the performances of different versions of Generative Pre-trained Transformer (GPT) and Enhanced Representation through Knowledge Integration (ERNIE) using historical TCMLE questions.
Results
ERNIE-4.0 outperformed all other models with an accuracy of 81.7%, followed by ERNIE-3.5 (75.2%), GPT-4o (74.8%), and GPT-4 turbo (50.7%). For questions related to Western internal medicine, all models showed high accuracy above 86.7%.
Conclusion
The study highlights the significance of cultural context in training data, influencing the performance of LLMs in specific medical examinations.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Medical Faculty Mannheim, Department of Surgery, University Medical Centre Mannheim, University of Heidelberg, Mannheim, Germany.
- Medical Faculty Mannheim, Department of Medical Statistics and Biomathematics, Heidelberg University, Mannheim, Germany.
- Department of Rheumatology and Immunology, Peking University International Hospital, Beijing, China.
- Department of Medical Rehabilitation, Zhaoqing First Hospital, Zhaoqing, China.
- College of Engineering, Boston University, Boston, MA, USA.
- Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education), Sarcoma Center, Peking University Cancer Hospital & Institute, Beijing, China.
ORCID-Profile: Erfan Ghanad, Chengpeng Li
Zitieren
APA
Ghanad, E., Weiß, C., Gao, H., Reißfelder, C., Hummedah, K., Han, L., Tong, L., Li, C., & Yang, C. (2025). GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?. Journal of Integrative and Complementary Medicine. https://doi.org/10.1089/jicm.2024.0902
Vancouver
Ghanad E, Weiß C, Gao H, Reißfelder C, Hummedah K, Han L, et al. GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?. Journal of Integrative and Complementary Medicine. 2025. doi:10.1089/jicm.2024.0902
BibTeX
@article{Ghanad2025,
author = {Ghanad, Erfan and Weiß, Christel and Gao, Hui and Reißfelder, Christoph and Hummedah, Kamal and Han, Lei and Tong, Leihui and Li, Chengpeng and Yang, Cui},
title = {GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?},
journal = {Journal of Integrative and Complementary Medicine},
year = {2025},
doi = {10.1089/jicm.2024.0902}
}
EUVIMED