EUVIMEDEUVIMED entdecken
Vergleichsstudie

GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?

Erfan Ghanad iD, Christel Weiß, Hui Gao, Christoph Reißfelder, Kamal Hummedah, Lei Han, Leihui Tong, Chengpeng Li iD, Cui Yang

Journal of Integrative and Complementary Medicine · 02.07.2025

DOI: 10.1089/jicm.2024.0902 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of Integrative and Complementary Medicine, online erschienen am 02.07.2025. Verfasst von Erfan Ghanad und 8 weiteren Autor:innen. Beteiligte Einrichtung: Medical Faculty Mannheim, Department of Surgery, University Medical Centre Mannheim, University of Heidelberg, Mannheim, Germany. Hauptthemen laut MEDLINE: Medicine, Chinese Traditional, Educational Measurement, Licensure, Medical. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 14 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

This study evaluates the performance of large language models (LLMs) in the context of the Chinese National Traditional Chinese Medicine Licensing Examination (TCMLE). Materials and Methods: We compared the performances of different versions of Generative Pre-trained Transformer (GPT) and Enhanced Representation through Knowledge Integration (ERNIE) using historical TCMLE questions.

Results

ERNIE-4.0 outperformed all other models with an accuracy of 81.7%, followed by ERNIE-3.5 (75.2%), GPT-4o (74.8%), and GPT-4 turbo (50.7%). For questions related to Western internal medicine, all models showed high accuracy above 86.7%.

Conclusion

The study highlights the significance of cultural context in training data, influencing the performance of LLMs in specific medical examinations.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Medicine, Chinese TraditionalEducational MeasurementLicensure, MedicalHumansLanguageChinaCulture

Stichwörter der Autor:innen

large language modelsmedical licensing examinationtraditional Chinese medicine

Autor:innen und Institutionen

  • Medical Faculty Mannheim, Department of Surgery, University Medical Centre Mannheim, University of Heidelberg, Mannheim, Germany.
  • Medical Faculty Mannheim, Department of Medical Statistics and Biomathematics, Heidelberg University, Mannheim, Germany.
  • Department of Rheumatology and Immunology, Peking University International Hospital, Beijing, China.
  • Department of Medical Rehabilitation, Zhaoqing First Hospital, Zhaoqing, China.
  • College of Engineering, Boston University, Boston, MA, USA.
  • Key Laboratory of Carcinogenesis and Translational Research (Ministry of Education), Sarcoma Center, Peking University Cancer Hospital & Institute, Beijing, China.

ORCID-Profile: Erfan Ghanad, Chengpeng Li

Zitieren

APA

Ghanad, E., Weiß, C., Gao, H., Reißfelder, C., Hummedah, K., Han, L., Tong, L., Li, C., & Yang, C. (2025). GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?. Journal of Integrative and Complementary Medicine. https://doi.org/10.1089/jicm.2024.0902

Vancouver

Ghanad E, Weiß C, Gao H, Reißfelder C, Hummedah K, Han L, et al. GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?. Journal of Integrative and Complementary Medicine. 2025. doi:10.1089/jicm.2024.0902

BibTeX
@article{Ghanad2025,
  author = {Ghanad, Erfan and Weiß, Christel and Gao, Hui and Reißfelder, Christoph and Hummedah, Kamal and Han, Lei and Tong, Leihui and Li, Chengpeng and Yang, Cui},
  title = {GPT Versus ERNIE for National Traditional Chinese Medicine Licensing Examination: Does Cultural Background Matter?},
  journal = {Journal of Integrative and Complementary Medicine},
  year = {2025},
  doi = {10.1089/jicm.2024.0902}
}
Kostenlos starten →