EUVIMEDEUVIMED entdecken
Randomisierte kontrollierte StudieOpen Access

Integrating ChatGPT in Orthopedic Education for Medical Undergraduates: Randomized Controlled Trial

Integration von ChatGPT in die orthopädische Ausbildung für Medizinstudierende: randomisierte kontrollierte Studie

Wenyi Gan iD, Jianfeng Ouyang iD, Hua Li iD, Zhaowen Xue iD, Yiming Zhang iD, Qiu Dong iD, Jiadong Huang iD, Xiaofei Zheng iD, Yiyi Zhang iD

Journal of Medical Internet Research · 20.08.2024 · 26 · e57037

DOI: 10.2196/57037 ↗

Auf einen Blick

In einer randomisierten Studie mit 129 Medizinstudierenden war ChatGPT als Lernhilfe mit besseren kurzfristigen Orthopädie-Testergebnissen verbunden. Auch in 2 Fächern lagen die Semesterendnoten höher.

Fragestellung

Wie genau beantwortet ChatGPT orthopädische MCQs und welche kurz- und langfristigen Effekte hat es als Lernhilfe bei Medizinstudierenden?

Methode

Zunächst wurde die Genauigkeit von ChatGPT bei orthopädischen MCQs geprüft. Danach erhielten 129 Studierende 2 Wochen lang ChatGPT oder kein KI-Tool; anschließende Orthopädie-Tests und Semesterendnoten wurden verglichen.

Ergebnisse

ChatGPT-4.0 beantwortete 1051 orthopädische MCQs mit 70,60 % (742/1051) richtig. In der Studie schnitt die ChatGPT-Gruppe im Orthopädie-Test besser ab (141,20 vs. 130,80; P=.04) sowie in Surgery und Obstetrics and Gynecology am Semesterende.

Schlussfolgerung der Autor:innen

ChatGPT beantwortet orthopädische MCQs genau, und Studierende mit ChatGPT erzielten bessere kurz- und langfristige Leistungen.

Kernaussagen

  • 70,60 % Trefferquote von ChatGPT-4.0 bei 1051 orthopädischen MCQs
  • Besserer Orthopädie-Test in der ChatGPT-Gruppe (141,20 vs. 130,80; P=.04)
  • Höhere Endnoten in Surgery und Obstetrics and Gynecology

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Randomisierte kontrollierte Studie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 20.08.2024. Verfasst von Wenyi Gan und 8 weiteren Autor:innen. Beteiligte Einrichtung: The First Clinical Medical College of Jinan University, The First Affiliated Hospital of Jinan University, Guangzhou, China. Hauptthemen laut MEDLINE: Education, Medical, Undergraduate, Orthopedics, Generative Artificial Intelligence. Die Arbeit wurde bisher 78-mal zitiert (Crossref) und nennt 56 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

ChatGPT is a natural language processing model developed by OpenAI, which can be iteratively updated and optimized to accommodate the changing and complex requirements of human verbal communication.

Objective

The study aimed to evaluate ChatGPT’s accuracy in answering orthopedics-related multiple-choice questions (MCQs) and assess its short-term effects as a learning aid through a randomized controlled trial. In addition, long-term effects on student performance in other subjects were measured using final examination results.

Methods

We first evaluated ChatGPT’s accuracy in answering MCQs pertaining to orthopedics across various question formats. Then, 129 undergraduate medical students participated in a randomized controlled study in which the ChatGPT group used ChatGPT as a learning tool, while the control group was prohibited from using artificial intelligence software to support learning. Following a 2-week intervention, the 2 groups’ understanding of orthopedics was assessed by an orthopedics test, and variations in the 2 groups’ performance in other disciplines were noted through a follow-up at the end of the semester.

Results

ChatGPT-4.0 answered 1051 orthopedics-related MCQs with a 70.60% (742/1051) accuracy rate, including 71.8% (237/330) accuracy for A1 MCQs, 73.7% (330/448) accuracy for A2 MCQs, 70.2% (92/131) accuracy for A3/4 MCQs, and 58.5% (83/142) accuracy for case analysis MCQs. As of April 7, 2023, a total of 129 individuals participated in the experiment. However, 19 individuals withdrew from the experiment at various phases; thus, as of July 1, 2023, a total of 110 individuals accomplished the trial and completed all follow-up work. After we intervened in the learning style of the students in the short term, the ChatGPT group answered more questions correctly than the control group (ChatGPT group: mean 141.20, SD 26.68; control group: mean 130.80, SD 25.56; P=.04) in the orthopedics test, particularly on A1 (ChatGPT group: mean 46.57, SD 8.52; control group: mean 42.18, SD 9.43; P=.01), A2 (ChatGPT group: mean 60.59, SD 10.58; control group: mean 56.66, SD 9.91; P=.047), and A3/4 MCQs (ChatGPT group: mean 19.57, SD 5.48; control group: mean 16.46, SD 4.58; P=.002). At the end of the semester, we found that the ChatGPT group performed better on final examinations in surgery (ChatGPT group: mean 76.54, SD 9.79; control group: mean 72.54, SD 8.11; P=.02) and obstetrics and gynecology (ChatGPT group: mean 75.98, SD 8.94; control group: mean 72.54, SD 8.66; P=.04) than the control group.

Conclusions

ChatGPT answers orthopedics-related MCQs accurately, and students using it excel in both short-term and long-term assessments. Our findings strongly support ChatGPT’s integration into medical education, enhancing contemporary instructional methods. Trial Registration Chinese Clinical Trial Registry Chictr2300071774; https://www.chictr.org.cn/hvshowproject.html ?id=225740&v=1.0

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Education, Medical, UndergraduateOrthopedicsGenerative Artificial IntelligenceFemaleHumansMaleYoung AdultEducational MeasurementNatural Language ProcessingStudents, Medical

Stichwörter der Autor:innen

ChatGPTartificial intelligencelarge language modellearning aidmedical educationnatural language processingorthopedicsrandomized controlled trial

Autor:innen und Institutionen

  • The First Clinical Medical College of Jinan University, The First Affiliated Hospital of Jinan University, Guangzhou, China.
  • Department of Joint Surgery and Sports Medicine, Zhuhai People's Hospital (Zhuhai Hospital Affiliated With Jinan University), Zhuhai, Guangdong, China.
  • Department of Orthopaedics, Beijing Jishuitan Hospital, Beijing, China.
  • Jinan University-University of Birmingham Joint Institute, Jinan University, Guangzhou, China.

ORCID-Profile: Wenyi Gan, Jianfeng Ouyang, Hua Li, Zhaowen Xue, Yiming Zhang, Qiu Dong, Jiadong Huang, Xiaofei Zheng, Yiyi Zhang

Zitieren

APA

Gan, W., Ouyang, J., Li, H., Xue, Z., Zhang, Y., Dong, Q., Huang, J., Zheng, X., & Zhang, Y. (2024). Integrating ChatGPT in Orthopedic Education for Medical Undergraduates: Randomized Controlled Trial. Journal of Medical Internet Research, 26, e57037. https://doi.org/10.2196/57037

Vancouver

Gan W, Ouyang J, Li H, Xue Z, Zhang Y, Dong Q, et al. Integrating ChatGPT in Orthopedic Education for Medical Undergraduates: Randomized Controlled Trial. Journal of Medical Internet Research. 2024;26:e57037. doi:10.2196/57037

BibTeX
@article{Gan2024,
  author = {Gan, Wenyi and Ouyang, Jianfeng and Li, Hua and Xue, Zhaowen and Zhang, Yiming and Dong, Qiu and Huang, Jiadong and Zheng, Xiaofei and Zhang, Yiyi},
  title = {Integrating ChatGPT in Orthopedic Education for Medical Undergraduates: Randomized Controlled Trial},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  volume = {26},
  pages = {e57037},
  doi = {10.2196/57037}
}
Kostenlos starten →