ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial
JMIR Research Protocols · 2025
DOI: 10.2196/63887 ↗Kurz gefasst
Randomisierte kontrollierte Studie in der Zeitschrift JMIR Research Protocols, 2025. Verfasst von Christian Burisch und 6 weiteren Autor:innen. Beteiligte Einrichtung: State of North Rhine-Westphalia, Regional Government Düsseldorf, Leibniz-Gymnasium, Essen, Germany. Hauptthemen laut MEDLINE: Education, Medical, Continuing, Artificial Intelligence. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 26 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
The increasing development and spread of artificial and assistive intelligence is opening up new areas of application not only in applied medicine but also in related fields such as continuing medical education (CME), which is part of the mandatory training program for medical doctors in Germany. This study aimed to determine whether medical laypersons can successfully conduct training courses specifically for physicians with the help of a large language model (LLM) such as ChatGPT-4. This study aims to qualitatively and quantitatively investigate the impact of using artificial intelligence (AI; specifically ChatGPT) on the acquisition of credit points in German postgraduate medical education.
Objective
Using this approach, we wanted to test further possible applications of AI in the postgraduate medical education setting and obtain results for practical use. Depending on the results, the potential influence of LLMs such as ChatGPT-4 on CME will be discussed, for example, as part of a SWOT (strengths, weaknesses, opportunities, threats) analysis.
Methods
We designed a randomized controlled trial, in which adult high school students attempt to solve CME tests across six medical specialties in three study arms in total with 18 CME training courses per study arm under different interventional conditions with varying amounts of permitted use of ChatGPT-4. Sample size calculation was performed including guess probability (20% correct answers, SD=40%; confidence level of 1–α=.95/α=.05; test power of 1–β=.95; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- State of North Rhine-Westphalia, Regional Government Düsseldorf, Leibniz-Gymnasium, Essen, Germany.
- Department of Didactics and Education Research in the Health Sector, Faculty of Health, Witten/Herdecke University, Witten, Germany.
- Faculty of Health, Witten/Herdecke University, Witten, Germany.
- Department of Cardiology, Pneumology, Neurology and Intensive Care Medicine, Klinik Kitzinger Land, Kitzingen, Germany.
- Department of Cardiology and Vascular Medicine, West German Heart and Vascular Center Essen, University Duisburg-Essen, Essen, Germany.
- Department of Anesthesiology, HELIOS University Hospital, Wuppertal, Germany.
ORCID-Profile: Christian Burisch, Abhav Bellary, Frank Breuckmann, Jan Ehlers, Serge C Thal, Timur Sellmann, Daniel Gödde
Zitieren
APA
Burisch, C., Bellary, A., Breuckmann, F., Ehlers, J., Thal, S. C., Sellmann, T., & Gödde, D. (2025). ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial. JMIR Research Protocols. https://doi.org/10.2196/63887
Vancouver
Burisch C, Bellary A, Breuckmann F, Ehlers J, Thal SC, Sellmann T, et al. ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial. JMIR Research Protocols. 2025. doi:10.2196/63887
BibTeX
@article{Burisch2025,
author = {Burisch, Christian and Bellary, Abhav and Breuckmann, Frank and Ehlers, Jan and Thal, Serge C and Sellmann, Timur and Gödde, Daniel},
title = {ChatGPT-4 Performance on German Continuing Medical Education—Friend or Foe (Trick or Treat)? Protocol for a Randomized Controlled Trial},
journal = {JMIR Research Protocols},
year = {2025},
doi = {10.2196/63887}
}
EUVIMED