Bridging AI and Medical Expertise: ChatGPT’s Success on the Medical Specialization Residency Admission Exam in Spain
Studies in Health Technology and Informatics · 2025
DOI: 10.3233/shti250544 ↗Kurz gefasst
Fachartikel in der Zeitschrift Studies in Health Technology and Informatics, 2025. Verfasst von Angela Leis und 2 weiteren Autor:innen. Beteiligte Einrichtung: Hospital del Mar Research Institute, Barcelona, Spain. Hauptthemen laut MEDLINE: Artificial Intelligence, Internship and Residency, Educational Measurement, Specialization. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
The growing use of Artificial Intelligence (AI) in healthcare, particularly focusing on the potential of generative AI models like ChatGPT-4 is a trending topic. The study examines how ChatGPT-4 performed on the national Medicine Residency exam in Spain, a highly selective test for accessing the medical specialization training program called MIR. ChatGPT-4 answered 210 questions, including 25 that required image interpretation. The chatbot correctly answered 150 out of 200 questions, achieving an estimated ranking of around 1900-2300 out of 11,577 candidates. This performance would allow access to most medical specialties in Spain. No significant differences were found between questions requiring image analysis and those that did not, but ChatGPT struggled with more difficult questions, showing a higher error rate for complex problems just like a human being. Despite its potential as an educational and problem-solving tool, the study highlights ChatGPT’s limitations, including occasional “AI hallucinations” (incorrect or nonsensical answers) and variability in responses when questions were repeated. The study emphasizes that while AI tools such as ChatGPT can assist in education and medical tasks, they cannot replace qualified healthcare professionals, and their output requires careful verification.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Hospital del Mar Research Institute, Barcelona, Spain.
- Hospital del Mar, Barcelona, Spain.
- Hospital Parc Taulí, Sabadell, Spain.
Zitieren
APA
Leis, A., Mayer, M. A., & Mayer, A. (2025). Bridging AI and Medical Expertise: ChatGPT’s Success on the Medical Specialization Residency Admission Exam in Spain. Studies in Health Technology and Informatics. https://doi.org/10.3233/shti250544
Vancouver
Leis A, Mayer MA, Mayer A. Bridging AI and Medical Expertise: ChatGPT’s Success on the Medical Specialization Residency Admission Exam in Spain. Studies in Health Technology and Informatics. 2025. doi:10.3233/shti250544
BibTeX
@article{Leis2025,
author = {Leis, Angela and Mayer, Miguel-Angel and Mayer, Alex},
title = {Bridging AI and Medical Expertise: ChatGPT’s Success on the Medical Specialization Residency Admission Exam in Spain},
journal = {Studies in Health Technology and Informatics},
year = {2025},
doi = {10.3233/shti250544}
}
EUVIMED