Prompt engineering to increase GPT3.5’s performance on the Plastic Surgery In-Service Exams
Journal of Plastic, Reconstructive & Aesthetic Surgery · 2024
DOI: 10.1016/j.bjps.2024.09.001 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Plastic, Reconstructive & Aesthetic Surgery, 2024. Verfasst von George R Nahass und 9 weiteren Autor:innen. Beteiligte Einrichtung: The Craniofacial Center, Division of Plastic, Reconstructive, and Cosmetic Surgery, University of Illinois at Chicago, Chicago, IL 60612, USA; Richar…. Hauptthemen laut MEDLINE: Surgery, Plastic, Educational Measurement, Clinical Competence. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 5 Literaturangaben.
Abstract (Originalsprache)
This study assesses ChatGPT's (GPT-3.5) performance on the 2021 ASPS Plastic Surgery In-Service Examination using prompt modifications and Retrieval Augmented Generation (RAG). ChatGPT was instructed to act as a "resident," "attending," or "medical student," and RAG utilized a curated vector database for context. Results showed no significant improvement, with the "resident" prompt yielding the highest accuracy at 54%, and RAG failing to enhance performance, with accuracy remaining at 54.3%. Despite appropriate reasoning when correct, ChatGPT's overall performance fell in the 10th percentile, indicating the need for fine-tuning and more sophisticated approaches to improve AI's utility in complex medical tasks.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- The Craniofacial Center, Division of Plastic, Reconstructive, and Cosmetic Surgery, University of Illinois at Chicago, Chicago, IL 60612, USA; Richard and Loan Hill Department of Biomedical Engineering, University of Il…
- The Craniofacial Center, Division of Plastic, Reconstructive, and Cosmetic Surgery, University of Illinois at Chicago, Chicago, IL 60612, USA.
- The Craniofacial Center, Division of Plastic, Reconstructive, and Cosmetic Surgery, University of Illinois at Chicago, Chicago, IL 60612, USA; Shriners Children's Chicago Hospital, Chicago, IL 60707, USA.
- The Craniofacial Center, Division of Plastic, Reconstructive, and Cosmetic Surgery, University of Illinois at Chicago, Chicago, IL 60612, USA; Shriners Children's Chicago Hospital, Chicago, IL 60707, USA. Electronic add…
Zitieren
APA
Nahass, G. R., Chin, S. W., Scharf, I. M., Kazmouz, S., Kaplan, N., Chiu, R., Yang, K., Bou Zeid, N., Corcoran, J., & Alkureishi, L. W. T. (2024). Prompt engineering to increase GPT3.5’s performance on the Plastic Surgery In-Service Exams. Journal of Plastic, Reconstructive & Aesthetic Surgery. https://doi.org/10.1016/j.bjps.2024.09.001
Vancouver
Nahass GR, Chin SW, Scharf IM, Kazmouz S, Kaplan N, Chiu R, et al. Prompt engineering to increase GPT3.5’s performance on the Plastic Surgery In-Service Exams. Journal of Plastic, Reconstructive & Aesthetic Surgery. 2024. doi:10.1016/j.bjps.2024.09.001
BibTeX
@article{Nahass2024,
author = {Nahass, George R and Chin, Sydney W and Scharf, Isabel M and Kazmouz, Sobhi and Kaplan, Nicolas and Chiu, Richard and Yang, Kevin and Bou Zeid, Naji and Corcoran, Julia and Alkureishi, Lee W T},
title = {Prompt engineering to increase GPT3.5’s performance on the Plastic Surgery In-Service Exams},
journal = {Journal of Plastic, Reconstructive & Aesthetic Surgery},
year = {2024},
doi = {10.1016/j.bjps.2024.09.001}
}
EUVIMED