EUVIMEDEUVIMED entdecken
Vergleichsstudie

Evaluating AI Chatbot Performance Against Prosthodontic Residents on the National Prosthodontic Resident Examination (NPRE) from 2011 to 2023

Despoina Bompolaki, Esha Mukherjee, Kelsey A K Lee, Christopher Yoon, Hongseok An

The International Journal of Prosthodontics · 2025

DOI: 10.11607/ijp.9486 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift The International Journal of Prosthodontics, 2025. Verfasst von Despoina Bompolaki und 4 weiteren Autor:innen. Hauptthemen laut MEDLINE: Prosthodontics, Internship and Residency, Artificial Intelligence, Educational Measurement.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

PURPOSE: To compare the performance of seven artificial intelligence (AI) chatbots (ChatGPT-3.5, ChatGPT-4, ChatGPT-4o, Gemini, Claude 3, Copilot, and Llama) to that of prosthodontic residents in the USA on the National Prosthodontic Resident Examination (NPRE) from 2011 to 2023. MATERIALS AND METHODS: NPRE exam files and answer keys were obtained from the American College of Prosthodontists (ACP). AI-generated responses were collected using standardized prompts that requested only letter answer choices. Each chatbot's first response was recorded and compared to the official exam key. Performance was evaluated as the percentage of correct responses, and results were compared to the average scores of human participants. Statistical analysis included repeated measures ANOVA and Bonferroni post-hoc comparisons. RESULTS: Statistically significant differences were found between the different groups (P < .001). ChatGPT-3.5 achieved the highest scores, followed by Claude 3 and Llama, while Copilot performed significantly worse than residents. Other chatbots demonstrated performance levels comparable to human participants, but with notable variability across AI models. The findings suggest that AI chatbots, particularly ChatGPT-3.5, can perform at or above the level of prosthodontic residents on standardized examinations. However, differences in chatbot capabilities, training data, and inconsistencies in performance raise concerns about reliability. While AI has potential as a supplementary learning tool, its use should be guided by critical evaluation. CONCLUSIONS: AI chatbots demonstrate adequate performance on specialized academic exams, but variability across models highlights the need for further assessment before integration into dental education and professional practice.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ProsthodonticsInternship and ResidencyArtificial IntelligenceEducational MeasurementHumansUnited StatesGenerative Artificial Intelligence

Zitieren

APA

Bompolaki, D., Mukherjee, E., Lee, K. A. K., Yoon, C., & An, H. (2025). Evaluating AI Chatbot Performance Against Prosthodontic Residents on the National Prosthodontic Resident Examination (NPRE) from 2011 to 2023. The International Journal of Prosthodontics. https://doi.org/10.11607/ijp.9486

Vancouver

Bompolaki D, Mukherjee E, Lee KAK, Yoon C, An H. Evaluating AI Chatbot Performance Against Prosthodontic Residents on the National Prosthodontic Resident Examination (NPRE) from 2011 to 2023. The International Journal of Prosthodontics. 2025. doi:10.11607/ijp.9486

BibTeX
@article{Bompolaki2025,
  author = {Bompolaki, Despoina and Mukherjee, Esha and Lee, Kelsey A K and Yoon, Christopher and An, Hongseok},
  title = {Evaluating AI Chatbot Performance Against Prosthodontic Residents on the National Prosthodontic Resident Examination (NPRE) from 2011 to 2023},
  journal = {The International Journal of Prosthodontics},
  year = {2025},
  doi = {10.11607/ijp.9486}
}
Kostenlos starten →