EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Performance of 7 Artificial Intelligence Chatbots on Board-style Endodontic Questions

Poorya Jalali, Hossein Mohammad-Rahimi, Feng-Ming Wang, Fatemeh Sohrabniya, Seyed AmirHossein Ourang, Yuke Tian, Frederico C Martinho, Ali Nosrat

Journal of Endodontics · 2025-10 · 51 (10) · 1413-1419

DOI: 10.1016/j.joen.2025.06.014 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Endodontics, 2025-10. Verfasst von Poorya Jalali und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Endodontics, Texas A&M College of Dentistry, Dallas, Texas. Hauptthemen laut MEDLINE: Endodontics, Artificial Intelligence, Educational Measurement, Education, Dental. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

INTRODUCTION: The aim of this study was to assess the overall performance of artificial intelligence chatbots in answering board-style endodontic questions. METHODS: One hundred multiple choice endodontic questions, following the style of American Board of Endodontics Written Exam, were generated by two board-certified endodontists. The questions were submitted to the following chatbots, three times in a row: Gemini Advanced, Gemini, Microsoft Copilot, GPT-3.5, GPT-4o, GPT-4.0, and Claude 3.5 Sonnet. The chatbot was asked to choose the correct response and to explain the justification. The response to the question was considered "correct" only if the chatbot picked the right choice in ALL 3 attempts. The quality of reasoning as to why the chatbot selected the answer choice was scored using a three-ordinal scale (0, 1, 2). Two calibrated reviewers scored all 2100 responses independently. Categorical data were analyzed using Chi-square test; ordinal data were analyzed using Kruskal-Wallis and Mann-Whitney tests. RESULTS: The accuracy scores ranged from 48% (Microsoft Copilot) to 71% (Gemini Advanced, GPT-3.5, and Claude 3.5 Sonnet) (P .05). GPT-3.5, GPT-4o, GPT-4.0 and Claude 3.5 Sonnet performed significantly better with textbook-based questions (P < .05). Reasoning scores showed different distribution among chatbots (P < .05). Gemini Advanced had the highest rate of score 2 (81%) and the lowest rate of score 0 (18.5%). CONCLUSIONS: Comprehensive assessment of seven AI chatbots' performance on board-style endodontic questions revealed their capacities and limitations as educational resources in the field of endodontics.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

EndodonticsArtificial IntelligenceEducational MeasurementEducation, DentalHumansGenerative Artificial Intelligence

Stichwörter der Autor:innen

Artificial intelligenceboard-stylechatboteducationendodonticslarge language models

Autor:innen und Institutionen

  • Department of Endodontics, Texas A&M College of Dentistry, Dallas, Texas.
  • Department of Dentistry and Oral Health, Aarhus University, Aarhus, Denmark; Conservative Dentistry and Periodontology, LMU Klinikum, LMU, Munich, Germany.
  • Topic Group Dental Diagnostics and Digital Dentistry, ITU/WHO Focus Group AI on Health, Berlin, Germany.
  • Dentofacial Deformities Research Center, Research Institute of Dental Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
  • Department of Statistics and Data Science, Southern Methodist University, Dallas, Texas.
  • Division of Endodontics, Department of Advanced Oral Sciences and Therapeutics, School of Dentistry, University of Maryland Baltimore, Baltimore, Maryland.

Zitieren

APA

Jalali, P., Mohammad-Rahimi, H., Wang, F. M., Sohrabniya, F., AmirHossein Ourang, S., Tian, Y., Martinho, F. C., & Nosrat, A. (2025). Performance of 7 Artificial Intelligence Chatbots on Board-style Endodontic Questions. Journal of Endodontics, 51(10), 1413-1419. https://doi.org/10.1016/j.joen.2025.06.014

Vancouver

Jalali P, Mohammad-Rahimi H, Wang FM, Sohrabniya F, AmirHossein Ourang S, Tian Y, et al. Performance of 7 Artificial Intelligence Chatbots on Board-style Endodontic Questions. Journal of Endodontics. 2025;51(10):1413-1419. doi:10.1016/j.joen.2025.06.014

BibTeX
@article{Jalali2025,
  author = {Jalali, Poorya and Mohammad-Rahimi, Hossein and Wang, Feng-Ming and Sohrabniya, Fatemeh and AmirHossein Ourang, Seyed and Tian, Yuke and Martinho, Frederico C and Nosrat, Ali},
  title = {Performance of 7 Artificial Intelligence Chatbots on Board-style Endodontic Questions},
  journal = {Journal of Endodontics},
  year = {2025},
  volume = {51},
  number = {10},
  pages = {1413--1419},
  doi = {10.1016/j.joen.2025.06.014}
}
Kostenlos starten →