EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY-NC-ND 4.0

Artificial intelligence performance in answering multiple-choice oral pathology questions: a comparative analysis

Birkan Eyup Yilmaz, Busra Nur Gokkurt Yilmaz, Furkan Ozbey

BMC Oral Health · 15.04.2025 · 25 (1)

DOI: 10.1186/s12903-025-05926-2 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift BMC Oral Health, online erschienen am 15.04.2025. Verfasst von Birkan Eyup Yilmaz und 2 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Dentistry, Department of Oral and Maxillofacial Surgery, Giresun University, Giresun, Türkiye. ylmzbirkan@gmail.com. Hauptthemen laut MEDLINE: Artificial Intelligence, Pathology, Oral, Educational Measurement, Education, Dental. Die Arbeit wurde bisher 77-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Artificial intelligence (AI) has rapidly advanced in healthcare and dental education, significantly impacting diagnostic processes, treatment planning, and academic training. The aim of this study is to evaluate the performance differences between different large language models (LLMs) by analyzing their accuracy rates in answers to multiple choice oral pathology questions. METHODS: This study evaluates the performance of eight LLMs (Gemini 1.5, Gemini 2, ChatGPT 4o, ChatGPT 4, ChatGPT o1, Copilot, Claude 3.5, Deepseek) in answering multiple-choice oral pathology questions from the Turkish Dental Specialization Examination (DUS). A total of 100 questions from 2012 to 2021 were analyzed. Questions were classified as "case-based" or "knowledge-based". The responses were classified as "correct" or "incorrect" based on official answer keys. To prevent learning biases, no follow-up questions or feedback were provided after the LLMs' responses. RESULTS: Significant performance differences were observed among the models (p < 0.001). ChatGPT o1 achieved the highest accuracy (96 correct, 4 incorrect), followed by Claude (84 correct), Gemini 2 and Deepseek (82 correct each). Copilot had the lowest performance (61 correct). Case-based questions showed notable performance variations (p = 0.034), where ChatGPT o1 and Claude excelled. For knowledge-based questions, ChatGPT o1 and Deepseek demonstrated the highest accuracy (p < 0.001). Post-hoc analysis revealed that ChatGPT o1 performed significantly better than most other models across both case-based and knowledge-based questions (p < 0.0031). CONCLUSION: LLMs demonstrated variable proficiency in oral pathology questions, with ChatGPT o1 showing higher accuracy. LLMs shows promise as a supplementary educational tool, though further validation is required.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligencePathology, OralEducational MeasurementEducation, DentalHumans

Stichwörter der Autor:innen

Artificial intelligenceLarge language modelsOral pathology

Autor:innen und Institutionen

  • Faculty of Dentistry, Department of Oral and Maxillofacial Surgery, Giresun University, Giresun, Türkiye. ylmzbirkan@gmail.com.
  • Giresun Oral and Dental Health Centre, Department of Dentomaxillofacial Radiology, Giresun, Türkiye.
  • Faculty of Dentistry, Department of Dentomaxillofacial Radiology, Afyonkarahisar Health Sciences University, Afyonkarahisar, Türkiye.

Zitieren

APA

Yilmaz, B. E., Gokkurt Yilmaz, B. N., & Ozbey, F. (2025). Artificial intelligence performance in answering multiple-choice oral pathology questions: a comparative analysis. BMC Oral Health, 25(1). https://doi.org/10.1186/s12903-025-05926-2

Vancouver

Yilmaz BE, Gokkurt Yilmaz BN, Ozbey F. Artificial intelligence performance in answering multiple-choice oral pathology questions: a comparative analysis. BMC Oral Health. 2025;25(1). doi:10.1186/s12903-025-05926-2

BibTeX
@article{Yilmaz2025,
  author = {Yilmaz, Birkan Eyup and Gokkurt Yilmaz, Busra Nur and Ozbey, Furkan},
  title = {Artificial intelligence performance in answering multiple-choice oral pathology questions: a comparative analysis},
  journal = {BMC Oral Health},
  year = {2025},
  volume = {25},
  number = {1},
  doi = {10.1186/s12903-025-05926-2}
}
Kostenlos starten →