Reliability and Performance of Four Large Language Models in Orthodontic Knowledge Assessment
Journal of Dental Education · 2026
DOI: 10.1002/jdd.14002 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Dental Education, 2026. Verfasst von Shankargouda Patil und 3 weiteren Autor:innen. Beteiligte Einrichtung: College of Dental Medicine, Roseman University of Health Sciences, South Jordan, Utah, USA. Hauptthemen laut MEDLINE: Orthodontics, Educational Measurement, Artificial Intelligence, Language. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 7 Literaturangaben.
Abstract (Originalsprache)
Artificial intelligence‐based large language models (LLMs) are gaining prominence as educational tools. This study evaluated the accuracy and reliability of four popular publicly available LLM models—ChatGPT 4.0, ChatGPT 4o, Google Gemini, and Microsoft CoPilot—in answering orthodontic questions from the National Board of Dental Examiners examinations. Each model was tested across three trials to assess response consistency. Reliability was analyzed using Cohen's and Fleiss' Kappa. Among the four tested models, Microsoft CoPilot demonstrated the highest reliability, while ChatGPT‐4.0 had the highest accuracy. Variability across trials suggests that AI‐generated responses remain inconsistent. The variable responses generated over time by LLMs limit their standalone applicability in orthodontic education. Older models at times outperformed newer models. AI model updates do not necessarily lead to improved reliability. Although AI models may show potential as supplementary study aids, their accuracy and stability require further refinement before being deployed in educational contexts.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Dental Medicine, Roseman University of Health Sciences, South Jordan, Utah, USA.
- Faculty of Medicine and Dentistry, Katz Group Centre for Pharmacy and Health Research, Mike Petryk School of Dentistry, University of Alberta, Edmonton, Alberta, Canada.
ORCID-Profile: Shankargouda Patil
Zitieren
APA
Patil, S., Eisenhuth, G., El-Bialy, T., & Licari, F. W. (2026). Reliability and Performance of Four Large Language Models in Orthodontic Knowledge Assessment. Journal of Dental Education. https://doi.org/10.1002/jdd.14002
Vancouver
Patil S, Eisenhuth G, El-Bialy T, Licari FW. Reliability and Performance of Four Large Language Models in Orthodontic Knowledge Assessment. Journal of Dental Education. 2026. doi:10.1002/jdd.14002
BibTeX
@article{Patil2026,
author = {Patil, Shankargouda and Eisenhuth, Gabriel and El-Bialy, Tarek and Licari, Frank W},
title = {Reliability and Performance of Four Large Language Models in Orthodontic Knowledge Assessment},
journal = {Journal of Dental Education},
year = {2026},
doi = {10.1002/jdd.14002}
}
EUVIMED