Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing
European Journal of Orthodontics · 16.12.2025 · 48 (1)
DOI: 10.1093/ejo/cjae017 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift European Journal of Orthodontics, online erschienen am 13.04.2024. Verfasst von Miltiadis A Makrygiannakis und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Dentistry, National and Kapodistrian University of Athens, Athens 11527, Greece. Hauptthemen laut MEDLINE: Artificial Intelligence, Orthodontics, Evidence-Based Dentistry, Software. Die Arbeit wurde bisher 127-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Background
The increasing utilization of large language models (LLMs) in Generative Artificial Intelligence across various medical and dental fields, and specifically orthodontics, raises questions about their accuracy.
Objective
This study aimed to assess and compare the answers offered by four LLMs: Google’s Bard, OpenAI’s ChatGPT-3.5, and ChatGPT-4, and Microsoft’s Bing, in response to clinically relevant questions within the field of orthodontics. Materials and methods Ten open-type clinical orthodontics-related questions were posed to the LLMs. The responses provided by the LLMs were assessed on a scale ranging from 0 (minimum) to 10 (maximum) points, benchmarked against robust scientific evidence, including consensus statements and systematic reviews, using a predefined rubric. After a 4-week interval from the initial evaluation, the answers were reevaluated to gauge intra-evaluator reliability. Statistical comparisons were conducted on the scores using Friedman’s and Wilcoxon’s tests to identify the model providing the answers with the most comprehensiveness, scientific accuracy, clarity, and relevance.
Results
Overall, no statistically significant differences between the scores given by the two evaluators, on both scoring occasions, were detected, so an average score for every LLM was computed. The LLM answers scoring the highest, were those of Microsoft Bing Chat (average score = 7.1), followed by ChatGPT 4 (average score = 4.7), Google Bard (average score = 4.6), and finally ChatGPT 3.5 (average score 3.8). While Microsoft Bing Chat statistically outperformed ChatGPT-3.5 (P-value = 0.017) and Google Bard (P-value = 0.029), as well, and Chat GPT-4 outperformed Chat GPT-3.5 (P-value = 0.011), all models occasionally produced answers with a lack of comprehensiveness, scientific accuracy, clarity, and relevance.
Limitations
The questions asked were indicative and did not cover the entire field of orthodontics.
Conclusions
Language models (LLMs) show great potential in supporting evidence-based orthodontics. However, their current limitations pose a potential risk of making incorrect healthcare decisions if utilized without careful consideration. Consequently, these tools cannot serve as a substitute for the orthodontist’s essential critical thinking and comprehensive subject knowledge. For effective integration into practice, further research, clinical validation, and enhancements to the models are essential. Clinicians must be mindful of the limitations of LLMs, as their imprudent utilization could have adverse effects on patient care.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Dentistry, National and Kapodistrian University of Athens, Athens 11527, Greece.
- School of Dentistry, European University Cyprus, Nicosia 2404, Cyprus.
- School of Dentistry, Aristotle University of Thessaloniki, Thessaloniki 54124, Greece.
- Hamdan bin Mohammed College of Dental Medicine, Mohammed bin Rashid University of Medicine and Health Sciences (MBRU), Dubai 505055, United Arab Emirates.
ORCID-Profile: Miltiadis A Makrygiannakis, Eleftherios G Kaklamanos
Zitieren
APA
Makrygiannakis, M. A., Giannakopoulos, K., & Kaklamanos, E. G. (2025). Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing. European Journal of Orthodontics, 48(1). https://doi.org/10.1093/ejo/cjae017
Vancouver
Makrygiannakis MA, Giannakopoulos K, Kaklamanos EG. Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing. European Journal of Orthodontics. 2025;48(1). doi:10.1093/ejo/cjae017
BibTeX
@article{Makrygiannakis2025,
author = {Makrygiannakis, Miltiadis A and Giannakopoulos, Kostis and Kaklamanos, Eleftherios G},
title = {Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing},
journal = {European Journal of Orthodontics},
year = {2025},
volume = {48},
number = {1},
doi = {10.1093/ejo/cjae017}
}
EUVIMED