EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing

Miltiadis A Makrygiannakis iD, Kostis Giannakopoulos, Eleftherios G Kaklamanos iD

European Journal of Orthodontics · 16.12.2025 · 48 (1)

DOI: 10.1093/ejo/cjae017 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift European Journal of Orthodontics, online erschienen am 13.04.2024. Verfasst von Miltiadis A Makrygiannakis und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Dentistry, National and Kapodistrian University of Athens, Athens 11527, Greece. Hauptthemen laut MEDLINE: Artificial Intelligence, Orthodontics, Evidence-Based Dentistry, Software. Die Arbeit wurde bisher 127-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The increasing utilization of large language models (LLMs) in Generative Artificial Intelligence across various medical and dental fields, and specifically orthodontics, raises questions about their accuracy.

Objective

This study aimed to assess and compare the answers offered by four LLMs: Google’s Bard, OpenAI’s ChatGPT-3.5, and ChatGPT-4, and Microsoft’s Bing, in response to clinically relevant questions within the field of orthodontics. Materials and methods Ten open-type clinical orthodontics-related questions were posed to the LLMs. The responses provided by the LLMs were assessed on a scale ranging from 0 (minimum) to 10 (maximum) points, benchmarked against robust scientific evidence, including consensus statements and systematic reviews, using a predefined rubric. After a 4-week interval from the initial evaluation, the answers were reevaluated to gauge intra-evaluator reliability. Statistical comparisons were conducted on the scores using Friedman’s and Wilcoxon’s tests to identify the model providing the answers with the most comprehensiveness, scientific accuracy, clarity, and relevance.

Results

Overall, no statistically significant differences between the scores given by the two evaluators, on both scoring occasions, were detected, so an average score for every LLM was computed. The LLM answers scoring the highest, were those of Microsoft Bing Chat (average score = 7.1), followed by ChatGPT 4 (average score = 4.7), Google Bard (average score = 4.6), and finally ChatGPT 3.5 (average score 3.8). While Microsoft Bing Chat statistically outperformed ChatGPT-3.5 (P-value = 0.017) and Google Bard (P-value = 0.029), as well, and Chat GPT-4 outperformed Chat GPT-3.5 (P-value = 0.011), all models occasionally produced answers with a lack of comprehensiveness, scientific accuracy, clarity, and relevance.

Limitations

The questions asked were indicative and did not cover the entire field of orthodontics.

Conclusions

Language models (LLMs) show great potential in supporting evidence-based orthodontics. However, their current limitations pose a potential risk of making incorrect healthcare decisions if utilized without careful consideration. Consequently, these tools cannot serve as a substitute for the orthodontist’s essential critical thinking and comprehensive subject knowledge. For effective integration into practice, further research, clinical validation, and enhancements to the models are essential. Clinicians must be mindful of the limitations of LLMs, as their imprudent utilization could have adverse effects on patient care.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceOrthodonticsEvidence-Based DentistrySoftwareLanguageHumansReproducibility of ResultsGenerative Artificial IntelligenceLarge Language Models

Stichwörter der Autor:innen

ChatGPTGoogle bardMicrosoft bing chatlarge language modelsorthodontics

Autor:innen und Institutionen

  • School of Dentistry, National and Kapodistrian University of Athens, Athens 11527, Greece.
  • School of Dentistry, European University Cyprus, Nicosia 2404, Cyprus.
  • School of Dentistry, Aristotle University of Thessaloniki, Thessaloniki 54124, Greece.
  • Hamdan bin Mohammed College of Dental Medicine, Mohammed bin Rashid University of Medicine and Health Sciences (MBRU), Dubai 505055, United Arab Emirates.

ORCID-Profile: Miltiadis A Makrygiannakis, Eleftherios G Kaklamanos

Zitieren

APA

Makrygiannakis, M. A., Giannakopoulos, K., & Kaklamanos, E. G. (2025). Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing. European Journal of Orthodontics, 48(1). https://doi.org/10.1093/ejo/cjae017

Vancouver

Makrygiannakis MA, Giannakopoulos K, Kaklamanos EG. Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing. European Journal of Orthodontics. 2025;48(1). doi:10.1093/ejo/cjae017

BibTeX
@article{Makrygiannakis2025,
  author = {Makrygiannakis, Miltiadis A and Giannakopoulos, Kostis and Kaklamanos, Eleftherios G},
  title = {Evidence-based potential of generative artificial intelligence large language models in orthodontics: a comparative study of ChatGPT, Google Bard, and Microsoft Bing},
  journal = {European Journal of Orthodontics},
  year = {2025},
  volume = {48},
  number = {1},
  doi = {10.1093/ejo/cjae017}
}
Kostenlos starten →