A Comparative Analysis of Three Large Language Models on Bruxism Knowledge
Journal of Oral Rehabilitation · 2025
DOI: 10.1111/joor.13948 ↗Kurz gefasst
Beobachtungsstudie in der Zeitschrift Journal of Oral Rehabilitation, 2025. Verfasst von Elisa Souza Camargo und 4 weiteren Autor:innen. Beteiligte Einrichtung: Graduate Program in Dentistry, Pontifícia Universidade Católica do Paraná, Curitiba, Brazil. Hauptthemen laut MEDLINE: Bruxism, Artificial Intelligence, Language, Health Knowledge, Attitudes, Practice. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 29 Literaturangaben.
Abstract (Originalsprache)
Background
Artificial Intelligence (AI) has been widely used in health research, but the effectiveness of large language models (LLMs) in providing accurate information on bruxism has not yet been evaluated.
Objectives
To assess the readability, accuracy and consistency of three LLMs in responding to frequently asked questions about bruxism.
Methods
This cross‐sectional observational study utilised the Google Trends tool to identify the 10 most frequent topics about bruxism. Thirty frequently asked questions were selected, which were submitted to ChatGPT‐3.5, ChatGPT‐4 and Gemini at two different times (T1 and T2). The readability was measured using the Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKG) metrics. The responses were evaluated for accuracy using a three‐point scale, and consistency was verified by comparing responses between T1 and T2. Statistical analysis included ANOVA, chi‐squared tests and Cohen's kappa coefficient considering a p value of 0.5.
Results
In terms of readability, there was no difference in FRE. The Gemini model showed lower FKG scores than the Generative Pretrained Transformer (GPT)‐3.5 and GPT‐4 models. The average accuracy of the responses was 68.33% for GPT‐3.5, 65% for GPT‐4 and 55% for Gemini, with no significant differences between the models ( p = 0.290). Consistency was substantial for all models, with the highest being in GPT‐3.5 (95%). The three LLMs demonstrated substantial agreement between T1 and T2.
Conclusion
Gemini's responses were potentially more accessible to a broader patient population. LLMs demonstrated substantial consistency and moderate accuracy, indicating that these tools should not replace professional dental guidance.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Graduate Program in Dentistry, Pontifícia Universidade Católica do Paraná, Curitiba, Brazil.
- Zenith Educação Continuada, Florianópolis, Brazil.
- School of Dentistry, Universidade Tuiuti do Paraná, Curitiba, Brazil.
- Instituto de Ensino Odontológico de Bauru, Bauru, Brazil.
- Bauru Orofacial Pain Group, Bauru, Brazil.
ORCID-Profile: Elisa Souza Camargo, Cristiano Miranda de Araujo, Juliana Stuginski-Barbosa
Zitieren
APA
Camargo, E. S., Quadras, I. C. C., Garanhani, R. R., de Araujo, C. M., & Stuginski-Barbosa, J. (2025). A Comparative Analysis of Three Large Language Models on Bruxism Knowledge. Journal of Oral Rehabilitation. https://doi.org/10.1111/joor.13948
Vancouver
Camargo ES, Quadras ICC, Garanhani RR, de Araujo CM, Stuginski-Barbosa J. A Comparative Analysis of Three Large Language Models on Bruxism Knowledge. Journal of Oral Rehabilitation. 2025. doi:10.1111/joor.13948
BibTeX
@article{Camargo2025,
author = {Camargo, Elisa Souza and Quadras, Isabella Christina Costa and Garanhani, Roberto Ramos and de Araujo, Cristiano Miranda and Stuginski-Barbosa, Juliana},
title = {A Comparative Analysis of Three Large Language Models on Bruxism Knowledge},
journal = {Journal of Oral Rehabilitation},
year = {2025},
doi = {10.1111/joor.13948}
}
EUVIMED