EUVIMEDEUVIMED entdecken
BeobachtungsstudieVergleichsstudie

A Comparative Analysis of Three Large Language Models on Bruxism Knowledge

Elisa Souza Camargo iD, Isabella Christina Costa Quadras, Roberto Ramos Garanhani, Cristiano Miranda de Araujo iD, Juliana Stuginski-Barbosa iD

Journal of Oral Rehabilitation · 2025

DOI: 10.1111/joor.13948 ↗

Kurz gefasst

Beobachtungsstudie in der Zeitschrift Journal of Oral Rehabilitation, 2025. Verfasst von Elisa Souza Camargo und 4 weiteren Autor:innen. Beteiligte Einrichtung: Graduate Program in Dentistry, Pontifícia Universidade Católica do Paraná, Curitiba, Brazil. Hauptthemen laut MEDLINE: Bruxism, Artificial Intelligence, Language, Health Knowledge, Attitudes, Practice. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 29 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Artificial Intelligence (AI) has been widely used in health research, but the effectiveness of large language models (LLMs) in providing accurate information on bruxism has not yet been evaluated.

Objectives

To assess the readability, accuracy and consistency of three LLMs in responding to frequently asked questions about bruxism.

Methods

This cross‐sectional observational study utilised the Google Trends tool to identify the 10 most frequent topics about bruxism. Thirty frequently asked questions were selected, which were submitted to ChatGPT‐3.5, ChatGPT‐4 and Gemini at two different times (T1 and T2). The readability was measured using the Flesch Reading Ease (FRE) and Flesch–Kincaid Grade Level (FKG) metrics. The responses were evaluated for accuracy using a three‐point scale, and consistency was verified by comparing responses between T1 and T2. Statistical analysis included ANOVA, chi‐squared tests and Cohen's kappa coefficient considering a p value of 0.5.

Results

In terms of readability, there was no difference in FRE. The Gemini model showed lower FKG scores than the Generative Pretrained Transformer (GPT)‐3.5 and GPT‐4 models. The average accuracy of the responses was 68.33% for GPT‐3.5, 65% for GPT‐4 and 55% for Gemini, with no significant differences between the models ( p = 0.290). Consistency was substantial for all models, with the highest being in GPT‐3.5 (95%). The three LLMs demonstrated substantial agreement between T1 and T2.

Conclusion

Gemini's responses were potentially more accessible to a broader patient population. LLMs demonstrated substantial consistency and moderate accuracy, indicating that these tools should not replace professional dental guidance.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BruxismArtificial IntelligenceLanguageHealth Knowledge, Attitudes, PracticeHealth LiteracyHumansCross-Sectional StudiesComprehensionLarge Language Models

Stichwörter der Autor:innen

artificial intelligencebruxismdental researchknowledge acquisitionmachine learningnatural language processing

Autor:innen und Institutionen

  • Graduate Program in Dentistry, Pontifícia Universidade Católica do Paraná, Curitiba, Brazil.
  • Zenith Educação Continuada, Florianópolis, Brazil.
  • School of Dentistry, Universidade Tuiuti do Paraná, Curitiba, Brazil.
  • Instituto de Ensino Odontológico de Bauru, Bauru, Brazil.
  • Bauru Orofacial Pain Group, Bauru, Brazil.

ORCID-Profile: Elisa Souza Camargo, Cristiano Miranda de Araujo, Juliana Stuginski-Barbosa

Zitieren

APA

Camargo, E. S., Quadras, I. C. C., Garanhani, R. R., de Araujo, C. M., & Stuginski-Barbosa, J. (2025). A Comparative Analysis of Three Large Language Models on Bruxism Knowledge. Journal of Oral Rehabilitation. https://doi.org/10.1111/joor.13948

Vancouver

Camargo ES, Quadras ICC, Garanhani RR, de Araujo CM, Stuginski-Barbosa J. A Comparative Analysis of Three Large Language Models on Bruxism Knowledge. Journal of Oral Rehabilitation. 2025. doi:10.1111/joor.13948

BibTeX
@article{Camargo2025,
  author = {Camargo, Elisa Souza and Quadras, Isabella Christina Costa and Garanhani, Roberto Ramos and de Araujo, Cristiano Miranda and Stuginski-Barbosa, Juliana},
  title = {A Comparative Analysis of Three Large Language Models on Bruxism Knowledge},
  journal = {Journal of Oral Rehabilitation},
  year = {2025},
  doi = {10.1111/joor.13948}
}
Kostenlos starten →