EUVIMEDOdkrijte EUVIMED
Open access

Performance of AI ‐Chatbots to Common Temporomandibular Joint Disorders ( TMDs ) Patient Queries: Accuracy, Completeness, Reliability and Readability

Mohamed G Hassan iD, Ahmed A Abdelaziz, Hams H Abdelrahman, Mostafa M Y Mohamed, Mohamed T Ellabban

Orthodontics & Craniofacial Research · 2025

DOI: 10.1111/ocr.12939 ↗

At a glance

Journal article in Orthodontics & Craniofacial Research, 2025. Written by Mohamed G Hassan and 4 co-authors. Institution: Division of Bone and Mineral Diseases, Department of Medicine, School of Medicine, Washington University in St. Louis, St. Louis, Missouri, USA. Major MEDLINE topics: Temporomandibular Joint Disorders, Comprehension. Cited 10 times so far (Crossref); 53 references. The full text is freely available in PubMed Central. Funded by: NIAMS NIH HHS.

PovzetekSubjectsBibliografski podatkiCiteRelated studies

Abstract (original language)

TMDs are a common group of conditions affecting the temporomandibular joint (TMJ) often resulting from factors like injury, stress or teeth grinding. This study aimed to evaluate the accuracy, completeness, reliability and readability of the responses generated by ChatGPT‐3.5, –4o and Google Gemini to TMD‐related inquiries. Forty‐five questions covering various aspects of TMDs were created by two experts and submitted by one author to ChatGPT‐3.5, ChatGPT‐4 and Google Gemini on the same day. The responses were evaluated for accuracy, completeness and reliability using modified Likert scales. Readability was analysed with six validated indices via a specialised tool. Additional features, such as the inclusion of graphical elements, references and safeguard mechanisms, were also documented and analysed. The Pearson Chi‐Square and One‐Way ANOVA tests were used for data analysis. Google Gemini achieved the highest accuracy, providing 100% correct responses, followed by ChatGPT‐3.5 (95.6%) and ChatGPT–4o (93.3%). ChatGPT–4o provided the most complete responses (91.1%), followed by ChatGPT‐03 (64.4%) and Google Gemini (42.2%). The majority of responses were reliable, with ChatGPT–4o at 93.3% ‘Absolutely Reliable’, compared to 46.7% for ChatGPT‐3.5 and 48.9% for Google Gemini. Both ChatGPT–4o and Google Gemini included references in responses, 22.2% and 13.3%, respectively, while ChatGPT‐3.5 included none. Google Gemini was the only model that included multimedia (6.7%). Readability scores were highest for ChatGPT‐3.5, suggesting its responses were more complex than those of Google Gemini and ChatGPT–4o. Both ChatGPT–4o and Google Gemini demonstrated accuracy and reliability in addressing TMD‐related questions, with their responses being clear, easy to understand and complemented by safeguard statements encouraging specialist consultation. However, both platforms lacked evidence‐based references. Only Google Gemini incorporated multimedia elements into its answers.

Subjects

MeSH terms (MEDLINE)

Temporomandibular Joint DisordersComprehensionHumansReproducibility of ResultsSurveys and QuestionnairesInternetGenerative Artificial Intelligence

Author keywords

ChatGPTGoogle Geminiartificial intelligencechatboteducationtemporomandibular jointtemporomandibular joint disorders

Authors and institutions

  • Division of Bone and Mineral Diseases, Department of Medicine, School of Medicine, Washington University in St. Louis, St. Louis, Missouri, USA.
  • Department of Orthodontics, Faculty of Dentistry, Assiut University, Assiut, Egypt.
  • Department of Pediatric Dentistry and Dental Public Health, Faculty of Dentistry, Alexandria University, Alexandria, Egypt.
  • Department of Oral Radiology, Faculty of Dentistry, Zarqa University, Zarqa, Jordan.

ORCID profiles: Mohamed G Hassan

Funding

  • NIAMS NIH HHS

Cite

APA

Hassan, M. G., Abdelaziz, A. A., Abdelrahman, H. H., Mohamed, M. M. Y., & Ellabban, M. T. (2025). Performance of AI ‐Chatbots to Common Temporomandibular Joint Disorders ( TMDs ) Patient Queries: Accuracy, Completeness, Reliability and Readability. Orthodontics & Craniofacial Research. https://doi.org/10.1111/ocr.12939

Vancouver

Hassan MG, Abdelaziz AA, Abdelrahman HH, Mohamed MMY, Ellabban MT. Performance of AI ‐Chatbots to Common Temporomandibular Joint Disorders ( TMDs ) Patient Queries: Accuracy, Completeness, Reliability and Readability. Orthodontics & Craniofacial Research. 2025. doi:10.1111/ocr.12939

BibTeX
@article{Hassan2025,
  author = {Hassan, Mohamed G and Abdelaziz, Ahmed A and Abdelrahman, Hams H and Mohamed, Mostafa M Y and Ellabban, Mohamed T},
  title = {Performance of AI ‐Chatbots to Common Temporomandibular Joint Disorders ( TMDs ) Patient Queries: Accuracy, Completeness, Reliability and Readability},
  journal = {Orthodontics & Craniofacial Research},
  year = {2025},
  doi = {10.1111/ocr.12939}
}
Start for free →