EUVIMEDEUVIMED entdecken
Vergleichsstudie

Quality, Readability, and Reliability of Artificial Intelligence Chatbot Responses to Parent‐Centered Questions on Newborn Hearing Screening: A Cross‐Sectional Comparative Study

Arif Ozses iD, Beyza Demirtas Yilmaz iD, Merve Ozses iD

Journal of Evaluation in Clinical Practice · 2026-12 · 32 (8)

DOI: 10.1111/jep.70721 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of Evaluation in Clinical Practice, online erschienen am 04.11.2026. Verfasst von Arif Ozses und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Otorhinolaryngology, Faculty of Medicine, Akdeniz University, Antalya, Türkiye. Hauptthemen laut MEDLINE: Neonatal Screening, Parents, Comprehension, Hearing Tests. Die Arbeit nennt 30 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Aims and Objectives Parents increasingly use large language models (LLMs) to obtain health‐related information; however, the quality, readability, and reliability of AI‐generated patient education remain uncertain. This study evaluated the performance of three LLM‐based chatbots in providing parent‐centred information regarding newborn hearing screening.

Methods

Thirty‐eight parent‐centred questions regarding newborn hearing screening were submitted to ChatGPT‐5.1, Gemini 2.5 Flash, and DeepSeek‐V3. Responses were assessed using a five‐point Likert scale for accuracy, comprehensiveness, clarity, and credibility. Responses were assessed using the modified EQIP for quality, and the Flesch Reading Ease Score (FRES) and Flesch–Kincaid Grade Level (FKGL) for readability.

Results

DeepSeek‐V3 outperformed both ChatGPT‐5.1 and Gemini 2.5 Flash in accuracy and comprehensiveness ( p < 0.05), whereas its clarity scores were lower ( p < 0.05). The FRES scores (51.5–59.3) indicated difficult readability, whereas the FKGL scores (8.4–8.9) corresponded to an eighth‐grade reading level. Gemini 2.5 Flash demonstrated higher FRES scores ( p = 0.001). mEQIP scores were not significant ( p = 0.148) and indicated good quality.

Conclusions

LLM‐based chatbots generally perform acceptably on newborn hearing screening questions, although substantial variability was observed across models. Despite acceptable readability levels, chatbot‐generated content remains relatively difficult to read. LLMs may serve as complementary information resources for parents seeking information about newborn hearing screening, although further research is needed before their routine clinical use.

Rationale

This study addresses the JECP journal's aim to evaluate the clinical applications of emerging artificial intelligence technologies in healthcare. Evaluating the quality, accuracy, readability, and reliability of chatbot‐generated information for newborn hearing screening provides evidence to support person‐centred care, patient education, and the responsible integration of AI into clinical practice.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Neonatal ScreeningParentsComprehensionHearing TestsArtificial IntelligencePatient Education as TopicHumansInfant, NewbornCross-Sectional StudiesReproducibility of ResultsLarge Language Models

Stichwörter der Autor:innen

artificial intelligencehealth literacylarge language modelsnewborn hearing screeningpatient education

Autor:innen und Institutionen

  • Department of Otorhinolaryngology, Faculty of Medicine, Akdeniz University, Antalya, Türkiye.
  • Department of Audiology, Faculty of Health Sciences, Erciyes University, Kayseri, Türkiye.
  • Department of Audiology, Faculty of Health Sciences, Aydın Adnan Menderes University, Aydın, Türkiye.

ORCID-Profile: Arif Ozses, Beyza Demirtas Yilmaz, Merve Ozses

Zitieren

APA

Ozses, A., Yilmaz, B. D., & Ozses, M. (2026). Quality, Readability, and Reliability of Artificial Intelligence Chatbot Responses to Parent‐Centered Questions on Newborn Hearing Screening: A Cross‐Sectional Comparative Study. Journal of Evaluation in Clinical Practice, 32(8). https://doi.org/10.1111/jep.70721

Vancouver

Ozses A, Yilmaz BD, Ozses M. Quality, Readability, and Reliability of Artificial Intelligence Chatbot Responses to Parent‐Centered Questions on Newborn Hearing Screening: A Cross‐Sectional Comparative Study. Journal of Evaluation in Clinical Practice. 2026;32(8). doi:10.1111/jep.70721

BibTeX
@article{Ozses2026,
  author = {Ozses, Arif and Yilmaz, Beyza Demirtas and Ozses, Merve},
  title = {Quality, Readability, and Reliability of Artificial Intelligence Chatbot Responses to Parent‐Centered Questions on Newborn Hearing Screening: A Cross‐Sectional Comparative Study},
  journal = {Journal of Evaluation in Clinical Practice},
  year = {2026},
  volume = {32},
  number = {8},
  doi = {10.1111/jep.70721}
}
Kostenlos starten →