Evaluation of Artificial Intelligence Chatbots in Providing Brucellosis‐Related Health Information: A Multidimensional Quality Assessment
Zoonoses and Public Health · 2026-06 · 73 (4) · 380-392
DOI: 10.1111/zph.70059 ↗Kurz gefasst
Fachartikel in der Zeitschrift Zoonoses and Public Health, online erschienen am 10.04.2026. Verfasst von Feyza İzci Çetinkaya und 4 weiteren Autor:innen. Beteiligte Einrichtung: Infectious Diseases Clinic, Çankırı State Hospital, Çankırı, Turkey. Hauptthemen laut MEDLINE: Brucellosis, Artificial Intelligence. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 41 Literaturangaben.
Abstract (Originalsprache)
Introduction
AI‐based chatbots are increasingly used in accessing health information. However, there are significant differences in the accuracy, transparency of sources, readability and reliability of the information provided by these systems. In infectious diseases with heterogeneous clinical courses, requiring long‐term follow‐up and where patient information is critical, such as brucellosis, the quality of digital information sources is of particular importance. This study aims to compare the multidimensional performance of different AI‐based chatbots in the delivery of health information related to brucellosis.
Methods
Eight chatbots (ChatGPT‐4o, Gemini 2.5 Pro, Claude 3.5 Sonnet, Microsoft Copilot, Perplexity AI, Grok‐1.5, Mistral Le Chat and DeepSeek) were evaluated using standardized clinical questions. Clinical accuracy, source transparency, readability/patient‐friendliness, ethical safety and perceived trust level were analyzed using the QUEST, DISCERN, JAMA criteria, PEMAT‐P, Ateşman Readability Index and Trust/Confidence scales. Scores were normalized to create a heat map. Subgroup analyses were also conducted.
Results
Significant performance differences were found among the chatbots. DeepSeek achieved the highest scores in clinical accuracy, structured information presentation and source transparency. Claude stood out with higher perceived trust in addition to accuracy and transparency. It was observed that readability and perceived trust do not always coincide, and response length alone is not an indicator of quality.
Conclusion
This study demonstrates that AI‐based chatbots should not be evaluated in a clinical context using a single ‘best practice’ approach. For public health‐critical diseases such as brucellosis, selective and controlled chatbot integration tailored to the intended use may offer a safer and more effective approach.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Infectious Diseases Clinic, Çankırı State Hospital, Çankırı, Turkey.
- Ankara Provincial Health Directorate, Public Health Services Presidency, Ankara, Turkey.
- Infectious Diseases Clinic, Yozgat City Hospital, Yozgat, Turkey.
- Department of Infectious Diseases and Clinical Microbiology, Kayseri City Hospital, Kayseri, Turkey.
- Infectious Diseases Clinic, Private Gurlife Hospital, Eskişehir, Turkey.
ORCID-Profile: Feyza İzci Çetinkaya, Arzu Mirza, Hilal Ekici, Esma Eryılmaz Eren, Zeynep Ture
Zitieren
APA
İzci Çetinkaya, F., Mirza, A., Ekici, H., Eryılmaz Eren, E., & Ture, Z. (2026). Evaluation of Artificial Intelligence Chatbots in Providing Brucellosis‐Related Health Information: A Multidimensional Quality Assessment. Zoonoses and Public Health, 73(4), 380-392. https://doi.org/10.1111/zph.70059
Vancouver
İzci Çetinkaya F, Mirza A, Ekici H, Eryılmaz Eren E, Ture Z. Evaluation of Artificial Intelligence Chatbots in Providing Brucellosis‐Related Health Information: A Multidimensional Quality Assessment. Zoonoses and Public Health. 2026;73(4):380-392. doi:10.1111/zph.70059
BibTeX
@article{zcietinkaya2026,
author = {İzci Çetinkaya, Feyza and Mirza, Arzu and Ekici, Hilal and Eryılmaz Eren, Esma and Ture, Zeynep},
title = {Evaluation of Artificial Intelligence Chatbots in Providing Brucellosis‐Related Health Information: A Multidimensional Quality Assessment},
journal = {Zoonoses and Public Health},
year = {2026},
volume = {73},
number = {4},
pages = {380--392},
doi = {10.1111/zph.70059}
}
EUVIMED