EUVIMEDEUVIMED entdecken
Crossref

Evaluation of Artificial Intelligence Chatbots in Providing Brucellosis‐Related Health Information: A Multidimensional Quality Assessment

Feyza İzci Çetinkaya iD, Arzu Mirza iD, Hilal Ekici iD, Esma Eryılmaz Eren iD, Zeynep Ture iD

Zoonoses and Public Health · 2026-06 · 73 (4) · 380-392

DOI: 10.1111/zph.70059 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Zoonoses and Public Health, online erschienen am 10.04.2026. Verfasst von Feyza İzci Çetinkaya und 4 weiteren Autor:innen. Beteiligte Einrichtung: Infectious Diseases Clinic, Çankırı State Hospital, Çankırı, Turkey. Hauptthemen laut MEDLINE: Brucellosis, Artificial Intelligence. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 41 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

AI‐based chatbots are increasingly used in accessing health information. However, there are significant differences in the accuracy, transparency of sources, readability and reliability of the information provided by these systems. In infectious diseases with heterogeneous clinical courses, requiring long‐term follow‐up and where patient information is critical, such as brucellosis, the quality of digital information sources is of particular importance. This study aims to compare the multidimensional performance of different AI‐based chatbots in the delivery of health information related to brucellosis.

Methods

Eight chatbots (ChatGPT‐4o, Gemini 2.5 Pro, Claude 3.5 Sonnet, Microsoft Copilot, Perplexity AI, Grok‐1.5, Mistral Le Chat and DeepSeek) were evaluated using standardized clinical questions. Clinical accuracy, source transparency, readability/patient‐friendliness, ethical safety and perceived trust level were analyzed using the QUEST, DISCERN, JAMA criteria, PEMAT‐P, Ateşman Readability Index and Trust/Confidence scales. Scores were normalized to create a heat map. Subgroup analyses were also conducted.

Results

Significant performance differences were found among the chatbots. DeepSeek achieved the highest scores in clinical accuracy, structured information presentation and source transparency. Claude stood out with higher perceived trust in addition to accuracy and transparency. It was observed that readability and perceived trust do not always coincide, and response length alone is not an indicator of quality.

Conclusion

This study demonstrates that AI‐based chatbots should not be evaluated in a clinical context using a single ‘best practice’ approach. For public health‐critical diseases such as brucellosis, selective and controlled chatbot integration tailored to the intended use may offer a safer and more effective approach.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

BrucellosisArtificial IntelligenceHumans

Stichwörter der Autor:innen

artificial intelligencebrucellosishealth literacylarge language modelspublic health communication

Autor:innen und Institutionen

  • Infectious Diseases Clinic, Çankırı State Hospital, Çankırı, Turkey.
  • Ankara Provincial Health Directorate, Public Health Services Presidency, Ankara, Turkey.
  • Infectious Diseases Clinic, Yozgat City Hospital, Yozgat, Turkey.
  • Department of Infectious Diseases and Clinical Microbiology, Kayseri City Hospital, Kayseri, Turkey.
  • Infectious Diseases Clinic, Private Gurlife Hospital, Eskişehir, Turkey.

ORCID-Profile: Feyza İzci Çetinkaya, Arzu Mirza, Hilal Ekici, Esma Eryılmaz Eren, Zeynep Ture

Zitieren

APA

İzci Çetinkaya, F., Mirza, A., Ekici, H., Eryılmaz Eren, E., & Ture, Z. (2026). Evaluation of Artificial Intelligence Chatbots in Providing Brucellosis‐Related Health Information: A Multidimensional Quality Assessment. Zoonoses and Public Health, 73(4), 380-392. https://doi.org/10.1111/zph.70059

Vancouver

İzci Çetinkaya F, Mirza A, Ekici H, Eryılmaz Eren E, Ture Z. Evaluation of Artificial Intelligence Chatbots in Providing Brucellosis‐Related Health Information: A Multidimensional Quality Assessment. Zoonoses and Public Health. 2026;73(4):380-392. doi:10.1111/zph.70059

BibTeX
@article{zcietinkaya2026,
  author = {İzci Çetinkaya, Feyza and Mirza, Arzu and Ekici, Hilal and Eryılmaz Eren, Esma and Ture, Zeynep},
  title = {Evaluation of Artificial Intelligence Chatbots in Providing Brucellosis‐Related Health Information: A Multidimensional Quality Assessment},
  journal = {Zoonoses and Public Health},
  year = {2026},
  volume = {73},
  number = {4},
  pages = {380--392},
  doi = {10.1111/zph.70059}
}
Kostenlos starten →