Evaluation of artificial intelligence chatbots in answering questions about unintentional weight loss: a comparative study
Annals of Clinical and Analytical Medicine · 01.06.2026 · 17 (6) · 555-560
DOI: 10.4328/acam.22948 ↗Kurz gefasst
Fachartikel in der Zeitschrift Annals of Clinical and Analytical Medicine, erschienen am 01.06.2026. Verfasst von Zekeriya Keskin und 2 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben.
Abstract (Originalsprache)
Aim
Our study aims to evaluate the accuracy comparatively reliability of the responses provided by artificial intelligence (AI) chatbots ChatGPT, DeepSeek, and Gemini to clinical questions regarding a symptom with a multidisciplinary and broad differential diagnosis, such as unintentional weight loss (UWL).
Methods
129 clinical questions, definitions, symptomatology, differential diagnosis, diagnostic approach, treatment and management, and patient questions compiled from various health websites, books, and guides were categorized under six main headings and directed to three different AI chatbots. Each chatbot was asked to score the difficulty level of each question, and the relationship between difficulty assessments and accuracy performance was analyzed. Each response was evaluated by three internal medicine specialists and scored 1–4 based on accuracy.
Results
ChatGPT and DeepSeek demonstrated similar performance with high accuracy rates, while Gemini performed at a significantly lower accuracy level. Significant differences were observed between the chatbots in five of the six question groups (p0.05).
Conclusion
While ChatGPT and DeepSeek offer high accuracy and reliability, Gemini has performed below these two AI chatbots. Our findings indicate that AI chatbots should not be used as standalone tools for diagnosis, treatment, and management in complex clinical decision-making processes. However, they can be considered an important complementary tool for rapid access to accurate information and supporting clinical decisions.
Zitieren
APA
Keskin, Z., Aşkın, M. F., & Büyüktekeli, O. (2026). Evaluation of artificial intelligence chatbots in answering questions about unintentional weight loss: a comparative study. Annals of Clinical and Analytical Medicine, 17(6), 555-560. https://doi.org/10.4328/acam.22948
Vancouver
Keskin Z, Aşkın MF, Büyüktekeli O. Evaluation of artificial intelligence chatbots in answering questions about unintentional weight loss: a comparative study. Annals of Clinical and Analytical Medicine. 2026;17(6):555-560. doi:10.4328/acam.22948
BibTeX
@article{Keskin2026,
author = {Keskin, Zekeriya and Aşkın, Muhammed Faruk and Büyüktekeli, Onur},
title = {Evaluation of artificial intelligence chatbots in answering questions about unintentional weight loss: a comparative study},
journal = {Annals of Clinical and Analytical Medicine},
year = {2026},
volume = {17},
number = {6},
pages = {555--560},
doi = {10.4328/acam.22948}
}
EUVIMED