Comparison of Emotional Content in Text Responses From Physicians and AI Chatbots to Patient Health Queries: Cross-Sectional Study
Journal of Medical Internet Research · 2026
DOI: 10.2196/85516 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Journal of Medical Internet Research, 2026. Verfasst von Daniel T Burns und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Mathematics & Statistics, University of Wyoming, Laramie, WY, United States. Hauptthemen laut MEDLINE: Physicians, Emotions, Artificial Intelligence. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 74 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIGMS NIH HHS.
Abstract (Originalsprache)
Background
Surveys show that many people are willing to use generative artificial intelligence (AI) for health questions. Prior research has largely focused on chatbot accuracy, with some studies finding that both physicians and consumers overwhelmingly prefer chatbot-generated text over physician responses.
Objective
This study aimed to characterize and compare the emotional content of responses from physicians and 2 AI chatbots (OpenAI’s ChatGPT and Google’s Gemini) and to assess differences in reading level and use of medical disclaimers.
Methods
A public, patient-deidentified telehealth website was used to compile 100 physician-answered questions. The same questions were posed to both chatbots between May 18 and 19, 2025. Two coders classified the emotional content of each sentence using a predefined codebook and reviewed for agreement. Emotions were ranked as primary, secondary, and tertiary by the proportion of sentences classified as each emotion per response. Multinomial logistic regression compared emotional rankings using physician responses as the reference. Word count, Flesch Reading Ease, and Flesch-Kincaid Grade Level were analyzed via ANOVA with the Tukey honestly significant difference test. Disclaimer use was compared between chatbots using a χ2 test.
Results
Primary emotions were overwhelmingly neutral, except for one response from each chatbot in which anger was primary. For secondary emotions, the odds ratio of hope was 80.28% (95% CI 37.71%-93.76%) lower for ChatGPT, while the odds ratio of fear was 3.29 (95% CI 1.44-7.49) times higher for Gemini. For tertiary emotions, the odds ratio of compassion was 1.94 (95% CI 1.06-3.54) times higher, and the odds ratio of having no tertiary emotion was 84.33% (95% CI 64.72%-93.04%) lower for Gemini. Gemini responses averaged 889.1 (SD 305.7) words, ChatGPT 476.5 (SD 109.5), and physicians 193.5 (SD 113.6). Gemini had the lowest average Flesch Reading Ease score at 39.9 (SD 8.8), followed by ChatGPT at 45.8 (SD 12.8), while physicians had the highest at 51.9 (SD 13.6). Gemini had the highest average Flesch-Kincaid Grade Level at 11.3 (SD 1.5), followed by ChatGPT at 9.9 (SD 1.9), and physicians at 9.2 (SD 2.4). Gemini was significantly more likely to include a disclaimer than ChatGPT (χ21=49.2; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Mathematics & Statistics, University of Wyoming, Laramie, WY, United States.
- Department of Journalism & Media Communication, Colorado State University, Fort Collins, CO, United States.
- Department of Otolaryngology- Head and Neck Surgery, University of Washington, Seattle, WA, United States.
- Data Science and Learning, Argonne National Laboratory, Lemont, IL, United States.
ORCID-Profile: Daniel T Burns, Channing Bice, Paul E Johnson, Nicholas Chia, Timothy Robinson
Förderung
- NIGMS NIH HHS
Zitieren
APA
Burns, D. T., Bice, C., Johnson, P. E., Chia, N., & Robinson, T. (2026). Comparison of Emotional Content in Text Responses From Physicians and AI Chatbots to Patient Health Queries: Cross-Sectional Study. Journal of Medical Internet Research. https://doi.org/10.2196/85516
Vancouver
Burns DT, Bice C, Johnson PE, Chia N, Robinson T. Comparison of Emotional Content in Text Responses From Physicians and AI Chatbots to Patient Health Queries: Cross-Sectional Study. Journal of Medical Internet Research. 2026. doi:10.2196/85516
BibTeX
@article{Burns2026,
author = {Burns, Daniel T and Bice, Channing and Johnson, Paul E and Chia, Nicholas and Robinson, Timothy},
title = {Comparison of Emotional Content in Text Responses From Physicians and AI Chatbots to Patient Health Queries: Cross-Sectional Study},
journal = {Journal of Medical Internet Research},
year = {2026},
doi = {10.2196/85516}
}
EUVIMED