EUVIMEDΑνακαλύψτε το EUVIMED
VergleichsstudieQuerschnittstudieOpen access

Comparison of Emotional Content in Text Responses From Physicians and AI Chatbots to Patient Health Queries: Cross-Sectional Study

Daniel T Burns iD, Channing Bice iD, Paul E Johnson iD, Nicholas Chia iD, Timothy Robinson iD

Journal of Medical Internet Research · 2026

DOI: 10.2196/85516 ↗

At a glance

Vergleichsstudie in Journal of Medical Internet Research, 2026. Written by Daniel T Burns and 4 co-authors. Institution: Department of Mathematics & Statistics, University of Wyoming, Laramie, WY, United States. Major MEDLINE topics: Physicians, Emotions, Artificial Intelligence. Cited 2 times so far (Crossref); 74 references. The full text is freely available in PubMed Central. Funded by: NIGMS NIH HHS.

ΠερίληψηSubjectsΒιβλιογραφικά στοιχείαCiteRelated studies

Abstract (original language)

Background

Surveys show that many people are willing to use generative artificial intelligence (AI) for health questions. Prior research has largely focused on chatbot accuracy, with some studies finding that both physicians and consumers overwhelmingly prefer chatbot-generated text over physician responses.

Objective

This study aimed to characterize and compare the emotional content of responses from physicians and 2 AI chatbots (OpenAI’s ChatGPT and Google’s Gemini) and to assess differences in reading level and use of medical disclaimers.

Methods

A public, patient-deidentified telehealth website was used to compile 100 physician-answered questions. The same questions were posed to both chatbots between May 18 and 19, 2025. Two coders classified the emotional content of each sentence using a predefined codebook and reviewed for agreement. Emotions were ranked as primary, secondary, and tertiary by the proportion of sentences classified as each emotion per response. Multinomial logistic regression compared emotional rankings using physician responses as the reference. Word count, Flesch Reading Ease, and Flesch-Kincaid Grade Level were analyzed via ANOVA with the Tukey honestly significant difference test. Disclaimer use was compared between chatbots using a χ2 test.

Results

Primary emotions were overwhelmingly neutral, except for one response from each chatbot in which anger was primary. For secondary emotions, the odds ratio of hope was 80.28% (95% CI 37.71%-93.76%) lower for ChatGPT, while the odds ratio of fear was 3.29 (95% CI 1.44-7.49) times higher for Gemini. For tertiary emotions, the odds ratio of compassion was 1.94 (95% CI 1.06-3.54) times higher, and the odds ratio of having no tertiary emotion was 84.33% (95% CI 64.72%-93.04%) lower for Gemini. Gemini responses averaged 889.1 (SD 305.7) words, ChatGPT 476.5 (SD 109.5), and physicians 193.5 (SD 113.6). Gemini had the lowest average Flesch Reading Ease score at 39.9 (SD 8.8), followed by ChatGPT at 45.8 (SD 12.8), while physicians had the highest at 51.9 (SD 13.6). Gemini had the highest average Flesch-Kincaid Grade Level at 11.3 (SD 1.5), followed by ChatGPT at 9.9 (SD 1.9), and physicians at 9.2 (SD 2.4). Gemini was significantly more likely to include a disclaimer than ChatGPT (χ21=49.2; P

Subjects

MeSH terms (MEDLINE)

PhysiciansEmotionsArtificial IntelligenceHumansCross-Sectional StudiesGenerative Artificial IntelligenceTelemedicine

Author keywords

AIChatGPTartificial intelligencechatbotemotionalhealthphysiciansresponses

Authors and institutions

  • Department of Mathematics & Statistics, University of Wyoming, Laramie, WY, United States.
  • Department of Journalism & Media Communication, Colorado State University, Fort Collins, CO, United States.
  • Department of Otolaryngology- Head and Neck Surgery, University of Washington, Seattle, WA, United States.
  • Data Science and Learning, Argonne National Laboratory, Lemont, IL, United States.

ORCID profiles: Daniel T Burns, Channing Bice, Paul E Johnson, Nicholas Chia, Timothy Robinson

Funding

  • NIGMS NIH HHS

Cite

APA

Burns, D. T., Bice, C., Johnson, P. E., Chia, N., & Robinson, T. (2026). Comparison of Emotional Content in Text Responses From Physicians and AI Chatbots to Patient Health Queries: Cross-Sectional Study. Journal of Medical Internet Research. https://doi.org/10.2196/85516

Vancouver

Burns DT, Bice C, Johnson PE, Chia N, Robinson T. Comparison of Emotional Content in Text Responses From Physicians and AI Chatbots to Patient Health Queries: Cross-Sectional Study. Journal of Medical Internet Research. 2026. doi:10.2196/85516

BibTeX
@article{Burns2026,
  author = {Burns, Daniel T and Bice, Channing and Johnson, Paul E and Chia, Nicholas and Robinson, Timothy},
  title = {Comparison of Emotional Content in Text Responses From Physicians and AI Chatbots to Patient Health Queries: Cross-Sectional Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  doi = {10.2196/85516}
}
Start for free →