EUVIMEDOpdag EUVIMED
Open access

Human-Delivered Conversation Versus AI Chatbot Conversation in Increasing Heart Attack Knowledge in Women in the United States: Quasi-Experimental Studies

Diane Dagyong Kim, Jingwen Zhang, Kenji Sagae, Holli A Devon, Thomas J Hoffmann, Lauren Rountree, Yoshimi Fukuoka

Journal of Medical Internet Research · 2025-10-17 · 27 · e73184-e73184

DOI: 10.2196/73184 ↗

At a glance

Journal article in Journal of Medical Internet Research, published online on 2025-10-17. Written by Diane Dagyong Kim and 6 co-authors. Institution: Department of Communication, University of California, Davis, 177 Kerr Hall, Davis, CA, 95616, United States, 1 530-754-1472. Major MEDLINE topics: Myocardial Infarction, Artificial Intelligence, Communication, Health Knowledge, Attitudes, Practice. Cited 5 times so far (Crossref); 41 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Artificial intelligence (AI) chatbots, driven by advances in natural language processing, can analyze and generate human language through computational linguistics and machine learning. Despite the rapid development of large language models, little investigation has been conducted to assess whether AI chatbot-delivered educational conversations can achieve a similar level of efficacy as human-delivered conversations.

Objective

This study aims to evaluate and explore the potential efficacy of human-delivered conversations versus AI chatbot conversations in increasing women’s knowledge and awareness of symptoms and response to a heart attack in the United States.

Methods

This is a secondary analysis of 2 datasets collected from the AI Chatbot Development Project. Women aged 25 years or older were recruited through flyers and social media. The first dataset contained conversational data where a research interventionist engaged in educational conversations with participants (human dataset), whereas the second dataset contained conversational data where an AI chatbot named HeartBot engaged in the same educational conversations with participants (HeartBot dataset). Knowledge and awareness of symptoms and response to a heart attack were measured at the pre- and post-interaction with either the human or HeartBot. Perceived message effectiveness and conversational quality were measured at the post-survey. Ordinal logistic regression analyses were conducted to explore factors predicting participants’ knowledge, adjusting for age, race or ethnicity, intervention group type, education, word count, message effectiveness, and message humanness.

Results

A total of 171 participants (mean age=41.06 y, SD=12.08) in the Human dataset and 92 participants (mean age=45.85 y, SD=11.94) in the HeartBot dataset completed the study. Both human-delivered conversations and HeartBot conversations were associated with significant improvements in participants’ ability to recognize heart attack symptoms (adjusted odds ratio [AOR] 15.19, 95% CI 8.46‐27.25, P

Subjects

MeSH terms (MEDLINE)

Myocardial InfarctionArtificial IntelligenceCommunicationHealth Knowledge, Attitudes, PracticeHumansFemaleUnited StatesMiddle AgedAdultGenerative Artificial Intelligence

Author keywords

artificial intelligencechatbotconversational agenthealth educationheart attackheart diseaseknowledgelarge language modelnatural language processingwomen

Authors and institutions

  • Department of Communication, University of California, Davis, 177 Kerr Hall, Davis, CA, 95616, United States, 1 530-754-1472.
  • Department of Communication, Department of Public Health Sciences, University of California, Davis, Davis, CA, United States.
  • Department of Linguistics, University of California, Davis, Davis, CA, United States.
  • UCLA School of Nursing, University of California, Los Angeles, Los Angeles, CA, United States.
  • Department of Epidemiology & Biostatistics, University of California, San Francisco, San Francisco, CA, United States.
  • Massachusetts General Hospital, Boston, MA, United States.

Cite

APA

Kim, D. D., Zhang, J., Sagae, K., Devon, H. A., Hoffmann, T. J., Rountree, L., & Fukuoka, Y. (2025). Human-Delivered Conversation Versus AI Chatbot Conversation in Increasing Heart Attack Knowledge in Women in the United States: Quasi-Experimental Studies. Journal of Medical Internet Research, 27, e73184-e73184. https://doi.org/10.2196/73184

Vancouver

Kim DD, Zhang J, Sagae K, Devon HA, Hoffmann TJ, Rountree L, et al. Human-Delivered Conversation Versus AI Chatbot Conversation in Increasing Heart Attack Knowledge in Women in the United States: Quasi-Experimental Studies. Journal of Medical Internet Research. 2025;27:e73184-e73184. doi:10.2196/73184

BibTeX
@article{Kim2025,
  author = {Kim, Diane Dagyong and Zhang, Jingwen and Sagae, Kenji and Devon, Holli A and Hoffmann, Thomas J and Rountree, Lauren and Fukuoka, Yoshimi},
  title = {Human-Delivered Conversation Versus AI Chatbot Conversation in Increasing Heart Attack Knowledge in Women in the United States: Quasi-Experimental Studies},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e73184--e73184},
  doi = {10.2196/73184}
}
Start for free →