EUVIMEDDescoperă EUVIMED
Open access

Examining Artificial Intelligence Chatbots’ Responses in Providing Human Papillomavirus Vaccine Information for Young Adults: Qualitative Content Analysis

Alfu Laily iD, Laura M Schwab-Reese iD, Megan Davish iD, Emily Cahue iD, Kathryn J LaRoche iD, Natalia M Rodriguez iD, Robert J Duncan iD, Randolph D Hubach iD, Monica L Kasting iD

JMIR Public Health and Surveillance · 2026

DOI: 10.2196/79720 ↗

At a glance

Journal article in JMIR Public Health and Surveillance, 2026. Written by Alfu Laily and 8 co-authors. Institution: Department of Public Health, Purdue University, West Lafayette, IN, United States. Major MEDLINE topics: Papillomavirus Vaccines, Artificial Intelligence. 89 references. The full text is freely available in PubMed Central.

RezumatSubjectsDate bibliograficeCiteRelated studies

Abstract (original language)

Background

The growing use of artificial intelligence (AI) chatbots for seeking health-related information is concerning, as they were not originally developed for delivering medical guidance. The quality of AI chatbots’ responses relies heavily on their training data and is often limited in medical contexts due to their lack of specific training data in medical literature. Findings on the quality of AI chatbot responses related to health are mixed. Some studies showed the quality surpassed physicians’ responses, while others revealed occasional major errors and low readability. This study addresses a critical gap by examining the performance of various AI chatbots in a complex, misinformation-rich environment.

Objective

This study examined AI chatbots' responses to human papillomavirus (HPV)–related questions by analyzing structure, linguistic features, information accuracy and currency, and vaccination stance.

Methods

We conducted a qualitative content analysis following the approach outlined by Schreier to examine 4 selected AI chatbots’ (ChatGPT 4, Claude 3.7 Sonnet, DeepSeek V3, and Docus [General AI Doctor]) responses to HPV vaccine questions. These questions, simulated by young adults, were adapted from items on the Vaccine Conspiracy Beliefs Scale and Google Trends. The selection criteria for AI chatbots included popularity, accessibility, countries of origin, response update methods, and intended use. Two researchers, simulating a 22-year-old man or woman, collected 8 conversations between February 22 and 28, 2025. We used a deductive approach to develop initial code groups, then an inductive approach to generate codes. The responses were analyzed based on a comprehensive codebook, with codes examining response structure, linguistic features, information accuracy and currency, and vaccination stance. We also assessed readability using the Flesch-Kincaid Grade Level and Reading Ease Score.

Results

All AI chatbots cited evidence-based sources from reputable health organizations. We found no fabricated information or inaccuracies in numerical data. For complex questions, all AI chatbots appropriately deferred to health care professionals’ suggestions. All AI chatbots maintained a neutral or provaccine stance, corresponding with scientific consensus. The mean and range of response lengths varied [word count; ChatGPT: 436.4 (218-954); Claude: 188.0 (138-255); DeepSeek: 510.0 (325-735); and Docus: 159.4 (61-200)], as did readability [Flesch-Kincaid Grade Level; ChatGPT: 10.7 (6.0-14.9); Claude: 13.2 (7.7-17.8); DeepSeek: 11.3 (7.0-14.7); and Docus: 12.2 (8.9-15.5); and Flesch-Kincaid Reading Ease Score; ChatGPT: 46.8 (25.4-72.2); Claude: 32.5 (6.3-67.3); DeepSeek: 43.7 (22.8-67.4); and Docus: 40.5 (19.6-58.2)]. ChatGPT and Claude offered personalized responses, while DeepSeek and Docus lacked this. Occasionally, some responses included broken or irrelevant links and medical jargon.

Conclusions

Amidst an online environment saturated with misinformation, AI chatbots have the potential to serve as an alternative source of accurate HPV-related information to conventional online platforms (websites and social media). Improvements in readability, personalization, and link accuracy are still needed. Furthermore, we recommend that users treat AI chatbots as complements, not replacements, to health care professionals’ guidance on clinical settings.

Subjects

MeSH terms (MEDLINE)

Papillomavirus VaccinesArtificial IntelligenceHumansQualitative ResearchPapillomavirus InfectionsYoung AdultFemaleMale

Author keywords

artificial intelligencehealth communicationlarge language modelspapillomavirus vaccinesqualitative research

Substances

Papillomavirus Vaccines

Authors and institutions

  • Department of Public Health, Purdue University, West Lafayette, IN, United States.
  • Cancer Prevention and Control Program, Indiana University Simon Comprehensive Cancer Center, Indianapolis, IN, United States.
  • Human Development and Family Studies, Colorado State University, Fort Collins, CO, United States.

ORCID profiles: Alfu Laily, Laura M Schwab-Reese, Megan Davish, Emily Cahue, Kathryn J LaRoche, Natalia M Rodriguez, Robert J Duncan, Randolph D Hubach, Monica L Kasting

Cite

APA

Laily, A., Schwab-Reese, L. M., Davish, M., Cahue, E., LaRoche, K. J., Rodriguez, N. M., Duncan, R. J., Hubach, R. D., & Kasting, M. L. (2026). Examining Artificial Intelligence Chatbots’ Responses in Providing Human Papillomavirus Vaccine Information for Young Adults: Qualitative Content Analysis. JMIR Public Health and Surveillance. https://doi.org/10.2196/79720

Vancouver

Laily A, Schwab-Reese LM, Davish M, Cahue E, LaRoche KJ, Rodriguez NM, et al. Examining Artificial Intelligence Chatbots’ Responses in Providing Human Papillomavirus Vaccine Information for Young Adults: Qualitative Content Analysis. JMIR Public Health and Surveillance. 2026. doi:10.2196/79720

BibTeX
@article{Laily2026,
  author = {Laily, Alfu and Schwab-Reese, Laura M and Davish, Megan and Cahue, Emily and LaRoche, Kathryn J and Rodriguez, Natalia M and Duncan, Robert J and Hubach, Randolph D and Kasting, Monica L},
  title = {Examining Artificial Intelligence Chatbots’ Responses in Providing Human Papillomavirus Vaccine Information for Young Adults: Qualitative Content Analysis},
  journal = {JMIR Public Health and Surveillance},
  year = {2026},
  doi = {10.2196/79720}
}
Start for free →