EUVIMEDEUVIMED entdecken
VergleichsstudieQuerschnittstudieOpen Access · CC BY 4.0

Evaluation of generative AI-driven chatbots as sources of consumer health information on hand, foot, and mouth disease: a cross-sectional comparative study of safety, accuracy, information quality, readability, and empathy

Zhaole Gong, Yan Na, Yi Guo, Xiaoya Zhu, Xiaojie Ma

Frontiers in Public Health · 11.08.2026 · 14

DOI: 10.3389/fpubh.2026.1924162 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Frontiers in Public Health, online erschienen am 11.08.2026. Verfasst von Zhaole Gong und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Pediatrics, Mudanjiang Second People's Hospital, Mudanjiang, Heilongjiang, China. Hauptthemen laut MEDLINE: Hand, Foot and Mouth Disease, Generative Artificial Intelligence, Consumer Health Information. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Generative artificial intelligence chatbots are increasingly used as sources of consumer health information. Although their performance has been examined in several medical conditions, evidence specific to hand, foot, and mouth disease (HFMD) remains limited.

Objective

To compare the safety, accuracy, empathy, information quality, and readability of HFMD-related responses generated by five publicly accessible chatbots.

Methods

In this exploratory cross-sectional study, 20 researcher-developed English-language prompts were constructed from authoritative public-health sources, Google Trends topic mapping, and caregiver-informed wording refinement. ChatGPT-4o, Gemini 2.5 Pro, Copilot, Doubao, and DeepSeek-V3.2-Exp were evaluated between April 2 and April 5, 2026. Five trained reviewers independently assessed responses against predefined reference standards using safety and accuracy criteria, an empathy scale, DISCERN, EQIP, the Global Quality Score, and JAMA benchmarks. Six formula-based readability indices were calculated. Paired comparisons used Friedman and Cochran Q tests, with prespecified post-hoc procedures and Benjamini-Hochberg correction.

Results

Unsafe-response rates ranged from 5.0 to 15.0%, with no statistically significant difference detected among chatbots ( p = 0.797). No statistically significant inter-model differences were detected for accuracy, empathy, DISCERN, EQIP, JAMA, or Global Quality Score in the 20-prompt set; because the study was exploratory and was not powered to establish equivalence, these findings do not demonstrate comparable or interchangeable performance. All six readability indices differed significantly among chatbots ( p = 0.030 to <0.001). ChatGPT and Doubao generally produced lower estimated grade-level complexity than Gemini and DeepSeek. Ten potentially unsafe or misleading responses were identified, mainly involving overgeneralization of EV71 vaccine protection, hand-hygiene qualification, and disinfection advice.

Conclusion

Across a limited set of standardized English-language prompts, the five chatbots often generated coherent HFMD information, but occasional safety-relevant inaccuracies and readability barriers remained. These systems may support general information seeking, but their responses require cautious interpretation and should not replace individualized professional advice.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Hand, Foot and Mouth DiseaseGenerative Artificial IntelligenceConsumer Health InformationCross-Sectional StudiesHumansEmpathyComprehension

Stichwörter der Autor:innen

HFMDand mouth diseaseartificial intelligencechatbotfoothandhealth communicationlarge language modelpediatric infectious diseasereadability

Autor:innen und Institutionen

  • Department of Pediatrics, Mudanjiang Second People's Hospital, Mudanjiang, Heilongjiang, China.
  • Department of Pediatrics, Renmin Hospital of Wuhan University, Wuhan, Hubei, China.

Zitieren

APA

Gong, Z., Na, Y., Guo, Y., Zhu, X., & Ma, X. (2026). Evaluation of generative AI-driven chatbots as sources of consumer health information on hand, foot, and mouth disease: a cross-sectional comparative study of safety, accuracy, information quality, readability, and empathy. Frontiers in Public Health, 14. https://doi.org/10.3389/fpubh.2026.1924162

Vancouver

Gong Z, Na Y, Guo Y, Zhu X, Ma X. Evaluation of generative AI-driven chatbots as sources of consumer health information on hand, foot, and mouth disease: a cross-sectional comparative study of safety, accuracy, information quality, readability, and empathy. Frontiers in Public Health. 2026;14. doi:10.3389/fpubh.2026.1924162

BibTeX
@article{Gong2026,
  author = {Gong, Zhaole and Na, Yan and Guo, Yi and Zhu, Xiaoya and Ma, Xiaojie},
  title = {Evaluation of generative AI-driven chatbots as sources of consumer health information on hand, foot, and mouth disease: a cross-sectional comparative study of safety, accuracy, information quality, readability, and empathy},
  journal = {Frontiers in Public Health},
  year = {2026},
  volume = {14},
  doi = {10.3389/fpubh.2026.1924162}
}
Kostenlos starten →