Assessing the Adherence of ChatGPT Chatbots to Public Health Guidelines for Smoking Cessation: Content Analysis
Journal of Medical Internet Research · 30.01.2025 · 27 · e66896
DOI: 10.2196/66896 ↗Kurz gefasst
Leitlinie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 30.01.2025. Verfasst von Lorien C Abroms und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Prevention & Community Health, Milken Institute School of Public Health, George Washington University, Washington, DC, United States. Hauptthemen laut MEDLINE: Smoking Cessation, Public Health, Guideline Adherence. Die Arbeit wurde bisher 26-mal zitiert (Crossref) und nennt 18 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Large language model (LLM) artificial intelligence chatbots using generative language can offer smoking cessation information and advice. However, little is known about the reliability of the information provided to users.
Objective
This study aims to examine whether 3 ChatGPT chatbots—the World Health Organization’s Sarah, BeFreeGPT, and BasicGPT—provide reliable information on how to quit smoking.
Methods
A list of quit smoking queries was generated from frequent quit smoking searches on Google related to “how to quit smoking” (n=12). Each query was given to each chatbot, and responses were analyzed for their adherence to an index developed from the US Preventive Services Task Force public health guidelines for quitting smoking and counseling principles. Responses were independently coded by 2 reviewers, and differences were resolved by a third coder.
Results
Across chatbots and queries, on average, chatbot responses were rated as being adherent to 57.1% of the items on the adherence index. Sarah’s adherence (72.2%) was significantly higher than BeFreeGPT (50%) and BasicGPT (47.8%; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Prevention & Community Health, Milken Institute School of Public Health, George Washington University, Washington, DC, United States.
- Department of Engineering Management and Systems Engineering, George Washington University, Washington, DC, United States.
- Department of Population Health, Grossman School of Medicine, New York University, New York, NY, United States.
ORCID-Profile: Lorien C Abroms, Artin Yousefi, Christina N Wysota, Tien-Chin Wu, David A Broniatowski
Zitieren
APA
Abroms, L. C., Yousefi, A., Wysota, C. N., Wu, T. C., & Broniatowski, D. A. (2025). Assessing the Adherence of ChatGPT Chatbots to Public Health Guidelines for Smoking Cessation: Content Analysis. Journal of Medical Internet Research, 27, e66896. https://doi.org/10.2196/66896
Vancouver
Abroms LC, Yousefi A, Wysota CN, Wu TC, Broniatowski DA. Assessing the Adherence of ChatGPT Chatbots to Public Health Guidelines for Smoking Cessation: Content Analysis. Journal of Medical Internet Research. 2025;27:e66896. doi:10.2196/66896
BibTeX
@article{Abroms2025,
author = {Abroms, Lorien C and Yousefi, Artin and Wysota, Christina N and Wu, Tien-Chin and Broniatowski, David A},
title = {Assessing the Adherence of ChatGPT Chatbots to Public Health Guidelines for Smoking Cessation: Content Analysis},
journal = {Journal of Medical Internet Research},
year = {2025},
volume = {27},
pages = {e66896},
doi = {10.2196/66896}
}
EUVIMED