Comparing a Large Language Model to Human-Generated Retention Messages for a Family Healthy Weight Program
American Journal of Health Promotion · 29.07.2026
DOI: 10.1177/08901171261472636 ↗Kurz gefasst
Fachartikel in der Zeitschrift American Journal of Health Promotion, online erschienen am 29.07.2026. Verfasst von Kayla Norton und 7 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health and Kinesiology, University of Utah, Salt Lake City, UT, USA. Die Arbeit nennt 20 Literaturangaben. Gefördert von: Centers for Disease Control and Prevention Foundation.
Abstract (Originalsprache)
Background
Text messaging can improve attendance and retention in community health programs; however, message development can be resource intensive.
Purpose
To compare human and large language model (LLM)-generated retention messages in terms of creation time, clarity, appropriateness, and alignment with behavior change principles.
Design
Mixed methods using expert message ratings and qualitative feedback from message developers.
Setting
Building Healthy Families, a family healthy weight program. Sample Experts in behavioral science or related fields (n = 21) and message developers (n = 3). Measures Matched message pairs were rated on a 5-point Likert scale for clarity, appropriateness, and alignment with Social Cognitive Theory principles. Analysis Wilcoxon signed-rank tests compared message ratings. Equivalence testing using a ±10% equivalence interval and 90% confidence intervals assessed similarity between message types.
Results
LLM-generated messages yielded significantly higher ratings than human-generated messages for 4 of 5 message pairs on clarity and 3 of 5 rated on appropriateness (all P ’s < 0.05). Messages were statistically equivalent for 3 of 5 message pairs rated for behavior change theory alignment (all P ’s < 0.05). Time to develop the human- and LLM-generated messages was approximately the same with similar averaged Flesch-Kincaid grade level scores. However, the LLM-generated messages were shorter and more concise.
Conclusion
This study suggests the potential for LLMs to create messages more efficiently and reduce, but not eliminate, workload for health promotion professionals.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Health and Kinesiology, University of Utah, Salt Lake City, UT, USA.
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, USA.
- Department of Population Health Sciences, University of Utah, Salt Lake City, UT, USA.
- Department of Biology, University of Nebraska at Kearney, Kearney, NE, USA.
ORCID-Profile: Kayla Norton, Ryan D Burns, Paul A Estabrooks
Förderung
- Centers for Disease Control and Prevention Foundation
Zitieren
APA
Norton, K., Burns, R. D., Del Fiol, G., Hill, J. L., Heelan, K., Golden, C. A., Mercado, A., & Estabrooks, P. A. (2026). Comparing a Large Language Model to Human-Generated Retention Messages for a Family Healthy Weight Program. American Journal of Health Promotion. https://doi.org/10.1177/08901171261472636
Vancouver
Norton K, Burns RD, Del Fiol G, Hill JL, Heelan K, Golden CA, et al. Comparing a Large Language Model to Human-Generated Retention Messages for a Family Healthy Weight Program. American Journal of Health Promotion. 2026. doi:10.1177/08901171261472636
BibTeX
@article{Norton2026,
author = {Norton, Kayla and Burns, Ryan D and Del Fiol, Guilherme and Hill, Jennie L and Heelan, Kate and Golden, Caitlin A and Mercado, Ali and Estabrooks, Paul A},
title = {Comparing a Large Language Model to Human-Generated Retention Messages for a Family Healthy Weight Program},
journal = {American Journal of Health Promotion},
year = {2026},
doi = {10.1177/08901171261472636}
}
EUVIMED