EUVIMEDSkopri EUVIMED
Open access

Predicting Early Dropout in a Digital Tobacco Cessation Intervention: Replication and Extension Study

Linda Q Yu iD, Michael S Amato iD, George D Papandonatos iD, Sarah Cha iD, Amanda L Graham iD

Journal of Medical Internet Research · 2024

DOI: 10.2196/54248 ↗

At a glance

Journal article in Journal of Medical Internet Research, 2024. Written by Linda Q Yu and 4 co-authors. Institution: Innovations Center, Truth Initiative, Washington, DC, United States. Major MEDLINE topics: Patient Dropouts. Cited 1 times so far (Crossref); 32 references. The full text is freely available in PubMed Central. Funded by: NIDA NIH HHS, NCI NIH HHS, NIMH NIH HHS.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Background

Detecting early dropout from digital interventions is crucial for developing strategies to enhance user retention and improve health-related behavioral outcomes. Bricker and colleagues proposed a single metric that accurately predicted early dropout from 4 digital tobacco cessation interventions based on log-in data in the initial week after registration. Generalization of this method to additional interventions and modalities would strengthen confidence in the approach and facilitate additional research drawing on it to increase user retention.

Objective

This study had two research questions (RQ): RQ1—can the study by Bricker and colleagues be replicated using data from a large-scale observational, multimodal intervention to predict early dropout? and RQ2—can first-week engagement patterns identify users at the greatest risk for early dropout, to inform development of potential “rescue” interventions?

Methods

Data from web users were drawn from EX, a freely available, multimodal digital intervention for tobacco cessation (N=70,265). First-week engagement was operationalized as any website page views or SMS text message responses within 1 week after registration. Early dropout was defined as having no subsequent engagement after that initial week through 1 year. First, a multivariate regression model was used to predict early dropout. Model predictors were dichotomous measures of engagement in each of the initial 6 days (days 2-7) following registration (day 1). Next, 6 univariate regression models were compared in terms of their discrimination ability to predict early dropout. The sole predictor of each model was a dichotomous measure of whether users had reengaged with the intervention by a particular day of the first week (calculated separately for each of 2-7 days).

Results

For RQ1, the area under the receiver operating characteristic curve (AUC) of the multivariate model in predicting dropout after 1 week was 0.72 (95% CI 0.71-0.73), which was within the range of AUC metrics found in the study by Bricker and colleagues. For RQ2, the AUCs of the univariate models increased with each successive day until day 4 (0.66, 95% CI 0.65-0.67). The sensitivity of the models decreased (range 0.79-0.59) and the specificity increased (range 0.48-0.73) with each successive day.

Conclusions

This study provides independent validation of the use of first-week engagement to predict early dropout, demonstrating that the method generalizes across intervention modalities and engagement metrics. As digital intervention researchers continue to address the challenges of low engagement and early dropout, these results suggest that first-week engagement is a useful construct with predictive validity that is robust across interventions and definitions. Future research should explore the applicability and efficiency of this model to develop interventions to increase retention and improve health behavioral outcomes.

Subjects

MeSH terms (MEDLINE)

Patient DropoutsHumansTobacco Use CessationMaleFemaleAdultMiddle AgedText MessagingSmoking CessationInternet

Author keywords

attritioncessationdigital interventionsdropoutengagementinternetmobile healthmobile phonesmokingtobacco

Authors and institutions

  • Innovations Center, Truth Initiative, Washington, DC, United States.
  • Department of Medicine, Mayo Clinic College of Medicine and Science, Rochester, MN, United States.
  • Center for Statistical Sciences, School of Public Health, Brown University, Providence, RI, United States.
  • Department of Oncology, Georgetown University Medical Center, Washington, DC, United States.

ORCID profiles: Linda Q Yu, Michael S Amato, George D Papandonatos, Sarah Cha, Amanda L Graham

Funding

  • NIDA NIH HHS
  • NCI NIH HHS
  • NIMH NIH HHS

Cite

APA

Yu, L. Q., Amato, M. S., Papandonatos, G. D., Cha, S., & Graham, A. L. (2024). Predicting Early Dropout in a Digital Tobacco Cessation Intervention: Replication and Extension Study. Journal of Medical Internet Research. https://doi.org/10.2196/54248

Vancouver

Yu LQ, Amato MS, Papandonatos GD, Cha S, Graham AL. Predicting Early Dropout in a Digital Tobacco Cessation Intervention: Replication and Extension Study. Journal of Medical Internet Research. 2024. doi:10.2196/54248

BibTeX
@article{Yu2024,
  author = {Yu, Linda Q and Amato, Michael S and Papandonatos, George D and Cha, Sarah and Graham, Amanda L},
  title = {Predicting Early Dropout in a Digital Tobacco Cessation Intervention: Replication and Extension Study},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  doi = {10.2196/54248}
}
Start for free →