EUVIMEDObjevte EUVIMED
ValidierungsstudieOpen access

Explainable Machine Learning for Assessing Digital Health Literacy in Older Adults: Validation and Development of a Two-Stage Model Integrating Performance-Based and Self-Assessed Indicators

Choonghee Park iD, Jiyeon Park iD, Seora Kim iD, Ye Seul Bae iD, Jae-Heon Kang iD, Tae-Min Kim iD, Ji-Won Chun iD

JMIR Medical Informatics · 2026

DOI: 10.2196/86171 ↗

At a glance

Validierungsstudie in JMIR Medical Informatics, 2026. Written by Choonghee Park and 6 co-authors. Institution: Department of Medical Informatics, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea. Major MEDLINE topics: Health Literacy, Machine Learning, Self-Assessment. Cited 4 times so far (Crossref); 78 references. The full text is freely available in PubMed Central.

AbstraktSubjectsBibliografické údajeCiteRelated studies

Abstract (original language)

Background

Digital health literacy (DHL) is the ability to locate, understand, evaluate, and apply health information in digital environments. It is essential for older adults to effectively engage with contemporary health care. However, existing DHL assessments primarily rely on self-reported measures, which are susceptible to subjective bias and often fail to capture actual performance. There is a need for a comprehensive, data-driven approach that integrates objective performance indicators with self-assessments to accurately predict and explain DHL levels in older adults.

Objective

This study develops and validates a machine learning approach to predict DHL levels in older adults by integrating performance-based and self-assessed evaluations.

Methods

We applied a 2-stage methodological framework using 2 independent datasets. In the first stage, to identify performance-based determinants, we assessed actual digital and information comprehension in a separate pilot cohort of 30 older adults (aged 60-74 years). In parallel, to measure self-reported DHL, we conducted an online survey with a distinct group of 1000 older adults (aged 55-74 years) using the Digital Health Literacy Scale and the Korean version of the eHealth Literacy Scale (KeHEALS). Bayesian linear regression was applied to both datasets to identify significant explanatory variables. In the second phase, we trained and validated a binary classification model to predict KeHEALS levels using the survey dataset (n=1000), leveraging the features identified in the first stage. Five machine learning algorithms were evaluated, and the best-performing model was interpreted using Shapley Additive Explanations (SHAP) analysis.

Results

In the pilot performance-based assessment, using a greater number of electronic devices and having higher educational attainment were positively associated with comprehension, whereas alcohol intake showed a negative association. In the self-assessed survey data, key correlates included interest in health-related apps, self-care confidence, age, smoking, alcohol intake, number of devices used, and exercise frequency. Among the machine learning models, categorical boosting demonstrated the most balanced performance (accuracy 0.785, precision 0.769, F1-score 0.765, area under the receiver operating characteristic curve 0.835), outperforming the dummy classifier (accuracy 0.540). SHAP analysis indicated that self-care confidence, health information search, interest in health-related apps, number of electronic devices used, and exercise frequency were the strongest positive contributors to high-DHL predictions, whereas older age and lifestyle factors (alcohol intake, smoking) contributed negatively.

Conclusions

By explicitly integrating performance-based and self-assessed indicators within an explainable machine learning framework, this study demonstrates that DHL in older adults is influenced by both digital engagement and health management factors. These findings suggest that the proposed framework can serve as a structured approach for evaluating DHL in older adults and inform the design of personalized digital health interventions in clinical and community settings.

Subjects

MeSH terms (MEDLINE)

Health LiteracyMachine LearningSelf-AssessmentHumansAgedDigital HealthFemaleMaleMiddle AgedSurveys and QuestionnairesPredictive Learning ModelsRepublic of Korea

Author keywords

digital health caredigital health literacyeHealth literacymHealthmachine learning

Authors and institutions

  • Department of Medical Informatics, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea.
  • The Catholic Medical Center Institute for Basic Medical Science, The Catholic University of Korea Catholic Medical Center, Seoul, Republic of Korea.
  • Big Data Research Center, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea.
  • Department of Family Medicine, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea.

ORCID profiles: Choonghee Park, Jiyeon Park, Seora Kim, Ye Seul Bae, Jae-Heon Kang, Tae-Min Kim, Ji-Won Chun

Cite

APA

Park, C., Park, J., Kim, S., Bae, Y. S., Kang, J. H., Kim, T. M., & Chun, J. W. (2026). Explainable Machine Learning for Assessing Digital Health Literacy in Older Adults: Validation and Development of a Two-Stage Model Integrating Performance-Based and Self-Assessed Indicators. JMIR Medical Informatics. https://doi.org/10.2196/86171

Vancouver

Park C, Park J, Kim S, Bae YS, Kang JH, Kim TM, et al. Explainable Machine Learning for Assessing Digital Health Literacy in Older Adults: Validation and Development of a Two-Stage Model Integrating Performance-Based and Self-Assessed Indicators. JMIR Medical Informatics. 2026. doi:10.2196/86171

BibTeX
@article{Park2026,
  author = {Park, Choonghee and Park, Jiyeon and Kim, Seora and Bae, Ye Seul and Kang, Jae-Heon and Kim, Tae-Min and Chun, Ji-Won},
  title = {Explainable Machine Learning for Assessing Digital Health Literacy in Older Adults: Validation and Development of a Two-Stage Model Integrating Performance-Based and Self-Assessed Indicators},
  journal = {JMIR Medical Informatics},
  year = {2026},
  doi = {10.2196/86171}
}
Start for free →