Explainable Machine Learning for Assessing Digital Health Literacy in Older Adults: Validation and Development of a Two-Stage Model Integrating Performance-Based and Self-Assessed Indicators
JMIR Medical Informatics · 2026
DOI: 10.2196/86171 ↗At a glance
Validierungsstudie in JMIR Medical Informatics, 2026. Written by Choonghee Park and 6 co-authors. Institution: Department of Medical Informatics, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea. Major MEDLINE topics: Health Literacy, Machine Learning, Self-Assessment. Cited 4 times so far (Crossref); 78 references. The full text is freely available in PubMed Central.
Topics: Künstliche Intelligenz in der Pflege, Selbstmanagement bei chronischen Erkrankungen, Gesundheitskompetenz
Abstract (original language)
Background
Digital health literacy (DHL) is the ability to locate, understand, evaluate, and apply health information in digital environments. It is essential for older adults to effectively engage with contemporary health care. However, existing DHL assessments primarily rely on self-reported measures, which are susceptible to subjective bias and often fail to capture actual performance. There is a need for a comprehensive, data-driven approach that integrates objective performance indicators with self-assessments to accurately predict and explain DHL levels in older adults.
Objective
This study develops and validates a machine learning approach to predict DHL levels in older adults by integrating performance-based and self-assessed evaluations.
Methods
We applied a 2-stage methodological framework using 2 independent datasets. In the first stage, to identify performance-based determinants, we assessed actual digital and information comprehension in a separate pilot cohort of 30 older adults (aged 60-74 years). In parallel, to measure self-reported DHL, we conducted an online survey with a distinct group of 1000 older adults (aged 55-74 years) using the Digital Health Literacy Scale and the Korean version of the eHealth Literacy Scale (KeHEALS). Bayesian linear regression was applied to both datasets to identify significant explanatory variables. In the second phase, we trained and validated a binary classification model to predict KeHEALS levels using the survey dataset (n=1000), leveraging the features identified in the first stage. Five machine learning algorithms were evaluated, and the best-performing model was interpreted using Shapley Additive Explanations (SHAP) analysis.
Results
In the pilot performance-based assessment, using a greater number of electronic devices and having higher educational attainment were positively associated with comprehension, whereas alcohol intake showed a negative association. In the self-assessed survey data, key correlates included interest in health-related apps, self-care confidence, age, smoking, alcohol intake, number of devices used, and exercise frequency. Among the machine learning models, categorical boosting demonstrated the most balanced performance (accuracy 0.785, precision 0.769, F1-score 0.765, area under the receiver operating characteristic curve 0.835), outperforming the dummy classifier (accuracy 0.540). SHAP analysis indicated that self-care confidence, health information search, interest in health-related apps, number of electronic devices used, and exercise frequency were the strongest positive contributors to high-DHL predictions, whereas older age and lifestyle factors (alcohol intake, smoking) contributed negatively.
Conclusions
By explicitly integrating performance-based and self-assessed indicators within an explainable machine learning framework, this study demonstrates that DHL in older adults is influenced by both digital engagement and health management factors. These findings suggest that the proposed framework can serve as a structured approach for evaluating DHL in older adults and inform the design of personalized digital health interventions in clinical and community settings.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Medical Informatics, College of Medicine, The Catholic University of Korea, Seoul, Republic of Korea.
- The Catholic Medical Center Institute for Basic Medical Science, The Catholic University of Korea Catholic Medical Center, Seoul, Republic of Korea.
- Big Data Research Center, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea.
- Department of Family Medicine, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea.
ORCID profiles: Choonghee Park, Jiyeon Park, Seora Kim, Ye Seul Bae, Jae-Heon Kang, Tae-Min Kim, Ji-Won Chun
Cite
APA
Park, C., Park, J., Kim, S., Bae, Y. S., Kang, J. H., Kim, T. M., & Chun, J. W. (2026). Explainable Machine Learning for Assessing Digital Health Literacy in Older Adults: Validation and Development of a Two-Stage Model Integrating Performance-Based and Self-Assessed Indicators. JMIR Medical Informatics. https://doi.org/10.2196/86171
Vancouver
Park C, Park J, Kim S, Bae YS, Kang JH, Kim TM, et al. Explainable Machine Learning for Assessing Digital Health Literacy in Older Adults: Validation and Development of a Two-Stage Model Integrating Performance-Based and Self-Assessed Indicators. JMIR Medical Informatics. 2026. doi:10.2196/86171
BibTeX
@article{Park2026,
author = {Park, Choonghee and Park, Jiyeon and Kim, Seora and Bae, Ye Seul and Kang, Jae-Heon and Kim, Tae-Min and Chun, Ji-Won},
title = {Explainable Machine Learning for Assessing Digital Health Literacy in Older Adults: Validation and Development of a Two-Stage Model Integrating Performance-Based and Self-Assessed Indicators},
journal = {JMIR Medical Informatics},
year = {2026},
doi = {10.2196/86171}
}
EUVIMED