Integrating economic considerations into cutpoint selection may help align clinical decision support toward value-based healthcare
Journal of the American Medical Informatics Association · 19.05.2023 · 30 (6) · 1103-1113
DOI: 10.1093/jamia/ocad042 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 25.03.2023. Verfasst von Rex Parsons und 3 weiteren Autor:innen. Beteiligte Einrichtung: Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health…. Hauptthemen laut MEDLINE: Decision Support Systems, Clinical. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Digital Health Cooperative Research Centre, Commonwealth’s Cooperative Research Centres, NHMRC.
Abstract (Originalsprache)
Objective
Clinical prediction models providing binary categorizations for clinical decision support require the selection of a probability threshold, or “cutpoint,” to classify individuals. Existing cutpoint selection approaches typically optimize test-specific metrics, including sensitivity and specificity, but overlook the consequences of correct or incorrect classification. We introduce a new cutpoint selection approach considering downstream consequences using net monetary benefit (NMB) and through simulations compared it with alternative approaches in 2 use-cases: (i) preventing intensive care unit readmission and (ii) preventing inpatient falls. Materials and methods Parameter estimates for costs and effectiveness from prior studies were included in Monte Carlo simulations. For each use-case, we simulated the expected NMB resulting from the model-guided decision using a range of cutpoint selection approaches, including our new value-optimizing approach. Sensitivity analyses applied alternative event rates, model discrimination, and calibration performance.
Results
The proposed approach that considered expected downstream consequences was frequently NMB-maximizing compared with other methods. Sensitivity analysis demonstrated that it was or closely tracked the optimal strategy under a range of scenarios. Under scenarios of relatively low event rates and discrimination that may be considered realistic for intensive care (prevalence = 0.025, area under the receiver operating characteristic curve [AUC] = 0.70) and falls (prevalence = 0.036, AUC = 0.70), our proposed cutpoint method was either the best or similar to the best of the compared methods regarding NMB, and was robust to model miscalibration.
Discussion
Our results highlight the potential value of conditioning cutpoints on the implementation setting, particularly for rare and costly events, which are often the target of prediction model development research.
Conclusions
This study proposes a cutpoint selection method that may optimize clinical decision support systems toward value-based care.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Australian Centre for Health Services Innovation and Centre for Healthcare Transformation, School of Public Health and Social Work, Faculty of Health, Queensland University of Technology, Kelvin Grove, Australia.
- Jamieson Trauma Institute, Royal Brisbane and Women's Hospital, Metro North Health, Herston, Australia.
- Digital Health and Informatics, Metro South Health, Woolloongabba, Australia.
ORCID-Profile: Rex Parsons, Robin Blythe
Förderung
- Digital Health Cooperative Research Centre
- Commonwealth’s Cooperative Research Centres
- NHMRC
Zitieren
APA
Parsons, R., Blythe, R., Cramb, S. M., & McPhail, S. M. (2023). Integrating economic considerations into cutpoint selection may help align clinical decision support toward value-based healthcare. Journal of the American Medical Informatics Association, 30(6), 1103-1113. https://doi.org/10.1093/jamia/ocad042
Vancouver
Parsons R, Blythe R, Cramb SM, McPhail SM. Integrating economic considerations into cutpoint selection may help align clinical decision support toward value-based healthcare. Journal of the American Medical Informatics Association. 2023;30(6):1103-1113. doi:10.1093/jamia/ocad042
BibTeX
@article{Parsons2023,
author = {Parsons, Rex and Blythe, Robin and Cramb, Susanna M and McPhail, Steven M},
title = {Integrating economic considerations into cutpoint selection may help align clinical decision support toward value-based healthcare},
journal = {Journal of the American Medical Informatics Association},
year = {2023},
volume = {30},
number = {6},
pages = {1103--1113},
doi = {10.1093/jamia/ocad042}
}
EUVIMED