Machine learning models’ assessment: trust and performance
Medical & Biological Engineering & Computing · 2024
DOI: 10.1007/s11517-024-03145-5 ↗Kurz gefasst
Fachartikel in der Zeitschrift Medical & Biological Engineering & Computing, 2024. Verfasst von S Sousa und 5 weiteren Autor:innen. Beteiligte Einrichtung: CISUC, Center for Informatics and Systems of University of Coimbra, University of Coimbra, Pólo II, 3030-290, Coimbra, Portugal. Hauptthemen laut MEDLINE: Machine Learning, Trust. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 22 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Instituto Politécnico de Coimbra.
Abstract (Originalsprache)
The common black box nature of machine learning models is an obstacle to their application in health care context. Their widespread application is limited by a significant “lack of trust.” So, the main goal of this work is the development of an evaluation approach that can assess, simultaneously, trust and performance. Trust assessment is based on (i) model robustness (stability assessment), (ii) confidence (95% CI of geometric mean), and (iii) interpretability (comparison of respective features ranking with clinical evidence). Performance is assessed through geometric mean. For validation, in patients’ stratification in cardiovascular risk assessment, a Portuguese dataset ( N =1544) was applied. Five different models were compared: (i) GRACE score, the most common risk assessment tool in Portugal for patients with acute coronary syndrome; (ii) logistic regression; (iii) Naïve Bayes; (iv) decision trees; and (v) rule-based approach, previously developed by this team. The obtained results confirm that the simultaneous assessment of trust and performance can be successfully implemented. The rule-based approach seems to have potential for clinical application. It provides a high level of trust in the respective operation while outperformed the GRACE model’s performance, enhancing the required physicians’ acceptance. This may increase the possibility to effectively aid the clinical decision. Graphical abstract
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- CISUC, Center for Informatics and Systems of University of Coimbra, University of Coimbra, Pólo II, 3030-290, Coimbra, Portugal.
- CISUC, Center for Informatics and Systems of University of Coimbra, University of Coimbra, Pólo II, 3030-290, Coimbra, Portugal. sparedes@isec.pt.
- Polytechnic Institute of Coimbra, Coimbra Institute of Engineering (IPC/ISEC), Rua Pedro Nunes, 3030-199, Coimbra, Portugal. sparedes@isec.pt.
- Polytechnic Institute of Coimbra, Coimbra Institute of Engineering (IPC/ISEC), Rua Pedro Nunes, 3030-199, Coimbra, Portugal.
- Department of Cardiology, Instituto Português de Oncologia do Porto Francisco Gentil, E.P.E., Porto, Portugal.
- Cardiology Department, Centro Hospitalar e Universitário de Coimbra, Praceta Professor Mota Pinto, 3004-561, Coimbra, Portugal.
Förderung
- Instituto Politécnico de Coimbra
Zitieren
APA
Sousa, S., Paredes, S., Rocha, T., Henriques, J., Sousa, J., & Gonçalves, L. (2024). Machine learning models’ assessment: trust and performance. Medical & Biological Engineering & Computing. https://doi.org/10.1007/s11517-024-03145-5
Vancouver
Sousa S, Paredes S, Rocha T, Henriques J, Sousa J, Gonçalves L. Machine learning models’ assessment: trust and performance. Medical & Biological Engineering & Computing. 2024. doi:10.1007/s11517-024-03145-5
BibTeX
@article{Sousa2024,
author = {Sousa, S and Paredes, S and Rocha, T and Henriques, J and Sousa, J and Gonçalves, L},
title = {Machine learning models’ assessment: trust and performance},
journal = {Medical & Biological Engineering & Computing},
year = {2024},
doi = {10.1007/s11517-024-03145-5}
}
EUVIMED