EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access · CC BY 4.0

Machine learning-based risk prediction for major adverse cardiovascular events in a Brazilian hospital: Development, external validation, and interpretability

Gilson Yuuji Shimizu iD, Michael Schrempf, Elen Almeida Romão, Stefanie Jauk, Diether Kramer, Peter P Rainer, José Abrão Cardeal da Costa, João Mazzoncini de Azevedo-Marques, Sandro Scarpelini, Katia Mitiko Firmino Suzuki, Hilton Vicente César, Paulo Mazzoncini de Azevedo-Marques iD

PLOS ONE · 11.10.2024 · 19 (10) · e0311719

DOI: 10.1371/journal.pone.0311719 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift PLOS ONE, online erschienen am 11.10.2024. Verfasst von Gilson Yuuji Shimizu und 11 weiteren Autor:innen. Beteiligte Einrichtung: Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, São Paulo, Brazil. Hauptthemen laut MEDLINE: Machine Learning, Cardiovascular Diseases. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fundação de Amparo à Pesquisa do Estado de São Paulo, Conselho Nacional de Desenvolvimento Científico e Tecnológico, ERA PerMed.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Studies of cardiovascular disease risk prediction by machine learning algorithms often do not assess their ability to generalize to other populations and few of them include an analysis of the interpretability of individual predictions. This manuscript addresses the development and validation, both internal and external, of predictive models for the assessment of risks of major adverse cardiovascular events (MACE). Global and local interpretability analyses of predictions were conducted towards improving MACE’s model reliability and tailoring preventive interventions.

Methods

The models were trained and validated on a retrospective cohort with the use of data from Ribeirão Preto Medical School (RPMS), University of São Paulo, Brazil. Data from Beth Israel Deaconess Medical Center (BIDMC), USA, were used for external validation. A balanced sample of 6,000 MACE cases and 6,000 non-MACE cases from RPMS was created for training and internal validation and an additional one of 8,000 MACE cases and 8,000 non-MACE cases from BIDMC was employed for external validation. Eight machine learning algorithms, namely Penalized Logistic Regression, Random Forest, XGBoost, Decision Tree, Support Vector Machine, k-Nearest Neighbors, Naive Bayes, and Multi-Layer Perceptron were trained to predict a 5-year risk of major adverse cardiovascular events and their predictive performance was evaluated regarding accuracy, ROC curve (receiver operating characteristic), and AUC (area under the ROC curve). LIME and Shapley values were applied towards insights about model interpretability.

Findings

Random Forest showed the best predictive performance in both internal validation (AUC = 0.871 (0.859–0.882); Accuracy = 0.794 (0.782–0.808)) and external one (AUC = 0.786 (0.778–0.792); Accuracy = 0.710 (0.704–0.717)). Compared to LIME, Shapley values suggest more consistent explanations on exploratory analysis and importance of features.

Conclusions

Among the machine learning algorithms evaluated, Random Forest showed the best generalization ability, both internally and externally. Shapley values for local interpretability were more informative than LIME ones, which is in line with our exploratory analysis and global interpretation of the final model. Machine learning algorithms with good generalization and accompanied by interpretability analyses are recommended for assessments of individual risks of cardiovascular diseases and development of personalized preventive actions.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningCardiovascular DiseasesHumansBrazilFemaleMaleMiddle AgedRetrospective StudiesRisk AssessmentAgedAlgorithmsHospitalsRisk FactorsReproducibility of Results

Autor:innen und Institutionen

  • Ribeirão Preto Medical School, University of São Paulo, Ribeirão Preto, São Paulo, Brazil.
  • Predicting Health GmbH, Graz, Austria.
  • Division of Cardiology, Medical University of Graz, Graz, Austria.
  • Steiermärkische Krankenanstaltengesellschaft m. b. H., Graz, Austria.
  • Medical University of Graz, Graz, Austria.

ORCID-Profile: Gilson Yuuji Shimizu, Paulo Mazzoncini de Azevedo-Marques

Förderung

  • Fundação de Amparo à Pesquisa do Estado de São Paulo
  • Conselho Nacional de Desenvolvimento Científico e Tecnológico
  • ERA PerMed

Zitieren

APA

Shimizu, G. Y., Schrempf, M., Romão, E. A., Jauk, S., Kramer, D., Rainer, P. P., Cardeal da Costa, J. A., de Azevedo-Marques, J. M., Scarpelini, S., Suzuki, K. M. F., César, H. V., & de Azevedo-Marques, P. M. (2024). Machine learning-based risk prediction for major adverse cardiovascular events in a Brazilian hospital: Development, external validation, and interpretability. PLOS ONE, 19(10), e0311719. https://doi.org/10.1371/journal.pone.0311719

Vancouver

Shimizu GY, Schrempf M, Romão EA, Jauk S, Kramer D, Rainer PP, et al. Machine learning-based risk prediction for major adverse cardiovascular events in a Brazilian hospital: Development, external validation, and interpretability. PLOS ONE. 2024;19(10):e0311719. doi:10.1371/journal.pone.0311719

BibTeX
@article{Shimizu2024,
  author = {Shimizu, Gilson Yuuji and Schrempf, Michael and Romão, Elen Almeida and Jauk, Stefanie and Kramer, Diether and Rainer, Peter P and Cardeal da Costa, José Abrão and de Azevedo-Marques, João Mazzoncini and Scarpelini, Sandro and Suzuki, Katia Mitiko Firmino and César, Hilton Vicente and de Azevedo-Marques, Paulo Mazzoncini},
  title = {Machine learning-based risk prediction for major adverse cardiovascular events in a Brazilian hospital: Development, external validation, and interpretability},
  journal = {PLOS ONE},
  year = {2024},
  volume = {19},
  number = {10},
  pages = {e0311719},
  doi = {10.1371/journal.pone.0311719}
}
Kostenlos starten →