EUVIMEDEUVIMED entdecken
Systematische ÜbersichtOpen Access · CC BY 4.0

A systematic review of machine learning-based prognostic models for acute pancreatitis: Towards improving methods and reporting quality

Eine systematische Übersichtsarbeit zu machine-learningbasierten Prognosemodellen für akute Pankreatitis: Methoden- und Berichtsqualität verbessern

Brian Critelli, Amier Hassan iD, Ila Lahooti iD, Lydia Noh, Jun Sung Park, Kathleen Tong iD, Ali Lahooti, Nathan Matzko, Jan Niklas Adams, Lukas Liss, Justin Quion iD, David Restrepo iD und 23 weitere

PLOS Medicine · 24.02.2025 · 22 (2) · e1004432

DOI: 10.1371/journal.pmed.1004432 ↗

Auf einen Blick

Die Übersichtsarbeit bewertete 30 ML-Prognosemodelle bei akuter Pankreatitis. Die Berichts- und Methodenqualität war deutlich eingeschränkt.

Fragestellung

Wie gut sind die Methodik und die Berichtsqualität kürzlich publizierter ML-basierter Prognosemodelle bei akuter Pankreatitis?

Methode

Systematische Übersichtsarbeit; Suche in MEDLINE, EMBASE sowie medRxiv, bioRxiv und arXiv für 2021 bis 2023; Bewertung mit Prediction Model Risk of Bias Assessment Tool und TRIPOD+AI.

Ergebnisse

6.480 Publikationen gescreent, 30 eingeschlossen, 39 neue ML-Modelle. Mittlere AUC 0,91 (SD 0,08); hohes Risiko für Bias in mindestens 1 Domäne bei allen 39 Modellen.

Einschränkungen

Meta-Analyse wegen Heterogenität im Studiendesign und in der Outcome-Definition nicht durchgeführt.

Schlussfolgerung der Autor:innen

Es bestehen erhebliche Defizite in Methodik und Berichterstattung der jüngst publizierten ML-basierten Prognosemodelle bei AP.

Kernaussagen

  • 30 Studien erfüllten die Einschlusskriterien; 39 neue ML-Modelle wurden entwickelt
  • Mittlere AUC aller Modelle und Endpunkte: 0,91 (SD 0,08)
  • Berichtsdefizite u. a. bei Human-AI-Interaktion, Code- und Datenzugang

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Systematische Übersicht in der Zeitschrift PLOS Medicine, online erschienen am 24.02.2025. Verfasst von Brian Critelli und 34 weiteren Autor:innen. Beteiligte Einrichtung: Department of Gastroenterology and Hepatology, Weill Cornell Medical College, New York, New York, United States of America. Hauptthemen laut MEDLINE: Machine Learning, Pancreatitis. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 82 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

An accurate prognostic tool is essential to aid clinical decision-making (e.g., patient triage) and to advance personalized medicine. However, such a prognostic tool is lacking for acute pancreatitis (AP). Increasingly machine learning (ML) techniques are being used to develop high-performing prognostic models in AP. However, methodologic and reporting quality has received little attention . High-quality reporting and study methodology are critical for model validity, reproducibility, and clinical implementation. In collaboration with content experts in ML methodology, we performed a systematic review critically appraising the quality of methodology and reporting of recently published ML AP prognostic models. Methods/findings Using a validated search strategy, we identified ML AP studies from the databases MEDLINE and EMBASE published between January 2021 and December 2023. We also searched pre-print servers medRxiv, bioRxiv, and arXiv for pre-prints registered between January 2021 and December 2023. Eligibility criteria included all retrospective or prospective studies that developed or validated new or existing ML models in patients with AP that predicted an outcome following an episode of AP. Meta-analysis was considered if there was homogeneity in the study design and in the type of outcome predicted. For risk of bias (ROB) assessment, we used the Prediction Model Risk of Bias Assessment Tool. Quality of reporting was assessed using the Transparent Reporting of a Multivariable Prediction Model of Individual Prognosis or Diagnosis—Artificial Intelligence (TRIPOD+AI) statement that defines standards for 27 items that should be reported in publications using ML prognostic models. The search strategy identified 6,480 publications of which 30 met the eligibility criteria. Studies originated from China (22), the United States (4), and other (4). All 30 studies developed a new ML model and none sought to validate an existing ML model, producing a total of 39 new ML models. AP severity (23/39) or mortality (6/39) were the most common outcomes predicted. The mean area under the curve for all models and endpoints was 0.91 (SD 0.08). The ROB was high for at least one domain in all 39 models, particularly for the analysis domain (37/39 models). Steps were not taken to minimize over-optimistic model performance in 27/39 models. Due to heterogeneity in the study design and in how the outcomes were defined and determined, meta-analysis was not performed. Studies reported on only 15/27 items from TRIPOD+AI standards, with only 7/30 justifying sample size and 13/30 assessing data quality. Other reporting deficiencies included omissions regarding human–AI interaction (28/30), handling low-quality or incomplete data in practice (27/30), sharing analytical codes (25/30), study protocols (25/30), and reporting source data (19/30).

Conclusions

There are significant deficiencies in the methodology and reporting of recently published ML based prognostic models in AP patients. These undermine the validity, reproducibility, and implementation of these prognostic models despite their promise of superior predictive accuracy. Registration Research Registry (reviewregistry1727)

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Machine LearningPancreatitisHumansPrognosisReproducibility of ResultsAcute Disease

Autor:innen und Institutionen

  • Department of Gastroenterology and Hepatology, Weill Cornell Medical College, New York, New York, United States of America.
  • Department of Gastroenterology and Hepatology, Ohio State University Wexner Medical Center, Columbus, Ohio, United States of America.
  • Northeast Ohio Medical School, Rootstown, Ohio, United States of America.
  • Department of Process and Data Science, Rheinisch-Westfälische Technische Hochschule Aachen University, Aachen, Germany.
  • Department of Computational Physiology, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States of America.
  • Department of Bioinformatics, Ohio State University Wexner Medical Center, Columbus, Ohio, United States of America.

ORCID-Profile: Amier Hassan, Ila Lahooti, Kathleen Tong, Justin Quion, David Restrepo, Stacey Culp, Jason Bischof, Sophie Terp, Alexandra Weissman, Mitchell Ramsey, Samuel Han, Erica Park, Raj Shah, Nikhil K Mull, Leo Anthony Celi, Peter Lee

Zitieren

APA

Critelli, B., Hassan, A., Lahooti, I., Noh, L., Park, J. S., Tong, K., Lahooti, A., Matzko, N., Adams, J. N., Liss, L., Quion, J., Restrepo, D., Nikahd, M., Culp, S., Lacy-Hulbert, A., Speake, C., Buxbaum, J., Bischof, J., Yazici, C., …, & Lee, P. (2025). A systematic review of machine learning-based prognostic models for acute pancreatitis: Towards improving methods and reporting quality. PLOS Medicine, 22(2), e1004432. https://doi.org/10.1371/journal.pmed.1004432

Vancouver

Critelli B, Hassan A, Lahooti I, Noh L, Park JS, Tong K, et al. A systematic review of machine learning-based prognostic models for acute pancreatitis: Towards improving methods and reporting quality. PLOS Medicine. 2025;22(2):e1004432. doi:10.1371/journal.pmed.1004432

BibTeX
@article{Critelli2025,
  author = {Critelli, Brian and Hassan, Amier and Lahooti, Ila and Noh, Lydia and Park, Jun Sung and Tong, Kathleen and Lahooti, Ali and Matzko, Nathan and Adams, Jan Niklas and Liss, Lukas and Quion, Justin and Restrepo, David and Nikahd, Melica and Culp, Stacey and Lacy-Hulbert, Adam and Speake, Cate and Buxbaum, James and Bischof, Jason and Yazici, Cemal and Evans-Phillips, Anna and Terp, Sophie and Weissman, Alexandra and Conwell, Darwin and Hart, Philip and Ramsey, Mitchell and Krishna, Somashekar and Han, Samuel and Park, Erica and Shah, Raj and Akshintala, Venkata and Windsor, John A and Mull, Nikhil K and Papachristou, Georgios and Celi, Leo Anthony and Lee, Peter},
  title = {A systematic review of machine learning-based prognostic models for acute pancreatitis: Towards improving methods and reporting quality},
  journal = {PLOS Medicine},
  year = {2025},
  volume = {22},
  number = {2},
  pages = {e1004432},
  doi = {10.1371/journal.pmed.1004432}
}
Kostenlos starten →