Houston, We Have AI Problem! Quality Issues with Neuroimaging‐Based Artificial Intelligence in Parkinson's Disease: A Systematic Review
Houston, wir haben ein KI-Problem! Qualitätsprobleme bei neuroimaging-basierten KI-Studien bei Parkinson: systematische Übersichtsarbeit
Movement Disorders · 2024-12 · 39 (12) · 2130-2143
DOI: 10.1002/mds.30002 ↗Auf einen Blick
Die Übersichtsarbeit fasst 244 neuroimaging-basierte KI-Studien zu Parkinson zusammen. Nur 20 % erfüllten alle fünf Minimalqualitätskriterien; Datenleckage war mit höheren Genauigkeiten verbunden.
Fragestellung
Überblick über neuroimaging-basierte KI-Studien zu Diagnose, Prognose oder Intervention bei Parkinson und Bewertung ihrer methodischen Qualität.
Methode
Systematische Übersichtsarbeit; PubMed-Suche bis zu 810 Studien, 244 eingeschlossen; Bewertung anhand von fünf Minimalqualitätskriterien zu Datenaufteilung, Datenleckage, Modellkomplexität, Berichterstattung und biologischer Plausibilität.
Ergebnisse
244 Studien. 54 % zielten auf die Unterscheidung von Parkinson und gesunden Kontrollen, 25 % auf atypische parkinsonistische Syndrome. Nur 20 % erfüllten alle fünf Kriterien; externe Testsets wurden in 8 % genutzt.
Einschränkungen
Sehr wenige Studien nutzten externe Testsets; 19 % berücksichtigten Datenungleichgewicht nicht.
Schlussfolgerung der Autor:innen
Die Autor:innen berichten, dass KI für viele Fragestellungen zu Parkinson eingesetzt wurde, die Zahl der Studien ohne Bestehen der Minimalqualitätskriterien aber alarmierend ist.
Kernaussagen
- Nur 20 % erfüllten alle fünf Minimalqualitätskriterien
- Datenleckage erhöhte die berichteten Genauigkeiten
- Externe Testsets wurden nur in 8 % der Studien verwendet
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
At a glance
Systematische Übersicht in Movement Disorders, published online on 2024-09-05. Written by Verena Dzialas and 7 co-authors. Institution: Department of Nuclear Medicine, Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany. Major MEDLINE topics: Parkinson Disease, Neuroimaging, Artificial Intelligence. Cited 27 times so far (Crossref); 89 references. The full text is freely available (open access, CC BY-NC-ND 4.0). Funded by: NIDCD NIH HHS, Deutsche Forschungsgemeinschaft, NINDS NIH HHS.
Abstract (original language)
In recent years, many neuroimaging studies have applied artificial intelligence (AI) to facilitate existing challenges in Parkinson's disease (PD) diagnosis, prognosis, and intervention. The aim of this systematic review was to provide an overview of neuroimaging‐based AI studies and to assess their methodological quality. A PubMed search yielded 810 studies, of which 244 that investigated the utility of neuroimaging‐based AI for PD diagnosis, prognosis, or intervention were included. We systematically categorized studies by outcomes and rated them with respect to five minimal quality criteria (MQC) pertaining to data splitting, data leakage, model complexity, performance reporting, and indication of biological plausibility. We found that the majority of studies aimed to distinguish PD patients from healthy controls (54%) or atypical parkinsonian syndromes (25%), whereas prognostic or interventional studies were sparse. Only 20% of evaluated studies passed all five MQC, with data leakage, non‐minimal model complexity, and reporting of biological plausibility as the primary factors for quality loss. Data leakage was associated with a significant inflation of accuracies. Very few studies employed external test sets (8%), where accuracy was significantly lower, and 19% of studies did not account for data imbalance. Adherence to MQC was low across all observed years and journal impact factors. This review outlines that AI has been applied to a wide variety of research questions pertaining to PD; however, the number of studies failing to pass the MQC is alarming. Therefore, we provide recommendations to enhance the interpretability, generalizability, and clinical utility of future AI applications using neuroimaging in PD. © 2024 The Author(s). Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Nuclear Medicine, Faculty of Medicine and University Hospital, University of Cologne, Cologne, Germany.
- Faculty of Mathematics and Natural Sciences, University of Cologne, Cologne, Germany.
- German Center for Neurodegenerative Diseases (DZNE), Bonn, Germany.
- Edmond J. Safra Parkinson Disease Program, Neurology Division, Krembil Brain Institute, University Health Network, Toronto, Canada.
- Brain Health Imaging Centre, Centre for Addiction and Mental Health, University of Toronto, Toronto, Canada.
- Temerty Faculty of Medicine, University of Toronto, Toronto, Canada.
ORCID profiles: Verena Dzialas, Elena Doering, Kristina Simonyan, Thilo van Eimeren
Funding
- NIDCD NIH HHS
- Deutsche Forschungsgemeinschaft
- NINDS NIH HHS
- NLM NIH HHS
Cite
APA
Dzialas, V., Doering, E., Eich, H., Strafella, A. P., Vaillancourt, D. E., Simonyan, K., van Eimeren, T., & International Parkinson Movement Disorders Society‐Neuroimaging Study Group (2024). Houston, We Have AI Problem! Quality Issues with Neuroimaging‐Based Artificial Intelligence in Parkinson's Disease: A Systematic Review. Movement Disorders, 39(12), 2130-2143. https://doi.org/10.1002/mds.30002
Vancouver
Dzialas V, Doering E, Eich H, Strafella AP, Vaillancourt DE, Simonyan K, et al. Houston, We Have AI Problem! Quality Issues with Neuroimaging‐Based Artificial Intelligence in Parkinson's Disease: A Systematic Review. Movement Disorders. 2024;39(12):2130-2143. doi:10.1002/mds.30002
BibTeX
@article{Dzialas2024,
author = {Dzialas, Verena and Doering, Elena and Eich, Helena and Strafella, Antonio P and Vaillancourt, David E and Simonyan, Kristina and van Eimeren, Thilo and International Parkinson Movement Disorders Society‐Neuroimaging Study Group, International Parkinson Movement Disorders Society‐Neuroimaging Study Group},
title = {Houston, We Have AI Problem! Quality Issues with Neuroimaging‐Based Artificial Intelligence in Parkinson's Disease: A Systematic Review},
journal = {Movement Disorders},
year = {2024},
volume = {39},
number = {12},
pages = {2130--2143},
doi = {10.1002/mds.30002}
}
EUVIMED