EUVIMEDEUVIMED entdecken
Open Access

More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA

David Lyell iD, Ying Wang iD, Enrico Coiera iD, Farah Magrabi iD

Journal of the American Medical Informatics Association · 20.06.2023 · 30 (7) · 1227-1236

DOI: 10.1093/jamia/ocad065 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 18.04.2023. Verfasst von David Lyell und 3 weiteren Autor:innen. Beteiligte Einrichtung: Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, NSW 2109, Australia. Hauptthemen laut MEDLINE: Device Approval, Algorithms. Die Arbeit wurde bisher 52-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NHMRC, Centre for Research Excellence, Macquarie University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To examine the real-world safety problems involving machine learning (ML)-enabled medical devices. Materials and Methods We analyzed 266 safety events involving approved ML medical devices reported to the US FDA’s MAUDE program between 2015 and October 2021. Events were reviewed against an existing framework for safety problems with Health IT to identify whether a reported problem was due to the ML device (device problem) or its use, and key contributors to the problem. Consequences of events were also classified.

Results

Events described hazards with potential to harm (66%), actual harm (16%), consequences for healthcare delivery (9%), near misses that would have led to harm if not for intervention (4%), no harm or consequences (3%), and complaints (2%). While most events involved device problems (93%), use problems (7%) were 4 times more likely to harm (relative risk 4.2; 95% CI 2.5–7). Problems with data input to ML devices were the top contributor to events (82%).

Discussion

Much of what is known about ML safety comes from case studies and the theoretical limitations of ML. We contribute a systematic analysis of ML safety problems captured as part of the FDA’s routine post-market surveillance. Most problems involved devices and concerned the acquisition of data for processing by algorithms. However, problems with the use of devices were more likely to harm.

Conclusions

Safety problems with ML devices involve more than algorithms, highlighting the need for a whole-of-system approach to safe implementation with a special focus on how users interact with devices.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Device ApprovalAlgorithmsUnited StatesDelivery of Health CareUnited States Food and Drug Administration

Stichwörter der Autor:innen

artificial intelligenceclinicalclinical decision-makingcomputer-assisteddecision support systemsdecision-makingmachine learningmedical devicessafety

Autor:innen und Institutionen

  • Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, NSW 2109, Australia.

ORCID-Profile: David Lyell, Ying Wang, Enrico Coiera, Farah Magrabi

Förderung

  • NHMRC
  • Centre for Research Excellence
  • Macquarie University

Zitieren

APA

Lyell, D., Wang, Y., Coiera, E., & Magrabi, F. (2023). More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA. Journal of the American Medical Informatics Association, 30(7), 1227-1236. https://doi.org/10.1093/jamia/ocad065

Vancouver

Lyell D, Wang Y, Coiera E, Magrabi F. More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA. Journal of the American Medical Informatics Association. 2023;30(7):1227-1236. doi:10.1093/jamia/ocad065

BibTeX
@article{Lyell2023,
  author = {Lyell, David and Wang, Ying and Coiera, Enrico and Magrabi, Farah},
  title = {More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA},
  journal = {Journal of the American Medical Informatics Association},
  year = {2023},
  volume = {30},
  number = {7},
  pages = {1227--1236},
  doi = {10.1093/jamia/ocad065}
}
Kostenlos starten →