EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Medical large language models are vulnerable to data-poisoning attacks

Daniel Alexander Alber, Zihao Yang, Anton Alyakin, Eunice Yang, Sumedha Rai, Aly A Valliani, Jeff Zhang, Gabriel R Rosenbaum, Ashley K Amend-Thomas, David B Kurland, Caroline M Kremer, Alexander Eremiev und 21 weitere

Nature Medicine · 2025-02 · 31 (2) · 618-626

DOI: 10.1038/s41591-024-03445-1 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Nature Medicine, online erschienen am 08.01.2025. Verfasst von Daniel Alexander Alber und 32 weiteren Autor:innen. Beteiligte Einrichtung: Department of Neurosurgery, NYU Langone Health, New York, NY, USA. daniel.alber@nyulangone.org. Die Arbeit wurde bisher 219-mal zitiert (Crossref) und nennt 70 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: U.S. Department of Health & Human Services | NIH | National Cancer Institute (NCI), Neurosurgery Research and Education Foundation -- Medical Student Summer Research Fellowship 2023.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The adoption of large language models (LLMs) in healthcare demands a careful analysis of their potential to spread false medical knowledge. Because LLMs ingest massive volumes of data from the open Internet during training, they are potentially exposed to unverified medical knowledge that may include deliberately planted misinformation. Here, we perform a threat assessment that simulates a data-poisoning attack against The Pile, a popular dataset used for LLM development. We find that replacement of just 0.001% of training tokens with medical misinformation results in harmful models more likely to propagate medical errors. Furthermore, we discover that corrupted models match the performance of their corruption-free counterparts on open-source benchmarks routinely used to evaluate medical LLMs. Using biomedical knowledge graphs to screen medical LLM outputs, we propose a harm mitigation strategy that captures 91.9% of harmful content (F1 = 85.7%). Our algorithm provides a unique method to validate stochastically generated LLM outputs against hard-coded relationships in knowledge graphs. In view of current calls for improved data provenance and transparent LLM development, we hope to raise awareness of emergent risks from LLMs trained indiscriminately on web-scraped data, particularly in healthcare where misinformation can potentially compromise patient safety.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

HumansAlgorithmsInternetCommunicationLarge Language Models

Autor:innen und Institutionen

  • Department of Neurosurgery, NYU Langone Health, New York, NY, USA. daniel.alber@nyulangone.org.
  • New York University Grossman School of Medicine, New York, NY, USA. daniel.alber@nyulangone.org.
  • Department of Neurosurgery, NYU Langone Health, New York, NY, USA.
  • Center for Data Science, New York University, New York, NY, USA.
  • Washington University School of Medicine, Saint Louis, MO, USA.
  • Columbia University Vagelos College of Physicians and Surgeons, New York, NY, USA.

Förderung

  • U.S. Department of Health & Human Services | NIH | National Cancer Institute (NCI)
  • Neurosurgery Research and Education Foundation -- Medical Student Summer Research Fellowship 2023

Zitieren

APA

Alber, D. A., Yang, Z., Alyakin, A., Yang, E., Rai, S., Valliani, A. A., Zhang, J., Rosenbaum, G. R., Amend-Thomas, A. K., Kurland, D. B., Kremer, C. M., Eremiev, A., Negash, B., Wiggan, D. D., Nakatsuka, M. A., Sangwon, K. L., Neifert, S. N., Khan, H. A., Save, A. V., …, & Oermann, E. K. (2025). Medical large language models are vulnerable to data-poisoning attacks. Nature Medicine, 31(2), 618-626. https://doi.org/10.1038/s41591-024-03445-1

Vancouver

Alber DA, Yang Z, Alyakin A, Yang E, Rai S, Valliani AA, et al. Medical large language models are vulnerable to data-poisoning attacks. Nature Medicine. 2025;31(2):618-626. doi:10.1038/s41591-024-03445-1

BibTeX
@article{Alber2025,
  author = {Alber, Daniel Alexander and Yang, Zihao and Alyakin, Anton and Yang, Eunice and Rai, Sumedha and Valliani, Aly A and Zhang, Jeff and Rosenbaum, Gabriel R and Amend-Thomas, Ashley K and Kurland, David B and Kremer, Caroline M and Eremiev, Alexander and Negash, Bruck and Wiggan, Daniel D and Nakatsuka, Michelle A and Sangwon, Karl L and Neifert, Sean N and Khan, Hammad A and Save, Akshay Vinod and Palla, Adhith and Grin, Eric A and Hedman, Monika and Nasir-Moin, Mustafa and Liu, Xujin Chris and Jiang, Lavender Yao and Mankowski, Michal A and Segev, Dorry L and Aphinyanaphongs, Yindalon and Riina, Howard A and Golfinos, John G and Orringer, Daniel A and Kondziolka, Douglas and Oermann, Eric Karl},
  title = {Medical large language models are vulnerable to data-poisoning attacks},
  journal = {Nature Medicine},
  year = {2025},
  volume = {31},
  number = {2},
  pages = {618--626},
  doi = {10.1038/s41591-024-03445-1}
}
Kostenlos starten →