EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Critical Appraisal of Artificial Intelligence for Rare-Event Recognition: Principles and Pharmacovigilance Case Studies

G Niklas Norén, Eva-Lisa Meldau, Johan Ellenius

Drug Safety · 2026

DOI: 10.1007/s40264-026-01649-7 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Drug Safety, 2026. Verfasst von G Niklas Norén und 2 weiteren Autor:innen. Beteiligte Einrichtung: Uppsala Monitoring Centre, Uppsala, Sweden. niklas.noren@who-umc.org. Hauptthemen laut MEDLINE: Adverse Drug Reaction Reporting Systems, Drug-Related Side Effects and Adverse Reactions, Generative Artificial Intelligence, Pharmacovigilance. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Many high-stakes artificial intelligence (AI) applications target low-prevalence events, where apparent accuracy can conceal limited real-world value. Relevant AI models range from expert-defined rules and traditional machine learning to generative large language models (LLMs) constrained for classification. As the effort and expertise required to develop modern AI decrease, there is a risk that organizations devote too little time to understanding their limitations and sources of error. We outline key dimensions for critical appraisal of AI in rare-event recognition, including problem framing and test set design, prevalence-aware statistical evaluation, robustness assessment, and integration into human workflows. In addition, we propose an approach to structured case-level examination (SCLE), to complement statistical performance evaluation, and a set of considerations to guide procurement or development of AI models for rare-event recognition. We instantiate the framework in pharmacovigilance, drawing on three studies: rule-based retrieval of pregnancy-related reports, duplicate detection combining machine learning with probabilistic record linkage, and automated redaction of person names using an LLM. We highlight pitfalls specific to the rare-event setting including optimism from unrealistic class balance and lack of difficult positive controls in test sets-and show how cost-sensitive targets align model performance with operational value. While grounded in pharmacovigilance practice, the principles generalize to domains where positives are scarce, and error costs may be asymmetric.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Adverse Drug Reaction Reporting SystemsDrug-Related Side Effects and Adverse ReactionsGenerative Artificial IntelligencePharmacovigilanceHumansPregnancyLarge Language ModelsMachine LearningFemale

Autor:innen und Institutionen

  • Uppsala Monitoring Centre, Uppsala, Sweden. niklas.noren@who-umc.org.
  • Uppsala Monitoring Centre, Uppsala, Sweden.

Zitieren

APA

Norén, G. N., Meldau, E. L., & Ellenius, J. (2026). Critical Appraisal of Artificial Intelligence for Rare-Event Recognition: Principles and Pharmacovigilance Case Studies. Drug Safety. https://doi.org/10.1007/s40264-026-01649-7

Vancouver

Norén GN, Meldau EL, Ellenius J. Critical Appraisal of Artificial Intelligence for Rare-Event Recognition: Principles and Pharmacovigilance Case Studies. Drug Safety. 2026. doi:10.1007/s40264-026-01649-7

BibTeX
@article{Norn2026,
  author = {Norén, G Niklas and Meldau, Eva-Lisa and Ellenius, Johan},
  title = {Critical Appraisal of Artificial Intelligence for Rare-Event Recognition: Principles and Pharmacovigilance Case Studies},
  journal = {Drug Safety},
  year = {2026},
  doi = {10.1007/s40264-026-01649-7}
}
Kostenlos starten →