EUVIMEDSkopri EUVIMED
ValidierungsstudieOpen access

Development of Interoperable Computable Phenotype Algorithms for Adverse Events of Special Interest to Be Used for Biologics Safety Surveillance: Validation Study

Entwicklung interoperabler computabler Phänotyp-Algorithmen für unerwünschte Ereignisse von besonderem Interesse zur Sicherheitsüberwachung von Biologika: Validierungsstudie

Ashley A Holdefer iD, Jeno Pizarro iD, Patrick Saunders-Hastings iD, Jeffrey Beers iD, Arianna Sang iD, Aaron Zachary Hettinger iD, Joseph Blumenthal iD, Erik Martinez iD, Lance Daniel Jones iD, Matthew Deady iD, Hussein Ezzeldin iD, Steven A Anderson iD

JMIR Public Health and Surveillance · 2024-07-15 · 10 · e49811

DOI: 10.2196/49811 ↗

Auf einen Blick

Die Studie validierte 5 regelbasierte Phänotyp-Algorithmen für AESIs in EHR-Daten. Am besten schnitt der Anaphylaxie-Algorithmus mit einer PPV von 93,3 % ab.

Fragestellung

Wie gut erkennen interoperable computable Phenotype-Algorithmen 5 AESIs mit hoher positiver Vorhersagewahrscheinlichkeit für die Sicherheitsüberwachung von Biologika?

Methode

Regelbasierte Algorithmen für EHR-Daten; Validierung an Daten eines US-amerikanischen akademischen Gesundheitssystems; Bewertung mit PPV.

Ergebnisse

PPV: Anaphylaxie 93,3 %; febriler Krampfanfall 89 %; Myokarditis/Perikarditis 83,5 %; Thrombose mit Thrombozytopenie-Syndrom 70,2 %; Guillain-Barré-Syndrom 47,2 %.

Schlussfolgerung der Autor:innen

Die Ergebnisse unterstützen laut Autor:innen weitere Forschung zum Einsatz interoperabler Algorithmen für die breite AESI-Detektion nach Markteinführung.

Kernaussagen

  • Anaphylaxie-Algorithmus: PPV 93,3 %
  • Weitere PPVs lagen zwischen 47,2 % und 89 %
  • Interoperable Algorithmen könnten die AESI-Überwachung unterstützen

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Validierungsstudie in JMIR Public Health and Surveillance, published online on 2024-07-15. Written by Ashley A Holdefer and 11 co-authors. Institution: IBM Consulting, Bethesda, MD, United States. Major MEDLINE topics: Algorithms, Phenotype. Cited 3 times so far (Crossref); 49 references. The full text is freely available in PubMed Central.

Auf einen BlickAstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

Background

Adverse events associated with vaccination have been evaluated by epidemiological studies and more recently have gained additional attention with the emergency use authorization of several COVID-19 vaccines. As part of its responsibility to conduct postmarket surveillance, the US Food and Drug Administration continues to monitor several adverse events of special interest (AESIs) to ensure vaccine safety, including for COVID-19.

Objective

This study is part of the Biologics Effectiveness and Safety Initiative, which aims to improve the Food and Drug Administration’s postmarket surveillance capabilities while minimizing public burden. This study aimed to enhance active surveillance efforts through a rules-based, computable phenotype algorithm to identify 5 AESIs being monitored by the Center for Disease Control and Prevention for COVID-19 or other vaccines: anaphylaxis, Guillain-Barré syndrome, myocarditis/pericarditis, thrombosis with thrombocytopenia syndrome, and febrile seizure. This study examined whether these phenotypes have sufficiently high positive predictive value (PPV) to ensure that the cases selected for surveillance are reasonably likely to be a postbiologic adverse event. This allows patient privacy, and security concerns for the data sharing of patients who had nonadverse events can be properly accounted for when evaluating the cost-benefit aspect of our approach.

Methods

AESI phenotype algorithms were developed to apply to electronic health record data at health provider organizations across the country by querying for standard and interoperable codes. The codes queried in the rules represent symptoms, diagnoses, or treatments of the AESI sourced from published case definitions and input from clinicians. To validate the performance of the algorithms, we applied them to electronic health record data from a US academic health system and provided a sample of cases for clinicians to evaluate. Performance was assessed using PPV.

Results

With a PPV of 93.3%, our anaphylaxis algorithm performed the best. The PPVs for our febrile seizure, myocarditis/pericarditis, thrombocytopenia syndrome, and Guillain-Barré syndrome algorithms were 89%, 83.5%, 70.2%, and 47.2%, respectively.

Conclusions

Given our algorithm design and performance, our results support continued research into using interoperable algorithms for widespread AESI postmarket detection.

Subjects

MeSH terms (MEDLINE)

AlgorithmsPhenotypeHumansUnited StatesBiological ProductsUnited States Food and Drug AdministrationAdverse Drug Reaction Reporting SystemsDrug-Related Side Effects and Adverse ReactionsProduct Surveillance, PostmarketingCOVID-19

Author keywords

COVID-19 vaccineFood and Drug Administrationadverse eventcomputable phenotypeelectronic health recordspostmarket surveillance systemreal-world datavaccine safetyvalidation study

Substances

Biological Products

Authors and institutions

  • IBM Consulting, Bethesda, MD, United States.
  • Accenture Inc, Ottawa, ON, Canada.
  • Center for Biostatistics, Informatics and Data Science, MedStar Health Research Institute, Columbia, MD, United States.
  • Department of Emergency Medicine, Georgetown University School of Medicine, Washington, DC, United States.
  • Center for Biologics Evaluation and Research, United States Food and Drug Administration, Silver Spring, MD, United States.

ORCID profiles: Ashley A Holdefer, Jeno Pizarro, Patrick Saunders-Hastings, Jeffrey Beers, Arianna Sang, Aaron Zachary Hettinger, Joseph Blumenthal, Erik Martinez, Lance Daniel Jones, Matthew Deady, Hussein Ezzeldin, Steven A Anderson

Cite

APA

Holdefer, A. A., Pizarro, J., Saunders-Hastings, P., Beers, J., Sang, A., Hettinger, A. Z., Blumenthal, J., Martinez, E., Jones, L. D., Deady, M., Ezzeldin, H., & Anderson, S. A. (2024). Development of Interoperable Computable Phenotype Algorithms for Adverse Events of Special Interest to Be Used for Biologics Safety Surveillance: Validation Study. JMIR Public Health and Surveillance, 10, e49811. https://doi.org/10.2196/49811

Vancouver

Holdefer AA, Pizarro J, Saunders-Hastings P, Beers J, Sang A, Hettinger AZ, et al. Development of Interoperable Computable Phenotype Algorithms for Adverse Events of Special Interest to Be Used for Biologics Safety Surveillance: Validation Study. JMIR Public Health and Surveillance. 2024;10:e49811. doi:10.2196/49811

BibTeX
@article{Holdefer2024,
  author = {Holdefer, Ashley A and Pizarro, Jeno and Saunders-Hastings, Patrick and Beers, Jeffrey and Sang, Arianna and Hettinger, Aaron Zachary and Blumenthal, Joseph and Martinez, Erik and Jones, Lance Daniel and Deady, Matthew and Ezzeldin, Hussein and Anderson, Steven A},
  title = {Development of Interoperable Computable Phenotype Algorithms for Adverse Events of Special Interest to Be Used for Biologics Safety Surveillance: Validation Study},
  journal = {JMIR Public Health and Surveillance},
  year = {2024},
  volume = {10},
  pages = {e49811},
  doi = {10.2196/49811}
}
Start for free →