Validation of an AI‐Assisted Framework for Systematic Bias Assessment in Observational Studies
Journal of Evaluation in Clinical Practice · 2026
DOI: 10.1111/jep.70593 ↗Kurz gefasst
Validierungsstudie in der Zeitschrift Journal of Evaluation in Clinical Practice, 2026. Verfasst von Mahyar Etminan und 2 weiteren Autor:innen. Beteiligte Einrichtung: Epilytics Consulting, Vancouver, British Columbia, Canada. Hauptthemen laut MEDLINE: Observational Studies as Topic, Artificial Intelligence, Pharmacoepidemiology. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Rationale
The rapid expansion of medical literature has led to variability and contradictions in study findings, making it increasingly difficult to distinguish meaningful signals from noise. Much of this variability arises from methodological limitations, including confounding, selection bias, and reverse causation. Although artificial intelligence (AI)‐assisted tools exist for risk‐of‐bias assessment, most are designed for systematic reviews and are not tailored to identifying epidemiologic biases in observational studies. Structured, scalable approaches are needed to evaluate validity in real‐world evidence research. Aims and Objectives To develop and validate EpiVise, an AI‐assisted, expert‐informed, rule‐based framework for identifying major sources of bias in pharmacoepidemiologic studies and to assess its agreement with expert epidemiologist evaluations.
Methods
Recently published pharmacoepidemiologic studies from high‐impact journals (post‐ July 2025) were independently evaluated by EpiVise and two expert epidemiologists across predefined bias domains, including measured confounding, confounding by indication, selection bias, immortal time bias, and disease latency bias. Agreement was assessed using weighted kappa statistics. In addition, synthetic study scenarios with predefined embedded biases were constructed to evaluate framework performance under controlled conditions.
Results
Among published studies (10 studies; 60 ratings), agreement between EpiVise and expert assessments was substantial (weighted κ = 0.75; 95% confidence interval [CI], 0.63–0.87). Twelve ratings (20.0%) were discordant, all limited to adjacent categories. In synthetic scenarios (10 studies; 50 ratings), agreement was also substantial, with 40 of 50 ratings concordant (80.0%) and a weighted κ of 0.72 (95% CI, 0.61–0.83).
Conclusion
EpiVise demonstrated substantial agreement with expert epidemiologist assessments in both published and synthetic study evaluations. As a scalable and reproducible framework for identifying common epidemiologic biases, EpiVise may enhance evidence appraisal, peer review, and clinical or regulatory decision‐making. Further validation across broader study designs and therapeutic areas is warranted.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Epilytics Consulting, Vancouver, British Columbia, Canada.
- Faculty of Pharmaceutical Sciences, University of British Columbia, Vancouver, British Columbia, Canada.
- Institute of Clinical Pharmacology and Toxicology, Charité - Universitätsmedizin Berlin, Berlin, Germany.
ORCID-Profile: Mahyar Etminan, Ramin Rezaeianzadeh, Antonios Douros
Zitieren
APA
Etminan, M., Rezaeianzadeh, R., & Douros, A. (2026). Validation of an AI‐Assisted Framework for Systematic Bias Assessment in Observational Studies. Journal of Evaluation in Clinical Practice. https://doi.org/10.1111/jep.70593
Vancouver
Etminan M, Rezaeianzadeh R, Douros A. Validation of an AI‐Assisted Framework for Systematic Bias Assessment in Observational Studies. Journal of Evaluation in Clinical Practice. 2026. doi:10.1111/jep.70593
BibTeX
@article{Etminan2026,
author = {Etminan, Mahyar and Rezaeianzadeh, Ramin and Douros, Antonios},
title = {Validation of an AI‐Assisted Framework for Systematic Bias Assessment in Observational Studies},
journal = {Journal of Evaluation in Clinical Practice},
year = {2026},
doi = {10.1111/jep.70593}
}
EUVIMED