EUVIMEDEUVIMED entdecken
ValidierungsstudieOpen Access

Validation of an AI‐Assisted Framework for Systematic Bias Assessment in Observational Studies

Mahyar Etminan iD, Ramin Rezaeianzadeh iD, Antonios Douros iD

Journal of Evaluation in Clinical Practice · 2026

DOI: 10.1111/jep.70593 ↗

Kurz gefasst

Validierungsstudie in der Zeitschrift Journal of Evaluation in Clinical Practice, 2026. Verfasst von Mahyar Etminan und 2 weiteren Autor:innen. Beteiligte Einrichtung: Epilytics Consulting, Vancouver, British Columbia, Canada. Hauptthemen laut MEDLINE: Observational Studies as Topic, Artificial Intelligence, Pharmacoepidemiology. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Rationale

The rapid expansion of medical literature has led to variability and contradictions in study findings, making it increasingly difficult to distinguish meaningful signals from noise. Much of this variability arises from methodological limitations, including confounding, selection bias, and reverse causation. Although artificial intelligence (AI)‐assisted tools exist for risk‐of‐bias assessment, most are designed for systematic reviews and are not tailored to identifying epidemiologic biases in observational studies. Structured, scalable approaches are needed to evaluate validity in real‐world evidence research. Aims and Objectives To develop and validate EpiVise, an AI‐assisted, expert‐informed, rule‐based framework for identifying major sources of bias in pharmacoepidemiologic studies and to assess its agreement with expert epidemiologist evaluations.

Methods

Recently published pharmacoepidemiologic studies from high‐impact journals (post‐ July 2025) were independently evaluated by EpiVise and two expert epidemiologists across predefined bias domains, including measured confounding, confounding by indication, selection bias, immortal time bias, and disease latency bias. Agreement was assessed using weighted kappa statistics. In addition, synthetic study scenarios with predefined embedded biases were constructed to evaluate framework performance under controlled conditions.

Results

Among published studies (10 studies; 60 ratings), agreement between EpiVise and expert assessments was substantial (weighted κ = 0.75; 95% confidence interval [CI], 0.63–0.87). Twelve ratings (20.0%) were discordant, all limited to adjacent categories. In synthetic scenarios (10 studies; 50 ratings), agreement was also substantial, with 40 of 50 ratings concordant (80.0%) and a weighted κ of 0.72 (95% CI, 0.61–0.83).

Conclusion

EpiVise demonstrated substantial agreement with expert epidemiologist assessments in both published and synthetic study evaluations. As a scalable and reproducible framework for identifying common epidemiologic biases, EpiVise may enhance evidence appraisal, peer review, and clinical or regulatory decision‐making. Further validation across broader study designs and therapeutic areas is warranted.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Observational Studies as TopicArtificial IntelligencePharmacoepidemiologyBiasHumansReproducibility of ResultsConfounding Factors, Epidemiologic

Stichwörter der Autor:innen

AI platformAI validationbias assessmentobservational studies

Autor:innen und Institutionen

  • Epilytics Consulting, Vancouver, British Columbia, Canada.
  • Faculty of Pharmaceutical Sciences, University of British Columbia, Vancouver, British Columbia, Canada.
  • Institute of Clinical Pharmacology and Toxicology, Charité - Universitätsmedizin Berlin, Berlin, Germany.

ORCID-Profile: Mahyar Etminan, Ramin Rezaeianzadeh, Antonios Douros

Zitieren

APA

Etminan, M., Rezaeianzadeh, R., & Douros, A. (2026). Validation of an AI‐Assisted Framework for Systematic Bias Assessment in Observational Studies. Journal of Evaluation in Clinical Practice. https://doi.org/10.1111/jep.70593

Vancouver

Etminan M, Rezaeianzadeh R, Douros A. Validation of an AI‐Assisted Framework for Systematic Bias Assessment in Observational Studies. Journal of Evaluation in Clinical Practice. 2026. doi:10.1111/jep.70593

BibTeX
@article{Etminan2026,
  author = {Etminan, Mahyar and Rezaeianzadeh, Ramin and Douros, Antonios},
  title = {Validation of an AI‐Assisted Framework for Systematic Bias Assessment in Observational Studies},
  journal = {Journal of Evaluation in Clinical Practice},
  year = {2026},
  doi = {10.1111/jep.70593}
}
Kostenlos starten →