EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Leveraging AI for Meta‐Analysis: Evaluating LLMs in Detecting Publication Bias for Next‐Generation Evidence Synthesis

Xing Xing iD, Lifeng Lin iD, Mohammad Hassan Murad iD, Jiayi Tong iD

Cochrane Evidence Synthesis and Methods · 2025-09 · 3 (5)

DOI: 10.1002/cesm.70047 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cochrane Evidence Synthesis and Methods, online erschienen am 18.09.2025. Verfasst von Xing Xing und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Baltimore Maryland USA. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Publication bias (PB) threatens the validity of meta‐analyses by distorting effect size estimates, potentially leading to misleading conclusions. With advanced pattern recognition and multimodal capabilities, large language models (LLMs) may be able to evaluate PB and make the systematic review process more efficient.

Methods

We evaluated the ability of two state‐of‐the‐art multimodal LLMs, GPT‐4o and Llama 3.2 Vision, to detect PB using funnel plots alone and in combination with quantitative inputs. We simulated meta‐analyses under varying conditions, including the absence of PB, different levels of presence of PB, varying total number of studies within a meta‐analysis, and differing degrees of between‐study heterogeneity.

Results

Neither GPT‐4o nor Llama 3.2 Vision consistently detected the presence of PB across various settings. Under no‐publication‐bias conditions, GPT‐4o achieved a higher specificity outperforming Llama 3.2 Vision, with the difference most shown in the meta‐analyses with 20 or more studies. The inclusion of quantitative inputs alongside funnel plots did not significantly improve performance. Additionally, between‐study heterogeneity and patterns of non‐reported studies had minimal impact on the models’ assessments.

Conclusions

The ability of LLMs to detect PB without fine‐tuning is limited at the present time. This study highlights the need for specialized model adaptation before LLMs can be effectively integrated into meta‐analysis workflows. Future research can focus on targeted refinements to enhance LLM performance and utility in evidence synthesis.

Schlagwörter

Stichwörter der Autor:innen

evidence synthesislarge language modelsmeta‐analysispublication bias

Autor:innen und Institutionen

  • Department of Biostatistics Johns Hopkins Bloomberg School of Public Health Baltimore Maryland USA.
  • Department of Epidemiology and Biostatistics University of Arizona Tucson Arizona USA.
  • Evidence-Based Practice Center, Mayo Clinic Rochester Minnesota USA.

ORCID-Profile: Xing Xing, Lifeng Lin, Mohammad Hassan Murad, Jiayi Tong

Zitieren

APA

Xing, X., Lin, L., Murad, M. H., & Tong, J. (2025). Leveraging AI for Meta‐Analysis: Evaluating LLMs in Detecting Publication Bias for Next‐Generation Evidence Synthesis. Cochrane Evidence Synthesis and Methods, 3(5). https://doi.org/10.1002/cesm.70047

Vancouver

Xing X, Lin L, Murad MH, Tong J. Leveraging AI for Meta‐Analysis: Evaluating LLMs in Detecting Publication Bias for Next‐Generation Evidence Synthesis. Cochrane Evidence Synthesis and Methods. 2025;3(5). doi:10.1002/cesm.70047

BibTeX
@article{Xing2025,
  author = {Xing, Xing and Lin, Lifeng and Murad, Mohammad Hassan and Tong, Jiayi},
  title = {Leveraging AI for Meta‐Analysis: Evaluating LLMs in Detecting Publication Bias for Next‐Generation Evidence Synthesis},
  journal = {Cochrane Evidence Synthesis and Methods},
  year = {2025},
  volume = {3},
  number = {5},
  doi = {10.1002/cesm.70047}
}
Kostenlos starten →