EUVIMEDEUVIMED entdecken
Crossref

metaScreener: A Plugin-Based Desktop Application for Human-in-the-Loop Systematic Literature Screening

Alejandro Reyes-Consuelo, Jocelyne Kiss, Julien Voisin

Journal of Open Research Software · 05.06.2026 · 14

DOI: 10.5334/jors.730 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Open Research Software, erschienen am 05.06.2026. Verfasst von Alejandro Reyes-Consuelo und 2 weiteren Autor:innen. Die Arbeit nennt 14 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

metaScreener is an open-source, cross-platform desktop application implemented in Python/Tkinter that delivers a sequential, multi-stage pipeline for systematic literature screening. The software integrates deterministic rule-based filters with large language model (LLM) inference to automate high-volume citation screening within structured evidence synthesis workflows. Its plugin architecture exposes seven interoperable modules covering reference marker extraction, bibliographic reference resolution, eligibility criteria structuring, heuristic-based filtering, and LLM-based full-record eligibility adjudication. Each screening decision is logged within a timestamped, SHA-256-verified bundle archive that satisfies the audit and reproducibility requirements expected in rigorous evidence synthesis methodology. In a demonstration use case comprising 776 candidate records, the pipeline reduced the corpus to 73 records requiring full human review—a 90.6% reduction—with deterministic pre-filtering accounting for 98.3% of exclusions and LLM-assisted stages providing fine-grained adjudication over the residual candidate set. A complementary human validation exercise covering 254 LLM-screening decisions across three LLM-adjudicated criteria reports observed human-versus-LLM agreement of 83.5% and 87.1% on the two exclusion criteria, with the inclusion criterion showing fair agreement (Cohen’s 𝜅 = 0.28) and a marked asymmetric pattern in which the LLM hedges toward an uncertain verdict on ambiguous abstracts. Complete decision traceability was preserved across all stages. metaScreener is released under the MIT licence and is publicly available at https://github.com/lars-ulaval/metaScreener.

Zitieren

APA

Reyes-Consuelo, A., Kiss, J., & Voisin, J. (2026). metaScreener: A Plugin-Based Desktop Application for Human-in-the-Loop Systematic Literature Screening. Journal of Open Research Software, 14. https://doi.org/10.5334/jors.730

Vancouver

Reyes-Consuelo A, Kiss J, Voisin J. metaScreener: A Plugin-Based Desktop Application for Human-in-the-Loop Systematic Literature Screening. Journal of Open Research Software. 2026;14. doi:10.5334/jors.730

BibTeX
@article{ReyesConsuelo2026,
  author = {Reyes-Consuelo, Alejandro and Kiss, Jocelyne and Voisin, Julien},
  title = {metaScreener: A Plugin-Based Desktop Application for Human-in-the-Loop Systematic Literature Screening},
  journal = {Journal of Open Research Software},
  year = {2026},
  volume = {14},
  doi = {10.5334/jors.730}
}
Kostenlos starten →