Beyond human gold standards: A multimodel framework for automated abstract classification and information extraction
Research Synthesis Methods · 2026
DOI: 10.1017/rsm.2025.10054 ↗Kurz gefasst
Fachartikel in der Zeitschrift Research Synthesis Methods, 2026. Verfasst von Delphine S Courvoisier und 5 weiteren Autor:innen. Beteiligte Einrichtung: Rheumatology, https://ror.org/01m1pv723Geneva University Hospitals, Switzerland. Hauptthemen laut MEDLINE: Data Mining, Abstracting and Indexing, Information Storage and Retrieval. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung.
Abstract (Originalsprache)
Meta-research and evidence synthesis require considerable resources. Large language models (LLMs) have emerged as promising tools to assist in these processes, yet their performance varies across models, limiting their reliability. Taking advantage of the large availability of small size (70 billion parameters), our framework demonstrated robust performance: several model combinations achieved accuracies above 95% exceeding the human gold standard on at least 85% of abstracts (e.g., 3 of 5 models, 4 of 6 models, or 5 of 7 models). Performance variability across individual models was not an issue, as low-performing models contributed fewer accepted decisions. This agreement-based framework offers a scalable solution that can replace human reviewers for most abstracts, reserving human expertise for more complex cases. Such frameworks could significantly reduce the manual burden in systematic reviews while maintaining high accuracy and reproducibility.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Rheumatology, https://ror.org/01m1pv723Geneva University Hospitals, Switzerland.
- Rheumatology, https://ror.org/01swzsf04University of Geneva, Switzerland.
- Internal Medicine, https://ror.org/01m1pv723Geneva University Hospitals, Switzerland.
ORCID-Profile: Delphine S Courvoisier, Denis Mongin
Förderung
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
Zitieren
APA
Courvoisier, D. S., Buitrago-Garcia, D., Buclin, C. P., Bürgisser, N., Iudici, M., & Mongin, D. (2026). Beyond human gold standards: A multimodel framework for automated abstract classification and information extraction. Research Synthesis Methods. https://doi.org/10.1017/rsm.2025.10054
Vancouver
Courvoisier DS, Buitrago-Garcia D, Buclin CP, Bürgisser N, Iudici M, Mongin D. Beyond human gold standards: A multimodel framework for automated abstract classification and information extraction. Research Synthesis Methods. 2026. doi:10.1017/rsm.2025.10054
BibTeX
@article{Courvoisier2026,
author = {Courvoisier, Delphine S and Buitrago-Garcia, Diana and Buclin, Clément P and Bürgisser, Nils and Iudici, Michele and Mongin, Denis},
title = {Beyond human gold standards: A multimodel framework for automated abstract classification and information extraction},
journal = {Research Synthesis Methods},
year = {2026},
doi = {10.1017/rsm.2025.10054}
}
EUVIMED