Human–AI collectives most accurately diagnose clinical vignettes
Proceedings of the National Academy of Sciences · 17.06.2025 · 122 (24)
DOI: 10.1073/pnas.2426153122 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the National Academy of Sciences, online erschienen am 13.06.2025. Verfasst von Nikolas Zöller und 12 weiteren Autor:innen. Beteiligte Einrichtung: Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin 14195, Germany. Hauptthemen laut MEDLINE: Artificial Intelligence. Die Arbeit wurde bisher 58-mal zitiert (Crossref) und nennt 97 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: EC | ERC | HORIZON EUROPE European Research Council (ERC), Deutsche Forschungsgemeinschaft (DFG).
Abstract (Originalsprache)
AI systems, particularly large language models (LLMs), are increasingly being employed in high-stakes decisions that impact both individuals and society at large, often without adequate safeguards to ensure safety, quality, and equity. Yet LLMs hallucinate, lack common sense, and are biased—shortcomings that may reflect LLMs’ inherent limitations and thus may not be remedied by more sophisticated architectures, more data, or more human feedback. Relying solely on LLMs for complex, high-stakes decisions is therefore problematic. Here, we present a hybrid collective intelligence system that mitigates these risks by leveraging the complementary strengths of human experience and the vast information processed by LLMs. We apply our method to open-ended medical diagnostics, combining 40,762 differential diagnoses made by physicians with the diagnoses of five state-of-the art LLMs across 2,133 text-based medical case vignettes. We show that hybrid collectives of physicians and LLMs outperform both single physicians and physician collectives, as well as single LLMs and LLM ensembles. This result holds across a range of medical specialties and professional experience and can be attributed to humans’ and LLMs’ complementary contributions that lead to different kinds of errors. Our approach highlights the potential for collective human and machine intelligence to improve accuracy in complex, open-ended domains like medical diagnostics.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin 14195, Germany.
- The Human Diagnosis Project, San Francisco, CA 94110.
- Department of Digital Humanities, University of Cologne, Cologne 50931, Germany.
- Harvey Mudd College, Claremont, CA 91711.
- Department of Politics and International Relations, Oxford University, Oxford OX13UQ, United Kingdom.
- Kaiser Permanente, Downey, CA 90242.
ORCID-Profile: Nikolas Zöller, Julian Berger, Nathan Fu, Gioele Barabucci, Kyle Laskowski, Victor Shia, Benjamin Harack, Eugene A Chu, Vito Trianni, Ralf H J M Kurvers, Stefan M Herzog
Förderung
- EC | ERC | HORIZON EUROPE European Research Council (ERC)
- Deutsche Forschungsgemeinschaft (DFG)
Zitieren
APA
Zöller, N., Berger, J., Lin, I., Fu, N., Komarneni, J., Barabucci, G., Laskowski, K., Shia, V., Harack, B., Chu, E. A., Trianni, V., Kurvers, R. H. J. M., & Herzog, S. M. (2025). Human–AI collectives most accurately diagnose clinical vignettes. Proceedings of the National Academy of Sciences, 122(24). https://doi.org/10.1073/pnas.2426153122
Vancouver
Zöller N, Berger J, Lin I, Fu N, Komarneni J, Barabucci G, et al. Human–AI collectives most accurately diagnose clinical vignettes. Proceedings of the National Academy of Sciences. 2025;122(24). doi:10.1073/pnas.2426153122
BibTeX
@article{Zller2025,
author = {Zöller, Nikolas and Berger, Julian and Lin, Irving and Fu, Nathan and Komarneni, Jayanth and Barabucci, Gioele and Laskowski, Kyle and Shia, Victor and Harack, Benjamin and Chu, Eugene A and Trianni, Vito and Kurvers, Ralf H J M and Herzog, Stefan M},
title = {Human–AI collectives most accurately diagnose clinical vignettes},
journal = {Proceedings of the National Academy of Sciences},
year = {2025},
volume = {122},
number = {24},
doi = {10.1073/pnas.2426153122}
}
EUVIMED