EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Crowdsourced versus large language models forecasting: evidence for the accuracy–correlation effect

Younes Jeddi iD, Jose Segovia-Martin iD, Emile Servan-Schreiber

Philosophical Transactions B · 16.04.2026 · 381 (1948)

DOI: 10.1098/rstb.2024.0456 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Philosophical Transactions B, online erschienen am 16.04.2026. Verfasst von Younes Jeddi und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Collective Intelligence, University Mohammed VI Polytechnic, Rabat, Rabat-Sale-Zemmour-Zaer11103, Morocco. Hauptthemen laut MEDLINE: Large Language Models, Crowdsourcing. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 117 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Over the past quarter century, crowdsourced forecasting has largely outperformed individual forecasters. Today, large language models (LLMs), aggregating human knowledge at scale, constitute a new form of collective intelligence (CI). A central question is how LLM predictive accuracy associates with human–AI correlation, and whether this relationship exceeds what would be expected if both merely track the same underlying truth. We investigate this through the accuracy–correlation effect (ACE), which posits that as algorithmic systems improve, they increasingly correlate with human predictions, potentially diminishing human value in hybrid ensembles. Using 76 model × prompt forecast sets from 16 LLMs on 580 resolved ForecastBench questions, we computed LLM accuracy and correlations with two human aggregates (superforecasters, general public), separately for databases (n = 526) and prediction markets (n = 54) questions. Linear mixed-effects models show a robust positive association between LLM accuracy and human–AI correlation that substantially exceeds independent-errors predictions. Correlations were lower for superforecasters than for the general public, and weaker for markets than for data questions. These results support ACE while indicating that increasing correlation reflects more than improved signal tracking alone, suggesting that simultaneous increases in accuracy and correlation may reduce optimal human weights in data-rich settings, while human judgement retains critical value in contextual reasoning scenarios. This article is part of the theme issue ‘The evolution of collective intelligence’.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsCrowdsourcingHumansForecastingPredictive Learning ModelsPrediction Algorithms

Stichwörter der Autor:innen

collective intelligenceforecastinghybrid groupslarge language models

Autor:innen und Institutionen

  • School of Collective Intelligence, University Mohammed VI Polytechnic, Rabat, Rabat-Sale-Zemmour-Zaer11103, Morocco.
  • Centre National de la Recherche Scientifique, Complex Systems Institute of Paris Ile-de-France, Paris, France.

ORCID-Profile: Younes Jeddi, Jose Segovia-Martin

Zitieren

APA

Jeddi, Y., Segovia-Martin, J., & Servan-Schreiber, E. (2026). Crowdsourced versus large language models forecasting: evidence for the accuracy–correlation effect. Philosophical Transactions B, 381(1948). https://doi.org/10.1098/rstb.2024.0456

Vancouver

Jeddi Y, Segovia-Martin J, Servan-Schreiber E. Crowdsourced versus large language models forecasting: evidence for the accuracy–correlation effect. Philosophical Transactions B. 2026;381(1948). doi:10.1098/rstb.2024.0456

BibTeX
@article{Jeddi2026,
  author = {Jeddi, Younes and Segovia-Martin, Jose and Servan-Schreiber, Emile},
  title = {Crowdsourced versus large language models forecasting: evidence for the accuracy–correlation effect},
  journal = {Philosophical Transactions B},
  year = {2026},
  volume = {381},
  number = {1948},
  doi = {10.1098/rstb.2024.0456}
}
Kostenlos starten →