EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance

Sascha Meyen iD, Frieder Göppert iD, Carina Schrenk iD, Ulrike von Luxburg, Volker H Franz iD

Open Mind · 17.10.2025 · 9 · 1728-1762

DOI: 10.1162/opmi.a.40 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Open Mind, online erschienen am 17.10.2025. Verfasst von Sascha Meyen und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, University of Tübingen, Tübingen, Germany. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 126 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deutsche Forschungsgemeinschaft, Institutional Strategy of University of Tübingen, Cluster of Excellence “Machine Learning: New Perspectives for Science”.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Metacognition comprises the ability to differentiate the accuracy of predictions about the world. This is often called Type 2 performance (with Type 1 performance being the overall accuracy). Typical measures of metacognition are based on signal detection theory and require the strong assumption of truncated normal noise underlying confidence ratings. To minimize distributional assumptions, measures based on classical information theory have been proposed. We further this approach by providing bounds on its key quantity, the transmitted information. We show that classifiers making predictions with a certain accuracy can transmit information only within a limited range, depending on the underlying noise distribution: The lowest transmitted information indicates the worst Type 2 performance and corresponds to binary noise; the highest transmitted information indicates the best Type 2 performance and corresponds to uniform noise. Because normal noise is only an intermediate case, traditional measures based on this assumption can bias interpretations of Type 2 performance. Based on these bounds, we suggest a new measure: Relative metainformation (RMI). RMI scales from 0 (lower bound) to 1 (upper bound) and therefore advances towards the much-needed decoupling of Type 2 from Type 1 performance measures. To demonstrate the strengths of RMI, we apply it to groups: In a setting where multiple independent group members with fixed accuracies combine their predictions in an optimal way, we show that the group performance depends directly on RMI: Group accuracy is best vs. worst if the group members have highest vs. lowest RMI values. Overall, our theoretical bounds allow to better evaluate measures of Type 2 and group performance.

Schlagwörter

Stichwörter der Autor:innen

confidence weighted majority votinggroup decisionsmetacognitionmetainformationsignal detection theory

Autor:innen und Institutionen

  • Department of Computer Science, University of Tübingen, Tübingen, Germany.
  • Tübingen AI Center, Tübingen, Germany.

ORCID-Profile: Sascha Meyen, Frieder Göppert, Carina Schrenk, Volker H Franz

Förderung

  • Deutsche Forschungsgemeinschaft
  • Institutional Strategy of University of Tübingen
  • Cluster of Excellence “Machine Learning: New Perspectives for Science”

Zitieren

APA

Meyen, S., Göppert, F., Schrenk, C., von Luxburg, U., & Franz, V. H. (2025). Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance. Open Mind, 9, 1728-1762. https://doi.org/10.1162/opmi.a.40

Vancouver

Meyen S, Göppert F, Schrenk C, von Luxburg U, Franz VH. Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance. Open Mind. 2025;9:1728-1762. doi:10.1162/opmi.a.40

BibTeX
@article{Meyen2025,
  author = {Meyen, Sascha and Göppert, Frieder and Schrenk, Carina and von Luxburg, Ulrike and Franz, Volker H},
  title = {Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance},
  journal = {Open Mind},
  year = {2025},
  volume = {9},
  pages = {1728--1762},
  doi = {10.1162/opmi.a.40}
}
Kostenlos starten →