Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance
Open Mind · 17.10.2025 · 9 · 1728-1762
DOI: 10.1162/opmi.a.40 ↗Kurz gefasst
Fachartikel in der Zeitschrift Open Mind, online erschienen am 17.10.2025. Verfasst von Sascha Meyen und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, University of Tübingen, Tübingen, Germany. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 126 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deutsche Forschungsgemeinschaft, Institutional Strategy of University of Tübingen, Cluster of Excellence “Machine Learning: New Perspectives for Science”.
Abstract (Originalsprache)
Metacognition comprises the ability to differentiate the accuracy of predictions about the world. This is often called Type 2 performance (with Type 1 performance being the overall accuracy). Typical measures of metacognition are based on signal detection theory and require the strong assumption of truncated normal noise underlying confidence ratings. To minimize distributional assumptions, measures based on classical information theory have been proposed. We further this approach by providing bounds on its key quantity, the transmitted information. We show that classifiers making predictions with a certain accuracy can transmit information only within a limited range, depending on the underlying noise distribution: The lowest transmitted information indicates the worst Type 2 performance and corresponds to binary noise; the highest transmitted information indicates the best Type 2 performance and corresponds to uniform noise. Because normal noise is only an intermediate case, traditional measures based on this assumption can bias interpretations of Type 2 performance. Based on these bounds, we suggest a new measure: Relative metainformation (RMI). RMI scales from 0 (lower bound) to 1 (upper bound) and therefore advances towards the much-needed decoupling of Type 2 from Type 1 performance measures. To demonstrate the strengths of RMI, we apply it to groups: In a setting where multiple independent group members with fixed accuracies combine their predictions in an optimal way, we show that the group performance depends directly on RMI: Group accuracy is best vs. worst if the group members have highest vs. lowest RMI values. Overall, our theoretical bounds allow to better evaluate measures of Type 2 and group performance.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science, University of Tübingen, Tübingen, Germany.
- Tübingen AI Center, Tübingen, Germany.
ORCID-Profile: Sascha Meyen, Frieder Göppert, Carina Schrenk, Volker H Franz
Förderung
- Deutsche Forschungsgemeinschaft
- Institutional Strategy of University of Tübingen
- Cluster of Excellence “Machine Learning: New Perspectives for Science”
Zitieren
APA
Meyen, S., Göppert, F., Schrenk, C., von Luxburg, U., & Franz, V. H. (2025). Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance. Open Mind, 9, 1728-1762. https://doi.org/10.1162/opmi.a.40
Vancouver
Meyen S, Göppert F, Schrenk C, von Luxburg U, Franz VH. Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance. Open Mind. 2025;9:1728-1762. doi:10.1162/opmi.a.40
BibTeX
@article{Meyen2025,
author = {Meyen, Sascha and Göppert, Frieder and Schrenk, Carina and von Luxburg, Ulrike and Franz, Volker H},
title = {Information-Theoretic Measures of Metacognition: Bounds and Relation to Group Performance},
journal = {Open Mind},
year = {2025},
volume = {9},
pages = {1728--1762},
doi = {10.1162/opmi.a.40}
}
EUVIMED