Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning
Entropy · 22.11.2023 · 25 (12) · 1574
DOI: 10.3390/e25121574 ↗Kurz gefasst
Fachartikel in der Zeitschrift Entropy, online erschienen am 22.11.2023. Verfasst von Emanuele Marconato und 2 weiteren Autor:innen. Beteiligte Einrichtung: Dipartimento di Ingegneria e Scienza dell'Informazione, University of Trento, 38123 Trento, Italy. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 127 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NextGenerationEU, EU Horizon 2020 research and innovation programme.
Abstract (Originalsprache)
Research on Explainable Artificial Intelligence has recently started exploring the idea of producing explanations that, rather than being expressed in terms of low-level features, are encoded in terms of interpretable concepts learned from data. How to reliably acquire such concepts is, however, still fundamentally unclear. An agreed-upon notion of concept interpretability is missing, with the result that concepts used by both post hoc explainers and concept-based neural networks are acquired through a variety of mutually incompatible strategies. Critically, most of these neglect the human side of the problem: a representation is understandable only insofar as it can be understood by the human at the receiving end. The key challenge in human-interpretable representation learning (hrl) is how to model and operationalize this human element. In this work, we propose a mathematical framework for acquiring interpretable representations suitable for both post hoc explainers and concept-based neural networks. Our formalization of hrl builds on recent advances in causal representation learning and explicitly models a human stakeholder as an external observer. This allows us derive a principled notion of alignment between the machine’s representation and the vocabulary of concepts understood by the human. In doing so, we link alignment and interpretability through a simple and intuitive name transfer game, and clarify the relationship between alignment and a well-known property of representations, namely disentanglement. We also show that alignment is linked to the issue of undesirable correlations among concepts, also known as concept leakage, and to content-style separation, all through a general information-theoretic reformulation of these properties. Our conceptualization aims to bridge the gap between the human and algorithmic sides of interpretability and establish a stepping stone for new research on human-interpretable representations.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Dipartimento di Ingegneria e Scienza dell'Informazione, University of Trento, 38123 Trento, Italy.
- Dipartimento di Informatica, University of Pisa, 56126 Pisa, Italy.
- Centro Interdipartimentale Mente/Cervello, University of Trento, 38123 Trento, Italy.
Förderung
- NextGenerationEU
- EU Horizon 2020 research and innovation programme
Zitieren
APA
Marconato, E., Passerini, A., & Teso, S. (2023). Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning. Entropy, 25(12), 1574. https://doi.org/10.3390/e25121574
Vancouver
Marconato E, Passerini A, Teso S. Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning. Entropy. 2023;25(12):1574. doi:10.3390/e25121574
BibTeX
@article{Marconato2023,
author = {Marconato, Emanuele and Passerini, Andrea and Teso, Stefano},
title = {Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning},
journal = {Entropy},
year = {2023},
volume = {25},
number = {12},
pages = {1574},
doi = {10.3390/e25121574}
}
EUVIMED