EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning

Emanuele Marconato, Andrea Passerini, Stefano Teso

Entropy · 22.11.2023 · 25 (12) · 1574

DOI: 10.3390/e25121574 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Entropy, online erschienen am 22.11.2023. Verfasst von Emanuele Marconato und 2 weiteren Autor:innen. Beteiligte Einrichtung: Dipartimento di Ingegneria e Scienza dell'Informazione, University of Trento, 38123 Trento, Italy. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 127 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NextGenerationEU, EU Horizon 2020 research and innovation programme.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Research on Explainable Artificial Intelligence has recently started exploring the idea of producing explanations that, rather than being expressed in terms of low-level features, are encoded in terms of interpretable concepts learned from data. How to reliably acquire such concepts is, however, still fundamentally unclear. An agreed-upon notion of concept interpretability is missing, with the result that concepts used by both post hoc explainers and concept-based neural networks are acquired through a variety of mutually incompatible strategies. Critically, most of these neglect the human side of the problem: a representation is understandable only insofar as it can be understood by the human at the receiving end. The key challenge in human-interpretable representation learning (hrl) is how to model and operationalize this human element. In this work, we propose a mathematical framework for acquiring interpretable representations suitable for both post hoc explainers and concept-based neural networks. Our formalization of hrl builds on recent advances in causal representation learning and explicitly models a human stakeholder as an external observer. This allows us derive a principled notion of alignment between the machine’s representation and the vocabulary of concepts understood by the human. In doing so, we link alignment and interpretability through a simple and intuitive name transfer game, and clarify the relationship between alignment and a well-known property of representations, namely disentanglement. We also show that alignment is linked to the issue of undesirable correlations among concepts, also known as concept leakage, and to content-style separation, all through a general information-theoretic reformulation of these properties. Our conceptualization aims to bridge the gap between the human and algorithmic sides of interpretability and establish a stepping stone for new research on human-interpretable representations.

Schlagwörter

Stichwörter der Autor:innen

alignmentcausal abstractionscausal representation learningconcept leakagedisentanglementexplainable AI

Autor:innen und Institutionen

  • Dipartimento di Ingegneria e Scienza dell'Informazione, University of Trento, 38123 Trento, Italy.
  • Dipartimento di Informatica, University of Pisa, 56126 Pisa, Italy.
  • Centro Interdipartimentale Mente/Cervello, University of Trento, 38123 Trento, Italy.

Förderung

  • NextGenerationEU
  • EU Horizon 2020 research and innovation programme

Zitieren

APA

Marconato, E., Passerini, A., & Teso, S. (2023). Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning. Entropy, 25(12), 1574. https://doi.org/10.3390/e25121574

Vancouver

Marconato E, Passerini A, Teso S. Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning. Entropy. 2023;25(12):1574. doi:10.3390/e25121574

BibTeX
@article{Marconato2023,
  author = {Marconato, Emanuele and Passerini, Andrea and Teso, Stefano},
  title = {Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning},
  journal = {Entropy},
  year = {2023},
  volume = {25},
  number = {12},
  pages = {1574},
  doi = {10.3390/e25121574}
}
Kostenlos starten →