EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Explanation Variability in Text Classification: Humans vs. LLMs

Jérémie Bogaert, Louis Escouflaire, Antonin Descampe, Cédrick Fairon, Marie-Catherine de Marneffe, François-Xavier Standaert

Computational Linguistics · 10.11.2026 · 1-31

DOI: 10.1162/coli.a.624 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computational Linguistics, online erschienen am 10.11.2026. Verfasst von Jérémie Bogaert und 5 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This article examines the variability of human and large language model (LLM) token-level explanations applied to the use case of French journalistic text classification between news and opinion articles. To do so, we compare human annotations (in the form of highlighted tokens indicating perceived decision-relevant words) with layer-wise relevance propagation (LRP) explanations from fine-tuned transformer models. Using ten texts annotated by multiple readers and classified by equivalent instantiations of a LLM, we analyze qualitative patterns and quantitative trends with a similarity score. Results show that humans highlight fewer, longer spans focused on linguistically salient cues while LRP produces more diffuse, token-level attributions. Humans also tend to agree more on the most important tokens, whereas models align better when considering all tokens, reflecting divergent sensitivities to granularity. Prediction class matters as well: Humans are more consistent on opinion texts, while models show greater stability on news. To refine our variability analysis, we apply discretization schemes aligning LRP values with categorical human judgments. Both linear and human-aligned discretization increase similarity with human explanations and thus improve visual plausibility in aggregated attention maps without altering model predictions. These findings suggest that model explanations are not systematically more variable than human ones but follow different dynamics depending on representation and scope. They also highlight the Rashomon Effect in LLM explainability, showing that agreement on outputs does not imply convergence in reasoning. Our work demonstrates how methodological choices shape explanation variability and offers practical insights for bridging faithfulness and plausibility in explainable NLP.

Zitieren

APA

Bogaert, J., Escouflaire, L., Descampe, A., Fairon, C., Marneffe, M. D., & Standaert, F. (2026). Explanation Variability in Text Classification: Humans vs. LLMs. Computational Linguistics, 1-31. https://doi.org/10.1162/coli.a.624

Vancouver

Bogaert J, Escouflaire L, Descampe A, Fairon C, Marneffe MD, Standaert F. Explanation Variability in Text Classification: Humans vs. LLMs. Computational Linguistics. 2026:1-31. doi:10.1162/coli.a.624

BibTeX
@article{Bogaert2026,
  author = {Bogaert, Jérémie and Escouflaire, Louis and Descampe, Antonin and Fairon, Cédrick and Marneffe, Marie-Catherine de and Standaert, François-Xavier},
  title = {Explanation Variability in Text Classification: Humans vs. LLMs},
  journal = {Computational Linguistics},
  year = {2026},
  pages = {1--31},
  doi = {10.1162/coli.a.624}
}
Kostenlos starten →