Benchmarking Psychological Lexicons and Large Language Models for Emotion Detection in Brazilian Portuguese
AI · 01.10.2025 · 6 (10) · 249
DOI: 10.3390/ai6100249 ↗Kurz gefasst
Fachartikel in der Zeitschrift AI, online erschienen am 01.10.2025. Verfasst von Thales David Domingues Aparecido und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: University of São Paulo for the International Exchange Scholarship, CALCOLO project, Fondazione VRT and Fondazione Caritro.
Abstract (Originalsprache)
Emotion detection in Brazilian Portuguese is less studied than in English. We benchmarked a large language model (Mistral 24B), a language-specific transformer model (BERTimbau), and the lexicon-based EmoAtlas for classifying emotions in Brazilian Portuguese text, with a focus on eight emotions derived from Plutchik’s model. Evaluation covered four corpora: 4000 stock-market tweets, 1000 news headlines, 5000 GoEmotions Reddit comments translated by LLMs, and 2000 DeepSeek-generated headlines. While BERTimbau achieved the highest average scores (accuracy 0.876, precision 0.529, and recall 0.423), an overlap with Mistral (accuracy 0.831, precision 0.522, and recall 0.539) and notable performance variability suggest there is no single top performer; however, both transformer-based models outperformed the lexicon-based EmoAtlas (accuracy 0.797) but required up to 40 times more computational resources. We also introduce a novel “emotional fingerprinting” methodology using a synthetically generated dataset to probe emotional alignment, which revealed an imperfect overlap in the emotional representations of the models. While LLMs deliver higher overall scores, EmoAtlas offers superior interpretability and efficiency, making it a cost-effective alternative. This work delivers the first quantitative benchmark for interpretable emotion detection in Brazilian Portuguese, with open datasets and code to foster research in multilingual natural language processing.
Förderung
- University of São Paulo for the International Exchange Scholarship
- CALCOLO project
- Fondazione VRT and Fondazione Caritro
- University of Trento
Zitieren
APA
Aparecido, T. D. D., Carrillo, A., Camargo, C. Q., & Stella, M. (2025). Benchmarking Psychological Lexicons and Large Language Models for Emotion Detection in Brazilian Portuguese. AI, 6(10), 249. https://doi.org/10.3390/ai6100249
Vancouver
Aparecido TDD, Carrillo A, Camargo CQ, Stella M. Benchmarking Psychological Lexicons and Large Language Models for Emotion Detection in Brazilian Portuguese. AI. 2025;6(10):249. doi:10.3390/ai6100249
BibTeX
@article{Aparecido2025,
author = {Aparecido, Thales David Domingues and Carrillo, Alexis and Camargo, Chico Q. and Stella, Massimo},
title = {Benchmarking Psychological Lexicons and Large Language Models for Emotion Detection in Brazilian Portuguese},
journal = {AI},
year = {2025},
volume = {6},
number = {10},
pages = {249},
doi = {10.3390/ai6100249}
}
EUVIMED