EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Benchmarking Psychological Lexicons and Large Language Models for Emotion Detection in Brazilian Portuguese

Thales David Domingues Aparecido, Alexis Carrillo, Chico Q. Camargo, Massimo Stella

AI · 01.10.2025 · 6 (10) · 249

DOI: 10.3390/ai6100249 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AI, online erschienen am 01.10.2025. Verfasst von Thales David Domingues Aparecido und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: University of São Paulo for the International Exchange Scholarship, CALCOLO project, Fondazione VRT and Fondazione Caritro.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Emotion detection in Brazilian Portuguese is less studied than in English. We benchmarked a large language model (Mistral 24B), a language-specific transformer model (BERTimbau), and the lexicon-based EmoAtlas for classifying emotions in Brazilian Portuguese text, with a focus on eight emotions derived from Plutchik’s model. Evaluation covered four corpora: 4000 stock-market tweets, 1000 news headlines, 5000 GoEmotions Reddit comments translated by LLMs, and 2000 DeepSeek-generated headlines. While BERTimbau achieved the highest average scores (accuracy 0.876, precision 0.529, and recall 0.423), an overlap with Mistral (accuracy 0.831, precision 0.522, and recall 0.539) and notable performance variability suggest there is no single top performer; however, both transformer-based models outperformed the lexicon-based EmoAtlas (accuracy 0.797) but required up to 40 times more computational resources. We also introduce a novel “emotional fingerprinting” methodology using a synthetically generated dataset to probe emotional alignment, which revealed an imperfect overlap in the emotional representations of the models. While LLMs deliver higher overall scores, EmoAtlas offers superior interpretability and efficiency, making it a cost-effective alternative. This work delivers the first quantitative benchmark for interpretable emotion detection in Brazilian Portuguese, with open datasets and code to foster research in multilingual natural language processing.

Förderung

  • University of São Paulo for the International Exchange Scholarship
  • CALCOLO project
  • Fondazione VRT and Fondazione Caritro
  • University of Trento

Zitieren

APA

Aparecido, T. D. D., Carrillo, A., Camargo, C. Q., & Stella, M. (2025). Benchmarking Psychological Lexicons and Large Language Models for Emotion Detection in Brazilian Portuguese. AI, 6(10), 249. https://doi.org/10.3390/ai6100249

Vancouver

Aparecido TDD, Carrillo A, Camargo CQ, Stella M. Benchmarking Psychological Lexicons and Large Language Models for Emotion Detection in Brazilian Portuguese. AI. 2025;6(10):249. doi:10.3390/ai6100249

BibTeX
@article{Aparecido2025,
  author = {Aparecido, Thales David Domingues and Carrillo, Alexis and Camargo, Chico Q. and Stella, Massimo},
  title = {Benchmarking Psychological Lexicons and Large Language Models for Emotion Detection in Brazilian Portuguese},
  journal = {AI},
  year = {2025},
  volume = {6},
  number = {10},
  pages = {249},
  doi = {10.3390/ai6100249}
}
Kostenlos starten →