EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance Metrics for Multilabel Emotion Classification: Comparing Micro, Macro, and Weighted F1-Scores

Maria Cristina Hinojosa Lee, Johan Braet, Johan Springael

Applied Sciences · 28.10.2024 · 14 (21) · 9863

DOI: 10.3390/app14219863 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 28.10.2024. Verfasst von Maria Cristina Hinojosa Lee und 2 weiteren Autor:innen. Die Arbeit wurde bisher 162-mal zitiert (Crossref) und nennt 61 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study compares various F1-score variants—micro, macro, and weighted—to assess their performance in evaluating text-based emotion classification. Lexicon distillation is employed using the multilabel emotion-annotated datasets XED and GoEmotions. The aim of this paper is to understand when each F1-score variant is better suited for evaluating text-based multilabel emotion classification. Unigram lexicons were derived from the annotated GoEmotions and XED datasets through a binary classification approach. The distilled lexicons were then applied to the GoEmotions and XED annotated datasets to calculate their emotional content, and the results were compared. The findings highlight the behavior of each F1-score variant under different class distributions, emphasizing the importance of appropriate metric selection for reliable model performance evaluation in imbalanced multilabel datasets. Additionally, this study also investigates the effect of the aggregation of negative emotions into broader categories on said F1 metrics. The contribution of this study is to provide insights into how different F1-score variants could improve the reliability of multilabel emotion classifier evaluation, particularly in the context of class imbalance present in the case of phishing emails.

Zitieren

APA

Lee, M. C. H., Braet, J., & Springael, J. (2024). Performance Metrics for Multilabel Emotion Classification: Comparing Micro, Macro, and Weighted F1-Scores. Applied Sciences, 14(21), 9863. https://doi.org/10.3390/app14219863

Vancouver

Lee MCH, Braet J, Springael J. Performance Metrics for Multilabel Emotion Classification: Comparing Micro, Macro, and Weighted F1-Scores. Applied Sciences. 2024;14(21):9863. doi:10.3390/app14219863

BibTeX
@article{Lee2024,
  author = {Lee, Maria Cristina Hinojosa and Braet, Johan and Springael, Johan},
  title = {Performance Metrics for Multilabel Emotion Classification: Comparing Micro, Macro, and Weighted F1-Scores},
  journal = {Applied Sciences},
  year = {2024},
  volume = {14},
  number = {21},
  pages = {9863},
  doi = {10.3390/app14219863}
}
Kostenlos starten →