EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification

Jahda Rusti Putri, Ermatita, Abdiansah

Journal of Information Systems and Informatics · 25.06.2026 · 8 (3) · 3595-3622

DOI: 10.63158/journalisi.v8i3.1664 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Information Systems and Informatics, online erschienen am 25.06.2026. Verfasst von Jahda Rusti Putri und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study aims to analyze class-level model behavior under metric disagreement in imbalanced multi-label Indonesian emotion classification, using the divergence between Macro F1 and Micro F1 as a diagnostic signal rather than a mere performance indicator. A machine-translated Indonesian version of the GoEmotions dataset, comprising approximately 58,000 samples across 28 fine-grained emotion categories, is used as the experimental setting. The translated dataset was not manually revalidated, and findings are scoped to this translated GoEmotions setting. Two transformer-based models are evaluated: IndoBERT, a monolingual Indonesian model, and DistilBERT, a multilingual model, both fine-tuned with class-specific threshold optimization. The results reveal opposing divergence patterns: IndoBERT achieves higher Micro F1 than Macro F1, indicating performance concentrated on high-frequency classes, while DistilBERT exhibits the reverse pattern, suggesting broader but less precise label activation. Per-class analysis further shows that most minority classes consistently fall into unstable or non-functional performance regimes across both models. This study concludes that aggregate metrics alone are insufficient for evaluating model behavior in imbalanced multi-label settings. A behavior-oriented interpretation framework for Macro–Micro F1 divergence and a regime-based class reliability categorization are proposed to support more structured and informative evaluation practices.

Zitieren

APA

Putri, J. R., Ermatita, & Abdiansah (2026). Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification. Journal of Information Systems and Informatics, 8(3), 3595-3622. https://doi.org/10.63158/journalisi.v8i3.1664

Vancouver

Putri JR, Ermatita, Abdiansah. Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification. Journal of Information Systems and Informatics. 2026;8(3):3595-3622. doi:10.63158/journalisi.v8i3.1664

BibTeX
@article{Putri2026,
  author = {Putri, Jahda Rusti and Ermatita and Abdiansah},
  title = {Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification},
  journal = {Journal of Information Systems and Informatics},
  year = {2026},
  volume = {8},
  number = {3},
  pages = {3595--3622},
  doi = {10.63158/journalisi.v8i3.1664}
}
Kostenlos starten →