Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification
Journal of Information Systems and Informatics · 25.06.2026 · 8 (3) · 3595-3622
DOI: 10.63158/journalisi.v8i3.1664 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Information Systems and Informatics, online erschienen am 25.06.2026. Verfasst von Jahda Rusti Putri und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This study aims to analyze class-level model behavior under metric disagreement in imbalanced multi-label Indonesian emotion classification, using the divergence between Macro F1 and Micro F1 as a diagnostic signal rather than a mere performance indicator. A machine-translated Indonesian version of the GoEmotions dataset, comprising approximately 58,000 samples across 28 fine-grained emotion categories, is used as the experimental setting. The translated dataset was not manually revalidated, and findings are scoped to this translated GoEmotions setting. Two transformer-based models are evaluated: IndoBERT, a monolingual Indonesian model, and DistilBERT, a multilingual model, both fine-tuned with class-specific threshold optimization. The results reveal opposing divergence patterns: IndoBERT achieves higher Micro F1 than Macro F1, indicating performance concentrated on high-frequency classes, while DistilBERT exhibits the reverse pattern, suggesting broader but less precise label activation. Per-class analysis further shows that most minority classes consistently fall into unstable or non-functional performance regimes across both models. This study concludes that aggregate metrics alone are insufficient for evaluating model behavior in imbalanced multi-label settings. A behavior-oriented interpretation framework for Macro–Micro F1 divergence and a regime-based class reliability categorization are proposed to support more structured and informative evaluation practices.
Zitieren
APA
Putri, J. R., Ermatita, & Abdiansah (2026). Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification. Journal of Information Systems and Informatics, 8(3), 3595-3622. https://doi.org/10.63158/journalisi.v8i3.1664
Vancouver
Putri JR, Ermatita, Abdiansah. Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification. Journal of Information Systems and Informatics. 2026;8(3):3595-3622. doi:10.63158/journalisi.v8i3.1664
BibTeX
@article{Putri2026,
author = {Putri, Jahda Rusti and Ermatita and Abdiansah},
title = {Class-Level Behavior Analysis under Metric Disagreement in Imbalanced Multi-Label Indonesian Emotion Classification},
journal = {Journal of Information Systems and Informatics},
year = {2026},
volume = {8},
number = {3},
pages = {3595--3622},
doi = {10.63158/journalisi.v8i3.1664}
}
EUVIMED