Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media
PeerJ Computer Science · 26.05.2026 · 12 · e3927
DOI: 10.7717/peerj-cs.3927 ↗Kurz gefasst
Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 26.05.2026. Verfasst von Amalia Amalia und 6 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Directorate of Research, Technology, and Community Service (DRTPM), Ministry of Education, Culture, Research, and Technology of the Republic of Indonesia.
Abstract (Originalsprache)
Text Emotion Analysis (TEA) has progressed rapidly, but many studies still approach emotion as a single-label problem and focus solely on text. Research on TEA in low-resource languages such as Indonesian remains relatively scarce, and most available datasets do not explicitly capture paralinguistic cues, such as emojis, even though emojis often convey strong emotional signals. To bridge this gap, we propose an emoji-aware multimodal framework for multi-label emotion classification in Indonesian social media, using the eight Plutchik emotions as labels. We constructed the dataset using a semi-automated annotation pipeline: distant supervision was used to bootstrap labels, and experts then refined them by considering both text and emoji signals. The release contains 139,414 cleaned Indonesian social media posts from YouTube, X (formerly Twitter), Reddit, and Steam. We provide (i) a mixed corpus with posts both with and without emojis and (ii) an emoji-rich subset for emoji-aware experiments, along with the annotation guidelines and fixed train/validation/test splits. For all model variants, Indonesian Bidirectional Encoder Representations from Transformers (IndoBERT) serves as the text backbone; the only difference lies in how emoji information is incorporated. We tested three integration options for emojis: adding them early (by concatenation or tokenizer expansion), fusing them mid-stream with cross-attention, or combining predictions late (average/weighted). Compared to a text-only baseline, emoji-aware models perform better overall, reflecting the fact that emojis often convey emotional cues that text alone misses. The best scores come from tokenizer expansion (micro-averaged F1 = 0.8554; macro-averaged F1 = 0.7769), likely because emojis are processed alongside words through IndoBERT’s contextual layers. Beyond model comparisons, we also introduce an Indonesian emoji-aware multi-label dataset and release fixed splits and guidelines to support reproducible research.
Förderung
- Directorate of Research, Technology, and Community Service (DRTPM), Ministry of Education, Culture, Research, and Technology of the Republic of Indonesia
Zitieren
APA
Amalia, A., Lydia, M. S., Rangkuti, R. P., Marulitua, F. P., Hanif, F., Fitra, M., & Gunawan, D. (2026). Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media. PeerJ Computer Science, 12, e3927. https://doi.org/10.7717/peerj-cs.3927
Vancouver
Amalia A, Lydia MS, Rangkuti RP, Marulitua FP, Hanif F, Fitra M, et al. Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media. PeerJ Computer Science. 2026;12:e3927. doi:10.7717/peerj-cs.3927
BibTeX
@article{Amalia2026,
author = {Amalia, Amalia and Lydia, Maya Silvi and Rangkuti, Rahmi Putri and Marulitua, Farhan Purwanto and Hanif, Fikri and Fitra, Mardanan and Gunawan, Dani},
title = {Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media},
journal = {PeerJ Computer Science},
year = {2026},
volume = {12},
pages = {e3927},
doi = {10.7717/peerj-cs.3927}
}
EUVIMED