EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media

Amalia Amalia, Maya Silvi Lydia, Rahmi Putri Rangkuti, Farhan Purwanto Marulitua, Fikri Hanif, Mardanan Fitra, Dani Gunawan

PeerJ Computer Science · 26.05.2026 · 12 · e3927

DOI: 10.7717/peerj-cs.3927 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PeerJ Computer Science, online erschienen am 26.05.2026. Verfasst von Amalia Amalia und 6 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Directorate of Research, Technology, and Community Service (DRTPM), Ministry of Education, Culture, Research, and Technology of the Republic of Indonesia.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text Emotion Analysis (TEA) has progressed rapidly, but many studies still approach emotion as a single-label problem and focus solely on text. Research on TEA in low-resource languages such as Indonesian remains relatively scarce, and most available datasets do not explicitly capture paralinguistic cues, such as emojis, even though emojis often convey strong emotional signals. To bridge this gap, we propose an emoji-aware multimodal framework for multi-label emotion classification in Indonesian social media, using the eight Plutchik emotions as labels. We constructed the dataset using a semi-automated annotation pipeline: distant supervision was used to bootstrap labels, and experts then refined them by considering both text and emoji signals. The release contains 139,414 cleaned Indonesian social media posts from YouTube, X (formerly Twitter), Reddit, and Steam. We provide (i) a mixed corpus with posts both with and without emojis and (ii) an emoji-rich subset for emoji-aware experiments, along with the annotation guidelines and fixed train/validation/test splits. For all model variants, Indonesian Bidirectional Encoder Representations from Transformers (IndoBERT) serves as the text backbone; the only difference lies in how emoji information is incorporated. We tested three integration options for emojis: adding them early (by concatenation or tokenizer expansion), fusing them mid-stream with cross-attention, or combining predictions late (average/weighted). Compared to a text-only baseline, emoji-aware models perform better overall, reflecting the fact that emojis often convey emotional cues that text alone misses. The best scores come from tokenizer expansion (micro-averaged F1 = 0.8554; macro-averaged F1 = 0.7769), likely because emojis are processed alongside words through IndoBERT’s contextual layers. Beyond model comparisons, we also introduce an Indonesian emoji-aware multi-label dataset and release fixed splits and guidelines to support reproducible research.

Förderung

  • Directorate of Research, Technology, and Community Service (DRTPM), Ministry of Education, Culture, Research, and Technology of the Republic of Indonesia

Zitieren

APA

Amalia, A., Lydia, M. S., Rangkuti, R. P., Marulitua, F. P., Hanif, F., Fitra, M., & Gunawan, D. (2026). Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media. PeerJ Computer Science, 12, e3927. https://doi.org/10.7717/peerj-cs.3927

Vancouver

Amalia A, Lydia MS, Rangkuti RP, Marulitua FP, Hanif F, Fitra M, et al. Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media. PeerJ Computer Science. 2026;12:e3927. doi:10.7717/peerj-cs.3927

BibTeX
@article{Amalia2026,
  author = {Amalia, Amalia and Lydia, Maya Silvi and Rangkuti, Rahmi Putri and Marulitua, Farhan Purwanto and Hanif, Fikri and Fitra, Mardanan and Gunawan, Dani},
  title = {Emoji-aware multimodal representations for multi-label emotion analysis in Indonesian social media},
  journal = {PeerJ Computer Science},
  year = {2026},
  volume = {12},
  pages = {e3927},
  doi = {10.7717/peerj-cs.3927}
}
Kostenlos starten →