Backbone-Dependent Gains from Recurrent Classification Heads on IndoBERT for Mental Health Help Indication in Indonesian Tweets
Journal of Computer Engineering · 10.11.2026 · 5 (02) · 180-193
DOI: 10.25124/cepat.v5i02.12438 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Computer Engineering, online erschienen am 10.11.2026. Verfasst von Rivaldo Jonathan Paath und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
Whether recurrent classification heads benefit pre-trained Transformers equally across backbones is unclear for informal Indonesian mental health text. We curated 33,641 tweets from a public, automatically labeled corpus into needs_help (need for professional help) and non_needs_help classes, using slang normalization without stopword removal or stemming. A three-seed 2×4 ablation compared IndoBERT-Base and IndoBERT-Lite with four heads: standard first-token, bidirectional long short-term memory (BiLSTM), BiLSTM with additive attention, and an extended head adding layer normalization, residual projection, and multi-sample dropout. On IndoBERT-Lite, BiLSTM-based heads raised threshold-tuned Macro-F1 by about 1.5 percentage points (pp) and area under the receiver operating characteristic curve (AUC) by about 1 pp. On IndoBERT-Base, AUC differed by under 0.2 pp across heads, and tuned Macro-F1 differences were small and inconclusive. Attention added no clear gain beyond the BiLSTM head, and the extended head did not reduce the dropout-free training–validation gap. Square-root class weighting shifted default operating points but did not measurably change AUC or tuned Macro-F1 in four configurations. Reference-style stopword-removal runs scored lower, but the training recipe also differed, so this cannot be attributed to stopword removal. Because labels were generated by rules, results may reflect rule learnability rather than clinical validity.
Zitieren
APA
Paath, R. J., Lantang, O., & Saputro, P. (2026). Backbone-Dependent Gains from Recurrent Classification Heads on IndoBERT for Mental Health Help Indication in Indonesian Tweets. Journal of Computer Engineering, 5(02), 180-193. https://doi.org/10.25124/cepat.v5i02.12438
Vancouver
Paath RJ, Lantang O, Saputro P. Backbone-Dependent Gains from Recurrent Classification Heads on IndoBERT for Mental Health Help Indication in Indonesian Tweets. Journal of Computer Engineering. 2026;5(02):180-193. doi:10.25124/cepat.v5i02.12438
BibTeX
@article{Paath2026,
author = {Paath, Rivaldo Jonathan and Lantang, Oktavian and Saputro, Pujo},
title = {Backbone-Dependent Gains from Recurrent Classification Heads on IndoBERT for Mental Health Help Indication in Indonesian Tweets},
journal = {Journal of Computer Engineering},
year = {2026},
volume = {5},
number = {02},
pages = {180--193},
doi = {10.25124/cepat.v5i02.12438}
}
EUVIMED