EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Backbone-Dependent Gains from Recurrent Classification Heads on IndoBERT for Mental Health Help Indication in Indonesian Tweets

Rivaldo Jonathan Paath, Oktavian Lantang, Pujo Saputro

Journal of Computer Engineering · 10.11.2026 · 5 (02) · 180-193

DOI: 10.25124/cepat.v5i02.12438 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Computer Engineering, online erschienen am 10.11.2026. Verfasst von Rivaldo Jonathan Paath und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Whether recurrent classification heads benefit pre-trained Transformers equally across backbones is unclear for informal Indonesian mental health text. We curated 33,641 tweets from a public, automatically labeled corpus into needs_help (need for professional help) and non_needs_help classes, using slang normalization without stopword removal or stemming. A three-seed 2×4 ablation compared IndoBERT-Base and IndoBERT-Lite with four heads: standard first-token, bidirectional long short-term memory (BiLSTM), BiLSTM with additive attention, and an extended head adding layer normalization, residual projection, and multi-sample dropout. On IndoBERT-Lite, BiLSTM-based heads raised threshold-tuned Macro-F1 by about 1.5 percentage points (pp) and area under the receiver operating characteristic curve (AUC) by about 1 pp. On IndoBERT-Base, AUC differed by under 0.2 pp across heads, and tuned Macro-F1 differences were small and inconclusive. Attention added no clear gain beyond the BiLSTM head, and the extended head did not reduce the dropout-free training–validation gap. Square-root class weighting shifted default operating points but did not measurably change AUC or tuned Macro-F1 in four configurations. Reference-style stopword-removal runs scored lower, but the training recipe also differed, so this cannot be attributed to stopword removal. Because labels were generated by rules, results may reflect rule learnability rather than clinical validity.

Zitieren

APA

Paath, R. J., Lantang, O., & Saputro, P. (2026). Backbone-Dependent Gains from Recurrent Classification Heads on IndoBERT for Mental Health Help Indication in Indonesian Tweets. Journal of Computer Engineering, 5(02), 180-193. https://doi.org/10.25124/cepat.v5i02.12438

Vancouver

Paath RJ, Lantang O, Saputro P. Backbone-Dependent Gains from Recurrent Classification Heads on IndoBERT for Mental Health Help Indication in Indonesian Tweets. Journal of Computer Engineering. 2026;5(02):180-193. doi:10.25124/cepat.v5i02.12438

BibTeX
@article{Paath2026,
  author = {Paath, Rivaldo Jonathan and Lantang, Oktavian and Saputro, Pujo},
  title = {Backbone-Dependent Gains from Recurrent Classification Heads on IndoBERT for Mental Health Help Indication in Indonesian Tweets},
  journal = {Journal of Computer Engineering},
  year = {2026},
  volume = {5},
  number = {02},
  pages = {180--193},
  doi = {10.25124/cepat.v5i02.12438}
}
Kostenlos starten →