MentalRoBERTa-Caps: A capsule-enhanced transformer model for mental health classification
MethodsX · 2025-12 · 15 · 103483
DOI: 10.1016/j.mex.2025.103483 ↗Kurz gefasst
Fachartikel in der Zeitschrift MethodsX, 2025-12. Verfasst von Faheem Ahmad Wagay und 2 weiteren Autor:innen. Beteiligte Einrichtung: Jamia Millia Islamia University, New Delhi 10025, India. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 11 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In recent years, the dominance of Large Language Models (LLMs) such as BERT and RoBERTa has led to remarkable improvements in NLP tasks, including mental illness detection from social media text. However, these models are often computationally intensive, requiring significant processing time and resources, which limits their applicability in real-time or resource-constrained environments. This paper proposes a lightweight yet effective hybrid model that integrates a 6-layer RoBERTa encoder with a capsule network architecture to balance performance, interpretability, and computational efficiency. The contextual embeddings generated by RoBERTa are transformed into primary capsules, and dynamic routing is employed to generate class capsule outputs that capture high-level abstractions. To validate performance and explainability, we employ LIME (Local Interpretable Model-Agnostic Explanations) to provide insights into feature contributions and model decisions. Experimental results on benchmark mental health datasets demonstrate that our approach achieves high accuracy while significantly reducing inference time, making it suitable for deployment in real-world mental health monitoring systems.1.To design a computationally efficient architecture for mental illness detection using a lightweight RoBERTa encoder integrated with capsule networks.2.To perform a detailed time complexity analysis highlighting the trade-offs between performance and efficiency.3.To enhance model interpretability through LIME-based feature attribution, supporting transparent and ex- plainable predictions.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Jamia Millia Islamia University, New Delhi 10025, India.
- Maulana Azad National Urdu University, Gachibowli, Hyderabad 500032, India.
Zitieren
APA
Wagay, F. A., Jahiruddin, & Altaf, Y. (2025). MentalRoBERTa-Caps: A capsule-enhanced transformer model for mental health classification. MethodsX, 15, 103483. https://doi.org/10.1016/j.mex.2025.103483
Vancouver
Wagay FA, Jahiruddin, Altaf Y. MentalRoBERTa-Caps: A capsule-enhanced transformer model for mental health classification. MethodsX. 2025;15:103483. doi:10.1016/j.mex.2025.103483
BibTeX
@article{Wagay2025,
author = {Wagay, Faheem Ahmad and Jahiruddin and Altaf, Yasir},
title = {MentalRoBERTa-Caps: A capsule-enhanced transformer model for mental health classification},
journal = {MethodsX},
year = {2025},
volume = {15},
pages = {103483},
doi = {10.1016/j.mex.2025.103483}
}
EUVIMED