EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Larger models yield better results? Streamlined severity classification of ADHD-related concerns using BERT-based knowledge distillation

Ahmed Akib Jawad Karim iD, Kazi Hafiz Md Asad iD, Md Golam Rabiul Alam

PLOS ONE · 06.02.2025 · 20 (2) · e0315829

DOI: 10.1371/journal.pone.0315829 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, online erschienen am 06.02.2025. Verfasst von Ahmed Akib Jawad Karim und 2 weiteren Autor:innen. Beteiligte Einrichtung: Computer Science and Engineering, BRAC University, Dhaka, Bangladesh. Hauptthemen laut MEDLINE: Attention Deficit Disorder with Hyperactivity, Natural Language Processing. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This work focuses on the efficiency of the knowledge distillation approach in generating a lightweight yet powerful BERT-based model for natural language processing (NLP) applications. After the model creation, we applied the resulting model, LastBERT, to a real-world task—classifying severity levels of Attention Deficit Hyperactivity Disorder (ADHD)-related concerns from social media text data. Referring to LastBERT, a customized student BERT model, we significantly lowered model parameters from 110 million BERT base to 29 million-resulting in a model approximately 73.64% smaller. On the General Language Understanding Evaluation (GLUE) benchmark, comprising paraphrase identification, sentiment analysis, and text classification, the student model maintained strong performance across many tasks despite this reduction. The model was also used on a real-world ADHD dataset with an accuracy of 85%, F1 score of 85%, precision of 85%, and recall of 85%. When compared to DistilBERT (66 million parameters) and ClinicalBERT (110 million parameters), LastBERT demonstrated comparable performance, with DistilBERT slightly outperforming it at 87%, and ClinicalBERT achieving 86% across the same metrics. These findings highlight the LastBERT model’s capacity to classify degrees of ADHD severity properly, so it offers a useful tool for mental health professionals to assess and comprehend material produced by users on social networking platforms. The study emphasizes the possibilities of knowledge distillation to produce effective models fit for use in resource-limited conditions, hence advancing NLP and mental health diagnosis. Furthermore underlined by the considerable decrease in model size without appreciable performance loss is the lower computational resources needed for training and deployment, hence facilitating greater applicability. Especially using readily available computational tools like Google Colab and Kaggle Notebooks. This study shows the accessibility and usefulness of advanced NLP methods in pragmatic world applications.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Attention Deficit Disorder with HyperactivityNatural Language ProcessingHumansSocial MediaSeverity of Illness Index

Autor:innen und Institutionen

  • Computer Science and Engineering, BRAC University, Dhaka, Bangladesh.
  • Electrical and Computer Engineering, North South University, Dhaka, Bangladesh.

ORCID-Profile: Ahmed Akib Jawad Karim, Kazi Hafiz Md Asad

Zitieren

APA

Karim, A. A. J., Asad, K. H. M., & Alam, M. G. R. (2025). Larger models yield better results? Streamlined severity classification of ADHD-related concerns using BERT-based knowledge distillation. PLOS ONE, 20(2), e0315829. https://doi.org/10.1371/journal.pone.0315829

Vancouver

Karim AAJ, Asad KHM, Alam MGR. Larger models yield better results? Streamlined severity classification of ADHD-related concerns using BERT-based knowledge distillation. PLOS ONE. 2025;20(2):e0315829. doi:10.1371/journal.pone.0315829

BibTeX
@article{Karim2025,
  author = {Karim, Ahmed Akib Jawad and Asad, Kazi Hafiz Md and Alam, Md Golam Rabiul},
  title = {Larger models yield better results? Streamlined severity classification of ADHD-related concerns using BERT-based knowledge distillation},
  journal = {PLOS ONE},
  year = {2025},
  volume = {20},
  number = {2},
  pages = {e0315829},
  doi = {10.1371/journal.pone.0315829}
}
Kostenlos starten →