EUVIMEDEUVIMED entdecken
Open Access

Automated classification of exposure and encourage events in speech data from pediatric OCD treatment

Juan Antonio Lossio-Ventura, Samuel Frank, Grace Ringlein, Kirsten Bonson, Ardyn Olszko, Abbey Knobel, Daniel S Pine, Jennifer B Freeman, Kristen Benito, David C Jangraw, Francisco Pereira

JAMIA Open · 03.11.2025 · 8 (6)

DOI: 10.1093/jamiaopen/ooaf127 ↗

Kurz gefasst

Fachartikel in der Zeitschrift JAMIA Open, online erschienen am 09.12.2025. Verfasst von Juan Antonio Lossio-Ventura und 10 weiteren Autor:innen. Beteiligte Einrichtung: Machine Learning Core, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892, United States. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIMH NIH HHS, National Institute of Mental Health, NIMH.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To develop and evaluate an automated classification system for labeling Exposure Process Coding System (EPCS) quality codes—specifically exposure and encourage events—during in-person exposure therapy sessions using automatic speech recognition (ASR) and natural language processing techniques. Materials and Methods The system was trained and tested on 360 manually labeled pediatric Obsessive-Compulsive Disorder (OCD) therapy sessions from 3 clinical trials. Audio recordings were transcribed using ASR tools (OpenAI’s Whisper and Google Speech-to-Text). Transcription accuracy was evaluated via word error rate (WER) on manual transcriptions of 2-minute audio segments compared against ASR-generated transcripts. The resulting text was analyzed with transformer-based models, including Bidirectional Encoder Representations from Transformers (BERT), Sentence-BERT, and Meta Llama 3. Models were trained to predict EPCS codes in 2 classification settings: sequence-level classification, where events are labeled in delimited text chunks, and token-level classification, where event boundaries are unknown. Classification was performed either with fine-tuned transformer-based models, or with logistic regression on embeddings produced by each model.

Results

With respect to transcription accuracy, Whisper outperformed Google Speech-to-Text with a lower WER (0.31 vs 0.51). For sequence classification setting, Llama 3 models achieved high performance with area under the ROC curve (AUC) scores of 0.95 for exposures and 0.75 for encourage events, outperforming traditional methods and standard BERT models. In the token-level setting, fine-tuned BERT models performed best, achieving AUC scores of 0.85 for exposures and 0.75 for encourage events. Discussion and Conclusion Current ASR and transformer-based models enable automated quality coding of in-person exposure therapy sessions. These findings demonstrate potential for real-time assessment in clinical practice and scalable research on effective therapy methods. Future work should focus on optimization, including improvements in ASR accuracy, expanding training datasets, and multimodal data integration.

Schlagwörter

Stichwörter der Autor:innen

anxietyautomatic speech recognitioncognitive behavioral therapylarge language modelsnatural language processingobsessive-compulsive disordertransformer models

Autor:innen und Institutionen

  • Machine Learning Core, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892, United States.
  • Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health, Bethesda, MD 20892, United States.
  • Department of Electrical and Biomedical Engineering, University of Vermont, Burlington, VT 05405, United States.
  • Psychiatry and Human Behavior, Warren Alpert Medical School, Brown University, East Providence, RI 02915, United States.

Förderung

  • NIMH NIH HHS
  • National Institute of Mental Health
  • NIMH
  • Intramural Research Program of the National Institute of Mental Health

Zitieren

APA

Lossio-Ventura, J. A., Frank, S., Ringlein, G., Bonson, K., Olszko, A., Knobel, A., Pine, D. S., Freeman, J. B., Benito, K., Jangraw, D. C., & Pereira, F. (2025). Automated classification of exposure and encourage events in speech data from pediatric OCD treatment. JAMIA Open, 8(6). https://doi.org/10.1093/jamiaopen/ooaf127

Vancouver

Lossio-Ventura JA, Frank S, Ringlein G, Bonson K, Olszko A, Knobel A, et al. Automated classification of exposure and encourage events in speech data from pediatric OCD treatment. JAMIA Open. 2025;8(6). doi:10.1093/jamiaopen/ooaf127

BibTeX
@article{LossioVentura2025,
  author = {Lossio-Ventura, Juan Antonio and Frank, Samuel and Ringlein, Grace and Bonson, Kirsten and Olszko, Ardyn and Knobel, Abbey and Pine, Daniel S and Freeman, Jennifer B and Benito, Kristen and Jangraw, David C and Pereira, Francisco},
  title = {Automated classification of exposure and encourage events in speech data from pediatric OCD treatment},
  journal = {JAMIA Open},
  year = {2025},
  volume = {8},
  number = {6},
  doi = {10.1093/jamiaopen/ooaf127}
}
Kostenlos starten →