EUVIMEDEUVIMED entdecken
Crossref

Predicting Satisfaction With Chat-Counseling at a 24/7 Chat Hotline for the Youth

Silvan Hornstein, Ulrike Lueken, Richard Wundrack, Hilbert, Kevin

edoc Publication server (Humboldt University of Berlin) · 18.02.2025

DOI: 10.18452/33052 ↗

Kurz gefasst

Fachartikel in der Zeitschrift edoc Publication server (Humboldt University of Berlin), erschienen am 18.02.2025. Verfasst von Silvan Hornstein und 4 weiteren Autor:innen.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Chat-based counseling services are popular for the low-threshold provision of mental health support to youth. In addition, they are particularly suitable for the utilization of natural language processing (NLP) for improved provision of care.

Objective

Consequently, this paper evaluates the feasibility of such a use case, namely, the NLP-based automated evaluation of satisfaction with the chat interaction. This preregistered approach could be used for evaluation and quality control procedures, as it is particularly relevant for those services.

Methods

The consultations of 2609 young chatters (around 140,000 messages) and corresponding feedback were used to train and evaluate classifiers to predict whether a chat was perceived as helpful or not. On the one hand, we trained a word vectorizer in combination with an extreme gradient boosting (XGBoost) classifier, applying cross-validation and extensive hyperparameter tuning. On the other hand, we trained several transformer-based models, comparing model types, preprocessing, and over- and undersampling techniques. For both model types, we selected the best-performing approach on the training set for a final performance evaluation on the 522 users in the final test set.

Results

The fine-tuned XGBoost classifier achieved an area under the receiver operating characteristic score of 0.69 (P

Schlagwörter

chat counselingyouth mental healthuser satisfaction predictionperceived helpfulnessnatural language processingXGBoosttransformer modelsLongformerSHAPquality control

Zitieren

APA

Hornstein, S., Lueken, U., Wundrack, R., Hilbert, & Kevin (2025). Predicting Satisfaction With Chat-Counseling at a 24/7 Chat Hotline for the Youth. edoc Publication server (Humboldt University of Berlin). https://doi.org/10.18452/33052

Vancouver

Hornstein S, Lueken U, Wundrack R, Hilbert, Kevin. Predicting Satisfaction With Chat-Counseling at a 24/7 Chat Hotline for the Youth. edoc Publication server (Humboldt University of Berlin). 2025. doi:10.18452/33052

BibTeX
@article{Hornstein2025,
  author = {Hornstein, Silvan and Lueken, Ulrike and Wundrack, Richard and Hilbert and Kevin},
  title = {Predicting Satisfaction With Chat-Counseling at a 24/7 Chat Hotline for the Youth},
  journal = {edoc Publication server (Humboldt University of Berlin)},
  year = {2025},
  doi = {10.18452/33052}
}
Kostenlos starten →