EUVIMEDEUVIMED entdecken
EvaluationsstudieOpen Access

A Multistage Fine-Tuning and Empathy Alignment Approach for an Emotional Support Dialogue System: Development and Evaluation Study

Xianbin Wang, Ruolin Wang, Wencheng Yang, Weijun Wang, Wei Pan, Min Jia, Shijie Gao, Xin Zhao

Journal of Medical Internet Research · 29.10.2026 · 28 · e95603-e95603

DOI: 10.2196/95603 ↗

Kurz gefasst

Evaluationsstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 29.10.2026. Verfasst von Xianbin Wang und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Psychology, Central China Normal University, No.152 Luoyu Road, Hongshan District, Wuhan, Hubei, 430079, China, 86 15377637661. Hauptthemen laut MEDLINE: Empathy, Emotions, Counseling, Communication. Die Arbeit nennt 76 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are increasingly being used for text-based emotional support. However, general-purpose LLMs may not consistently recognize emotional cues, adapt supportive strategies to contexts, or generate responses aligned with psychologically informed communication. The limited availability of large-scale, structured Chinese counseling data further constrains the development of models for context-sensitive emotional support.

Objective

We developed and evaluated YuanChat, a family of specialized LLMs for context-sensitive emotional support; assessed whether E-URIAL improved response quality; and compared YuanChat with base LLMs.

Methods

We constructed 2 datasets from anonymized real-world counseling sessions: ECD-3 (104,897 counselor response turns) for fine-tuning and D gold (9413 manually annotated counselor responses). Annotations covered emotional support conversation stages (exploration, comfort, and action), counseling techniques (clarification, support, and challenge), and empathic effects. We developed 2 YuanChat variants using GLM-4-9B-Chat and Huozi 3.5 as backbone models using parameter-efficient fine-tuning with low-rank adaptation (LoRA). We then developed E-URIAL, an inference-time framework combining predicted emotion and support strategy labels with retrieved labeled exemplars. Given the dialogue history through the help seeker’s latest utterance, each model generated a counselor-style response. We compared YuanChat with GPT-4, ERNIE 4.0, CharacterGLM, GLM-4-9B-Chat, and Huozi 3.5. The evaluation comprised (1) automatic evaluation on in-domain and external datasets, assessing empathy identification and empathic expression; (2) evaluation of empathy-related situational reasoning using an expert-developed multiple-choice benchmark; (3) evaluation by clinical experts (n=8) of empathic attunement, support strategy appropriateness, perceived helpfulness, and safety and professional boundary adherence; and (4) a complementary evaluation by volunteer master’s students in clinical and counseling psychology (n=50) of response fluency, empathy, and perceived helpfulness.

Results

Fine-tuning increased YuanChat-Huozi3.5’s empathic expression score from 0.57 to 0.68; empathy identification score increased from 0.54 to 0.59. In expert evaluation, YuanChat-Huozi3.5 received a higher empathic attunement score than the base Huozi3.5 model (1.279 vs 0.327; mean difference 0.952, 95% CI 0.746‐1.152). Using professional counselor responses as the reference, the mean differences for YuanChat-Huozi3.5 were 0.148, 0.079, 0.039, and 0.002 in empathic attunement, supportive appropriateness, perceived helpfulness, and safety and professional boundary adherence, respectively. No statistically significant differences were detected between YuanChat-Huozi3.5 and the counselor references (all Holm-adjusted P ≥.29). Higher scores were observed when E-URIAL was applied. When E-URIAL was applied with 3 retrieved exemplars, YuanChat-Huozi3.5 achieved an empathy identification score of 0.69 versus 0.59 without E-URIAL, an empathic expression score of 0.79 versus 0.68, and a perceived helpfulness score of 0.74 versus 0.61.

Conclusions

The findings indicate that fine-tuning on real-world counseling data and annotating models with structured emotional support labels improved context-sensitive empathic responding in LLMs. Without updating model parameters, E-URIAL further improved empathy identification and empathic expression across multiple models. Together, these methods provide a scalable, theory-informed approach to enhancing the emotional support capabilities of LLMs.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

EmpathyEmotionsCounselingCommunicationHumansLarge Language Models

Stichwörter der Autor:innen

YuanChatempathic dialogue systemsfine-tuninggenerative AIin-context learninglarge language modelspsychological counseling datasettext-based emotional support

Autor:innen und Institutionen

  • School of Psychology, Central China Normal University, No.152 Luoyu Road, Hongshan District, Wuhan, Hubei, 430079, China, 86 15377637661.
  • School of Modern Information Industry, Guangzhou College of Commerce, Guangzhou, Guangdong, China.
  • BeiCheng Lvluo Institute for Social Psychological Intelligence Innovation and Development, Beijing City University, Beijing, China.
  • Manchester Institute of Education, The University of Manchester, Manchester, United Kingdom.

Zitieren

APA

Wang, X., Wang, R., Yang, W., Wang, W., Pan, W., Jia, M., Gao, S., & Zhao, X. (2026). A Multistage Fine-Tuning and Empathy Alignment Approach for an Emotional Support Dialogue System: Development and Evaluation Study. Journal of Medical Internet Research, 28, e95603-e95603. https://doi.org/10.2196/95603

Vancouver

Wang X, Wang R, Yang W, Wang W, Pan W, Jia M, et al. A Multistage Fine-Tuning and Empathy Alignment Approach for an Emotional Support Dialogue System: Development and Evaluation Study. Journal of Medical Internet Research. 2026;28:e95603-e95603. doi:10.2196/95603

BibTeX
@article{Wang2026,
  author = {Wang, Xianbin and Wang, Ruolin and Yang, Wencheng and Wang, Weijun and Pan, Wei and Jia, Min and Gao, Shijie and Zhao, Xin},
  title = {A Multistage Fine-Tuning and Empathy Alignment Approach for an Emotional Support Dialogue System: Development and Evaluation Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e95603--e95603},
  doi = {10.2196/95603}
}
Kostenlos starten →