Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study
Journal of Medical Internet Research · 28.11.2025 · 27 · e82818
DOI: 10.2196/82818 ↗Kurz gefasst
Beobachtungsstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 28.11.2025. Verfasst von Lea Herschbach und 7 weiteren Autor:innen. Beteiligte Einrichtung: TIME - Tuebingen Institute for Medical Education, University of Tuebingen, Tuebingen, Germany. Hauptthemen laut MEDLINE: Artificial Intelligence, Communication, Health Personnel. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Effective communication is essential in medical practice, especially in dealing with the increasing number of patients presenting with mental health conditions. Feedback plays a crucial role in improving communication skills but is often difficult to implement. Artificial Intelligence (AI)–based tools offer promising support for practicing and improving communication techniques among physicians.
Objective
This proof-of-concept study investigated the capability of an AI-based chatbot that provided physicians with real-time feedback to train communication techniques. Specifically, it addressed three research questions: (1) How accurate is the AI-generated feedback, and how is it perceived by participants? (2) Does the use of an AI-based chatbot and AI-generated feedback lead to an increased frequency in the use of specific communication techniques? (3) Do physicians perceive the training with the AI-based chatbot and AI-generated feedback as beneficial for their daily clinical practice?
Methods
The study used a proof-of-concept design using 56 anonymized chat transcripts between physicians and simulated patients. Participating physicians received automated feedback generated by the AI-based chatbot, which was designed to assess and encourage the use of specific communication techniques. Feedback accuracy and user perception were evaluated, and changes in communication behavior were assessed. Finally, participants completed a postintervention questionnaire to evaluate perceived benefits for clinical practice.
Results
The AI-generated feedback was found to be very accurate with 85.38% of the real-time feedback being partially or even totally correct. In addition, most of the participants considered the feedback accurate, with 83.64% (n=55) agreeing or totally agreeing. Furthermore, 87.27% (n=55) of the participants agreed or totally agreed to the fact that the feedback was helpful. It also appeared to support the repeated use of recommended communication techniques. Furthermore, most of the participants agreed that practicing with an AI-based chatbot helped them practice and apply new communication techniques in clinical interactions, improved their physician-patient communication, and helped them recognize mental health conditions better in everyday medical practice.
Conclusions
The results highlight the potential of AI-supported training to address key communication challenges in medicine, particularly in the context of mental health care. The combination of real-time practice and immediate feedback may foster sustained improvements in physician-patient interactions. Integrating such tools into medical education could offer a valuable complement to traditional training methods. Future research may aim to refine AI models to improve their reliability and investigate the long-term effects and objective measures.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- TIME - Tuebingen Institute for Medical Education, University of Tuebingen, Tuebingen, Germany.
- Department of Psychosomatic Medicine and Psychotherapy, University Hospital Tuebingen, Tuebingen, Germany.
ORCID-Profile: Lea Herschbach, Teresa Festl-Wietek, Christian Stegemann-Philipps, Alessandra Sonanini, Bernd Herrmann, Rebecca Erschens, Anne Herrmann-Werner, Friederike Holderried
Zitieren
APA
Herschbach, L., Festl-Wietek, T., Stegemann-Philipps, C., Sonanini, A., Herrmann, B., Erschens, R., Herrmann-Werner, A., & Holderried, F. (2025). Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study. Journal of Medical Internet Research, 27, e82818. https://doi.org/10.2196/82818
Vancouver
Herschbach L, Festl-Wietek T, Stegemann-Philipps C, Sonanini A, Herrmann B, Erschens R, et al. Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study. Journal of Medical Internet Research. 2025;27:e82818. doi:10.2196/82818
BibTeX
@article{Herschbach2025,
author = {Herschbach, Lea and Festl-Wietek, Teresa and Stegemann-Philipps, Christian and Sonanini, Alessandra and Herrmann, Bernd and Erschens, Rebecca and Herrmann-Werner, Anne and Holderried, Friederike},
title = {Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study},
journal = {Journal of Medical Internet Research},
year = {2025},
volume = {27},
pages = {e82818},
doi = {10.2196/82818}
}
EUVIMED