EUVIMEDEUVIMED entdecken
BeobachtungsstudieOpen Access

Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study

Lea Herschbach iD, Teresa Festl-Wietek iD, Christian Stegemann-Philipps iD, Alessandra Sonanini iD, Bernd Herrmann iD, Rebecca Erschens iD, Anne Herrmann-Werner iD, Friederike Holderried iD

Journal of Medical Internet Research · 28.11.2025 · 27 · e82818

DOI: 10.2196/82818 ↗

Kurz gefasst

Beobachtungsstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 28.11.2025. Verfasst von Lea Herschbach und 7 weiteren Autor:innen. Beteiligte Einrichtung: TIME - Tuebingen Institute for Medical Education, University of Tuebingen, Tuebingen, Germany. Hauptthemen laut MEDLINE: Artificial Intelligence, Communication, Health Personnel. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 60 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Effective communication is essential in medical practice, especially in dealing with the increasing number of patients presenting with mental health conditions. Feedback plays a crucial role in improving communication skills but is often difficult to implement. Artificial Intelligence (AI)–based tools offer promising support for practicing and improving communication techniques among physicians.

Objective

This proof-of-concept study investigated the capability of an AI-based chatbot that provided physicians with real-time feedback to train communication techniques. Specifically, it addressed three research questions: (1) How accurate is the AI-generated feedback, and how is it perceived by participants? (2) Does the use of an AI-based chatbot and AI-generated feedback lead to an increased frequency in the use of specific communication techniques? (3) Do physicians perceive the training with the AI-based chatbot and AI-generated feedback as beneficial for their daily clinical practice?

Methods

The study used a proof-of-concept design using 56 anonymized chat transcripts between physicians and simulated patients. Participating physicians received automated feedback generated by the AI-based chatbot, which was designed to assess and encourage the use of specific communication techniques. Feedback accuracy and user perception were evaluated, and changes in communication behavior were assessed. Finally, participants completed a postintervention questionnaire to evaluate perceived benefits for clinical practice.

Results

The AI-generated feedback was found to be very accurate with 85.38% of the real-time feedback being partially or even totally correct. In addition, most of the participants considered the feedback accurate, with 83.64% (n=55) agreeing or totally agreeing. Furthermore, 87.27% (n=55) of the participants agreed or totally agreed to the fact that the feedback was helpful. It also appeared to support the repeated use of recommended communication techniques. Furthermore, most of the participants agreed that practicing with an AI-based chatbot helped them practice and apply new communication techniques in clinical interactions, improved their physician-patient communication, and helped them recognize mental health conditions better in everyday medical practice.

Conclusions

The results highlight the potential of AI-supported training to address key communication challenges in medicine, particularly in the context of mental health care. The combination of real-time practice and immediate feedback may foster sustained improvements in physician-patient interactions. Integrating such tools into medical education could offer a valuable complement to traditional training methods. Future research may aim to refine AI models to improve their reliability and investigate the long-term effects and objective measures.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligenceCommunicationHealth PersonnelHumansFeedbackMaleProof of Concept StudyFemaleAdultPhysician-Patient RelationsGenerative Artificial Intelligence

Stichwörter der Autor:innen

artificial intelligenceclinical competencecommunicationfeedbackmedical educationphysician-patient relationssimulated patients

Autor:innen und Institutionen

  • TIME - Tuebingen Institute for Medical Education, University of Tuebingen, Tuebingen, Germany.
  • Department of Psychosomatic Medicine and Psychotherapy, University Hospital Tuebingen, Tuebingen, Germany.

ORCID-Profile: Lea Herschbach, Teresa Festl-Wietek, Christian Stegemann-Philipps, Alessandra Sonanini, Bernd Herrmann, Rebecca Erschens, Anne Herrmann-Werner, Friederike Holderried

Zitieren

APA

Herschbach, L., Festl-Wietek, T., Stegemann-Philipps, C., Sonanini, A., Herrmann, B., Erschens, R., Herrmann-Werner, A., & Holderried, F. (2025). Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study. Journal of Medical Internet Research, 27, e82818. https://doi.org/10.2196/82818

Vancouver

Herschbach L, Festl-Wietek T, Stegemann-Philipps C, Sonanini A, Herrmann B, Erschens R, et al. Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study. Journal of Medical Internet Research. 2025;27:e82818. doi:10.2196/82818

BibTeX
@article{Herschbach2025,
  author = {Herschbach, Lea and Festl-Wietek, Teresa and Stegemann-Philipps, Christian and Sonanini, Alessandra and Herrmann, Bernd and Erschens, Rebecca and Herrmann-Werner, Anne and Holderried, Friederike},
  title = {Evaluation of an AI-Based Chatbot Providing Real-Time Feedback in Communication Training for Mental Health Care Professionals: Proof-of-Concept Observational Study},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e82818},
  doi = {10.2196/82818}
}
Kostenlos starten →