EUVIMEDOpdag EUVIMED
Open access

Conversational AI in Hereditary Cancer Care: Sociotechnical Study of Responsible Design Requirements

Conversational AI in der hereditären Krebsversorgung: soziotechnische Studie zu Anforderungen an ein verantwortungsvolles Design

Dominic Lammert, Jacqueline Lammert, Justin Hofenbitzer, Tristan Radtke, Yingping Sun, Martin Haimerl, Stefanie Betz, Doran Holger Scholl, Jürgen Pfeffer

JMIR Human Factors · 2026-09-30 · 13 · e97869-e97869

DOI: 10.2196/97869 ↗

Auf einen Blick

In 2 Workshops mit 18 Stakeholdern wurden Anforderungen an einen klinisch begrenzten Conversational-AI-Ansatz für BRCA1/2-bezogene Informationen erarbeitet. Zentrale Spannungen betrafen Zugänglichkeit, Präzision und Governance.

Fragestellung

Welche soziotechnischen Designanforderungen und Governance-Bedingungen werden für ein klinisch begrenztes Conversational-AI-System in der BRCA1/2-bezogenen Versorgung benötigt?

Methode

Explorative qualitative Vorentwicklungsstudie mit 2 strukturierten partizipativen Workshops; Analyse mit induktiver thematischer Analyse.

Ergebnisse

Es zeigten sich 3 wiederkehrende Spannungen: Unterstützung vs. Rollenbegrenzung, Vereinfachung/Emotionsberuhigung vs. medizinische Präzision, Skalierbarkeit vs. Governance/Datenschutz/Verantwortung. Daraus folgten Anforderungen an begrenzte Systemautorität, menschliche Aufsicht und kontextsensitiven Dialog.

Schlussfolgerung der Autor:innen

Conversational AI sollte in diesem Kontext explizit begrenzt und in klinische Governance-Strukturen eingebettet sein, um als verantwortungsvoll zu gelten.

Kernaussagen

  • 2 Workshops mit 18 interdisziplinären Stakeholdern
  • 3 zentrale Spannungsfelder prägten die Designanforderungen
  • Gefordert wurden Begrenzung, menschliche Aufsicht und Governance-Einbettung

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Journal article in JMIR Human Factors, published online on 2026-09-30. Written by Dominic Lammert and 8 co-authors. Institution: Chair of Computational Social Science, TUM School of Social Science and Technology, Technical University of Munich, Hochschule für Politik München, R…. Major MEDLINE topics: Artificial Intelligence, Genetic Counseling, Breast Neoplasms, Communication. 54 references. The full text is freely available in PubMed Central.

Auf einen BlickAbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Individuals with BRCA1/2 germline variants face complex, preference-sensitive medical decisions under psychological distress. Structural constraints in genetic counseling create support gaps during waiting periods, prompting patients to rely on fragmented or misleading online information. Conversational AI may offer low-threshold, continuous informational support, yet methods for its responsible integration into clinical care remain unclear.

Objective

This study identifies and systematizes stakeholder-informed sociotechnical design requirements and governance conditions for a clinically bounded conversational AI system intended to support comprehension, appraisal, and appropriate use of complex hereditary cancer information in BRCA1/2 -related care.

Methods

We conducted an exploratory, qualitative, predevelopment requirements study using 2 structured participatory workshops with interdisciplinary stakeholders (N=18). Guided exercises included stakeholder analysis, the value proposition canvas, and the sustainability awareness framework. Structured workshop outputs, field notes, and artifacts were analyzed using inductive thematic analysis.

Results

Stakeholder workshops revealed that responsible design was shaped less by consensus around standalone responsible AI principles than by three recurrent cross-stakeholder tensions: (1) accessibility and continuity of support versus clinical role boundaries, (2) simplification and emotional reassurance versus medical precision and epistemic transparency, and (3) technical scalability versus governance, data protection, and institutional accountability. These tensions translated into design requirements for constrained system authority, mandatory human oversight, context-sensitive communication, and integration into clinical and regulatory governance structures.

Conclusions

Conversational AI in this context should be explicitly constrained and embedded within clinical governance structures to be perceived as responsible. Responsible implementation requires transparent design, human oversight, integration into existing care pathways, and early alignment with German/European Union (EU) regulatory requirements concerning data protection, medical device classification, genetic data, and AI transparency.

Subjects

MeSH terms (MEDLINE)

Artificial IntelligenceGenetic CounselingBreast NeoplasmsCommunicationHumansFemaleQualitative Research

Author keywords

BRCA1/2breast cancer gene 1 and 2conversational AIgenetic counselingparticipatory designpatient decision supportresponsible AI

Authors and institutions

  • Chair of Computational Social Science, TUM School of Social Science and Technology, Technical University of Munich, Hochschule für Politik München, Richard-Wagner-Straße 1, Munich, Bavaria, 80333, Germany, +49 899077932…
  • Chair of Medical Informatics, Institute for AI and Informatics in Medicine, TUM School of Medicine and Health, TUM University Hospital, Technical University of Munich, Munich, Germany.
  • Chair of Law and Regulation of the Digital Transformation, Department of Governance, TUM School of Social Sciences and Technology, Technical University of Munich, Munich, Germany.
  • School of Business, Shandong Normal University, Jinan, Shandong, China.
  • Faculty II: Engineering and Technology, Furtwangen University, Furtwangen, Baden-Wurttemberg, Germany.
  • Faculty I:, Computer Science and Applications, Furtwangen University, Furtwangen, Baden-Wurttemberg, Germany.

Cite

APA

Lammert, D., Lammert, J., Hofenbitzer, J., Radtke, T., Sun, Y., Haimerl, M., Betz, S., Scholl, D. H., & Pfeffer, J. (2026). Conversational AI in Hereditary Cancer Care: Sociotechnical Study of Responsible Design Requirements. JMIR Human Factors, 13, e97869-e97869. https://doi.org/10.2196/97869

Vancouver

Lammert D, Lammert J, Hofenbitzer J, Radtke T, Sun Y, Haimerl M, et al. Conversational AI in Hereditary Cancer Care: Sociotechnical Study of Responsible Design Requirements. JMIR Human Factors. 2026;13:e97869-e97869. doi:10.2196/97869

BibTeX
@article{Lammert2026,
  author = {Lammert, Dominic and Lammert, Jacqueline and Hofenbitzer, Justin and Radtke, Tristan and Sun, Yingping and Haimerl, Martin and Betz, Stefanie and Scholl, Doran Holger and Pfeffer, Jürgen},
  title = {Conversational AI in Hereditary Cancer Care: Sociotechnical Study of Responsible Design Requirements},
  journal = {JMIR Human Factors},
  year = {2026},
  volume = {13},
  pages = {e97869--e97869},
  doi = {10.2196/97869}
}
Start for free →