EUVIMEDOpdag EUVIMED
Qualitative StudieOpen access

Evaluating the Prototype of a Clinical Decision Support System in Primary Care: Qualitative Study

Susanne M Köhler iD, Svea Holtz iD, Michaela C Neff iD, Jannik Schaaf iD, Michael von Wagner iD, Beate S Müller iD, Dania Schütze iD

JMIR Formative Research · 2025-08-20 · 9 · e69875-e69875

DOI: 10.2196/69875 ↗

At a glance

Qualitative Studie in JMIR Formative Research, published online on 2025-08-20. Written by Susanne M Köhler and 6 co-authors. Institution: Institute of General Practice, Goethe University Frankfurt, Theodor-Stern-Kai 7, Frankfurt am Main, 60590, Germany, 49 696301 ext 7267. Major MEDLINE topics: Decision Support Systems, Clinical, Primary Health Care, General Practitioners. Cited 7 times so far (Crossref); 41 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

General practitioners are confronted with a wide variety of diseases and sometimes diagnostic uncertainty. Clinical decision support systems could be valuable to improve diagnosis, but existing tools are not adapted to the requirements and workflow in the primary setting. In project SATURN (Smart physician portal for patients with unclear disease), the prototype of a clinical decision support system based on artificial intelligence is being developed together with users specifically for primary care in Germany. It aims to reduce diagnostic uncertainty in cases of unclear and rare diseases and focuses on 3 medical fields. A user-centered design approach is applied for prototype development and evaluation.

Objective

This study evaluates the usability of a high-fidelity prototype and explores aspects of user experience like the subjective impression, satisfaction, and areas of improvement.

Methods

A total of 5 general practitioners participated in the evaluation, which consisted of (1) a remote think-aloud test, (2) a postsession interview, and (3) a survey with the System Usability Scale. During the think-aloud tests, the participants verbalized their thoughts and actions and solved several vignette-based tasks. Remarkable observations were logged, transcribed with quotes, and analyzed for usability problems and positive findings. All observations and interview responses were deductively assigned to the following categories: (1) content, (2) comprehensibility, (3) user-friendliness, (4) layout, (5) feedback, and (6) navigation. Usability problems were described in detail and solutions for improvement proposed. Median and total scores were calculated for all questionnaire items.

Results

The evaluation detected both strengths and areas for improvement. The participants particularly liked the clear and well-structured layout of the prototype. Key issues identified were content-related limitations, such as the inability to enter unlisted symptoms, medications, and examination findings. Also, participants found the terminology for laboratory not suitable to their needs. Another key issue was a lack of user-friendliness concerning the time required to input medication plans and lab values. Participants expressed a need for faster data entry, potentially through direct imports from practice management systems or laboratory files. Adding symptom duration, weighting symptoms, and incorporating hereditary factors were suggestions made for improvement. Overall, the SATURN prototype was deemed useful and promising for future clinical use, despite the need for further refinements, particularly in the areas of data entry, as this is a key obstacle to its use.

Conclusions

The usability evaluation methods combined proved to be location independent and easy to use. They provided important findings on usability issues and improvements that will be implemented in a second high-fidelity prototype, which will also be tested by users. Technically demanding user requirements, such as direct data transfer from the practice management system and entry options that require complex data models, were beyond the scope of this project, but should be considered in future development projects.

Subjects

MeSH terms (MEDLINE)

Decision Support Systems, ClinicalPrimary Health CareGeneral PractitionersHumansQualitative ResearchGermanyMaleFemaleUser-Centered DesignMiddle AgedAdultSurveys and QuestionnairesArtificial Intelligence

Author keywords

clinical decision support systemsprimary carequalitative researchusability testinguser-centered design

Authors and institutions

  • Institute of General Practice, Goethe University Frankfurt, Theodor-Stern-Kai 7, Frankfurt am Main, 60590, Germany, 49 696301 ext 7267.
  • Institute of Medical Informatics, Goethe University Frankfurt, University Medicine Frankfurt, Frankfurt am Main, Germany.
  • Department of Medical Information Systems and Digitalization, University Medicine Frankfurt, Frankfurt am Main, Germany.
  • Institute of General Practice, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne, Germany.

ORCID profiles: Susanne M Köhler, Svea Holtz, Michaela C Neff, Jannik Schaaf, Michael von Wagner, Beate S Müller, Dania Schütze

Cite

APA

Köhler, S. M., Holtz, S., Neff, M. C., Schaaf, J., von Wagner, M., Müller, B. S., & Schütze, D. (2025). Evaluating the Prototype of a Clinical Decision Support System in Primary Care: Qualitative Study. JMIR Formative Research, 9, e69875-e69875. https://doi.org/10.2196/69875

Vancouver

Köhler SM, Holtz S, Neff MC, Schaaf J, von Wagner M, Müller BS, et al. Evaluating the Prototype of a Clinical Decision Support System in Primary Care: Qualitative Study. JMIR Formative Research. 2025;9:e69875-e69875. doi:10.2196/69875

BibTeX
@article{Khler2025,
  author = {Köhler, Susanne M and Holtz, Svea and Neff, Michaela C and Schaaf, Jannik and von Wagner, Michael and Müller, Beate S and Schütze, Dania},
  title = {Evaluating the Prototype of a Clinical Decision Support System in Primary Care: Qualitative Study},
  journal = {JMIR Formative Research},
  year = {2025},
  volume = {9},
  pages = {e69875--e69875},
  doi = {10.2196/69875}
}
Start for free →