EUVIMEDEUVIMED entdecken
Crossref

Evaluating a Large Language Model Conversational Agent for Collecting Standardized Back Pain Data: A Cross-Over Study of Accuracy, Usability, and User Preference

David Fraile Navarro, Mor Peleg

Methods of Information in Medicine · 20.11.2026

DOI: 10.1055/a-3009-7011 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Methods of Information in Medicine, erschienen am 20.11.2026. Verfasst von David Fraile Navarro und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Collecting standardized patient-reported outcome data through forms is burdensome for patients and clinicians.

Objectives

To evaluate a large language model (LLM)-based conversational agent (CA), intended for patients with back pain, that collects standardized back pain data through conversation: its data quality and perceived usability, and its completion time and user preference compared with a form.

Methods

We developed a CA using Generative Pre-trained Transformer (GPT) 5.0 to collect the 43-item NIH Recommended Minimal Dataset for chronic low back pain through conversation. In a cross-over study with 30 university students serving as proxies for patients with back pain, we compared CA-mediated data collection against standard form completion. Outcomes included completeness, correctness, and efficiency of data capture; usability and perceived usefulness (System Usability Scale, Technology Acceptance Model); conversational quality (Grice's maxims); intention to use; task completion time; and user preference. Qualitative analysis of conversations identified behavioral themes.

Results

Completeness was 100% of the items asked and correctness 98.3% ± 2.9%. Efficiency was 2.37 ± 0.52 correct data items per CA utterance. Usefulness and conversational quality were rated highly; the mean System Usability Scale score was 71.8 (SD 18.2). CA data collection took significantly longer than form completion (median 13.6 vs. 6.4 minutes; p = .001). Overall, 56% preferred the CA, 30% the form, and 15% had no preference. Qualitative analysis revealed intelligent behaviors (contextual inference, handling contradictory information, flexible coding) and limitations (overconfident coding, misuse of confidence indicators, occasional miscoding).

Conclusions

The CA collected structured clinical data accurately through conversation and was rated as usable by the student participants, though at increased completion time. Qualitative analysis identified areas for prompt refinement, particularly confidence signaling and ambiguous responses. Evaluation with patients is the next step.

Zitieren

APA

Navarro, D. F., & Peleg, M. (2026). Evaluating a Large Language Model Conversational Agent for Collecting Standardized Back Pain Data: A Cross-Over Study of Accuracy, Usability, and User Preference. Methods of Information in Medicine. https://doi.org/10.1055/a-3009-7011

Vancouver

Navarro DF, Peleg M. Evaluating a Large Language Model Conversational Agent for Collecting Standardized Back Pain Data: A Cross-Over Study of Accuracy, Usability, and User Preference. Methods of Information in Medicine. 2026. doi:10.1055/a-3009-7011

BibTeX
@article{Navarro2026,
  author = {Navarro, David Fraile and Peleg, Mor},
  title = {Evaluating a Large Language Model Conversational Agent for Collecting Standardized Back Pain Data: A Cross-Over Study of Accuracy, Usability, and User Preference},
  journal = {Methods of Information in Medicine},
  year = {2026},
  doi = {10.1055/a-3009-7011}
}
Kostenlos starten →