Evaluating a Large Language Model Conversational Agent for Collecting Standardized Back Pain Data: A Cross-Over Study of Accuracy, Usability, and User Preference
Methods of Information in Medicine · 20.11.2026
DOI: 10.1055/a-3009-7011 ↗Kurz gefasst
Fachartikel in der Zeitschrift Methods of Information in Medicine, erschienen am 20.11.2026. Verfasst von David Fraile Navarro und 1 weiteren Autor:innen.
Abstract (Originalsprache)
Background
Collecting standardized patient-reported outcome data through forms is burdensome for patients and clinicians.
Objectives
To evaluate a large language model (LLM)-based conversational agent (CA), intended for patients with back pain, that collects standardized back pain data through conversation: its data quality and perceived usability, and its completion time and user preference compared with a form.
Methods
We developed a CA using Generative Pre-trained Transformer (GPT) 5.0 to collect the 43-item NIH Recommended Minimal Dataset for chronic low back pain through conversation. In a cross-over study with 30 university students serving as proxies for patients with back pain, we compared CA-mediated data collection against standard form completion. Outcomes included completeness, correctness, and efficiency of data capture; usability and perceived usefulness (System Usability Scale, Technology Acceptance Model); conversational quality (Grice's maxims); intention to use; task completion time; and user preference. Qualitative analysis of conversations identified behavioral themes.
Results
Completeness was 100% of the items asked and correctness 98.3% ± 2.9%. Efficiency was 2.37 ± 0.52 correct data items per CA utterance. Usefulness and conversational quality were rated highly; the mean System Usability Scale score was 71.8 (SD 18.2). CA data collection took significantly longer than form completion (median 13.6 vs. 6.4 minutes; p = .001). Overall, 56% preferred the CA, 30% the form, and 15% had no preference. Qualitative analysis revealed intelligent behaviors (contextual inference, handling contradictory information, flexible coding) and limitations (overconfident coding, misuse of confidence indicators, occasional miscoding).
Conclusions
The CA collected structured clinical data accurately through conversation and was rated as usable by the student participants, though at increased completion time. Qualitative analysis identified areas for prompt refinement, particularly confidence signaling and ambiguous responses. Evaluation with patients is the next step.
Zitieren
APA
Navarro, D. F., & Peleg, M. (2026). Evaluating a Large Language Model Conversational Agent for Collecting Standardized Back Pain Data: A Cross-Over Study of Accuracy, Usability, and User Preference. Methods of Information in Medicine. https://doi.org/10.1055/a-3009-7011
Vancouver
Navarro DF, Peleg M. Evaluating a Large Language Model Conversational Agent for Collecting Standardized Back Pain Data: A Cross-Over Study of Accuracy, Usability, and User Preference. Methods of Information in Medicine. 2026. doi:10.1055/a-3009-7011
BibTeX
@article{Navarro2026,
author = {Navarro, David Fraile and Peleg, Mor},
title = {Evaluating a Large Language Model Conversational Agent for Collecting Standardized Back Pain Data: A Cross-Over Study of Accuracy, Usability, and User Preference},
journal = {Methods of Information in Medicine},
year = {2026},
doi = {10.1055/a-3009-7011}
}
EUVIMED