Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data Entry
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · 13.05.2024 · 8 (2) · 1-39
DOI: 10.1145/3659624 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, online erschienen am 15.05.2024. Verfasst von Andrea Cuadra und 8 weiteren Autor:innen. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 102 Literaturangaben. Gefördert von: Gordon and Betty Moore Foundation, Memorial Sloan-Kettering Cancer Center.
Abstract (Originalsprache)
Digital forms help us access services and opportunities, but they are not equally accessible to everyone, such as older adults or those with sensory impairments. Large language models (LLMs) and multimodal interfaces offer a unique opportunity to increase form accessibility. Informed by prior literature and needfinding, we built a holistic multimodal LLM agent for health data entry. We describe the process of designing and building our system, and the results of a study with older adults (N =10). All participants, regardless of age or disability status, were able to complete a standard 47-question form independently using our system---one blind participant said it was "a prayer answered." Our video analysis revealed how different modalities provided alternative interaction paths in complementary ways (e.g., the buttons helped resolve transcription errors and speech helped provide more options when the pre-canned answer choices were insufficient). We highlight key design guidelines, such as designing systems that dynamically adapt to individual needs.
Förderung
- Gordon and Betty Moore Foundation
- Memorial Sloan-Kettering Cancer Center
Zitieren
APA
Cuadra, A., Breuch, J., Estrada, S., Ihim, D., Hung, I., Askaryar, D., Hassanien, M., Fessele, K. L., & Landay, J. A. (2024). Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data Entry. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 8(2), 1-39. https://doi.org/10.1145/3659624
Vancouver
Cuadra A, Breuch J, Estrada S, Ihim D, Hung I, Askaryar D, et al. Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data Entry. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. 2024;8(2):1-39. doi:10.1145/3659624
BibTeX
@article{Cuadra2024,
author = {Cuadra, Andrea and Breuch, Justine and Estrada, Samantha and Ihim, David and Hung, Isabelle and Askaryar, Derek and Hassanien, Marwan and Fessele, Kristen L. and Landay, James A.},
title = {Digital Forms for All: A Holistic Multimodal Large Language Model Agent for Health Data Entry},
journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies},
year = {2024},
volume = {8},
number = {2},
pages = {1--39},
doi = {10.1145/3659624}
}
EUVIMED