EUVIMEDOpdag EUVIMED
Crossref

ChitChatGuide: Conversational Interaction Using Large Language Models for Assisting People with Visual Impairments to Explore a Shopping Mall

Yuka Kaniwa, Masaki Kuribayashi, Seita Kayukawa, Daisuke Sato, Hironobu Takagi, Chieko Asakawa, Shigeo Morishima

Proceedings of the ACM on Human-Computer Interaction · 2024-09-24 · 8 (MHCI) · 1-25

DOI: 10.1145/3676492 ↗

At a glance

Journal article in Proceedings of the ACM on Human-Computer Interaction, published online on 2024-09-24. Written by Yuka Kaniwa and 6 co-authors. Cited 23 times so far (Crossref); 83 references. Funded by: JSPS KAKENHI.

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

To enable people with visual impairments (PVI) to explore shopping malls, it is important to provide information for selecting destinations and obtaining information based on the individual's interests. We achieved this through conversational interaction by integrating a large language model (LLM) with a navigation system. ChitChatGuide allows users to plan a tour through contextual conversations, receive personalized descriptions of surroundings based on transit time, and make inquiries during navigation. We conducted a study in a shopping mall with 11 PVI, and the results reveal that the system allowed them to explore the facility with increased enjoyment. The LLM-based conversational interaction, by understanding vague and context-based questions, enabled the participants to explore unfamiliar environments effectively. The personalized and in-situ information generated by the LLM was both useful and enjoyable. Considering the limitations we identified, we discuss the criteria for integrating LLMs into navigation systems to enhance the exploration experiences of PVI.

Funding

  • JSPS KAKENHI

Cite

APA

Kaniwa, Y., Kuribayashi, M., Kayukawa, S., Sato, D., Takagi, H., Asakawa, C., & Morishima, S. (2024). ChitChatGuide: Conversational Interaction Using Large Language Models for Assisting People with Visual Impairments to Explore a Shopping Mall. Proceedings of the ACM on Human-Computer Interaction, 8(MHCI), 1-25. https://doi.org/10.1145/3676492

Vancouver

Kaniwa Y, Kuribayashi M, Kayukawa S, Sato D, Takagi H, Asakawa C, et al. ChitChatGuide: Conversational Interaction Using Large Language Models for Assisting People with Visual Impairments to Explore a Shopping Mall. Proceedings of the ACM on Human-Computer Interaction. 2024;8(MHCI):1-25. doi:10.1145/3676492

BibTeX
@article{Kaniwa2024,
  author = {Kaniwa, Yuka and Kuribayashi, Masaki and Kayukawa, Seita and Sato, Daisuke and Takagi, Hironobu and Asakawa, Chieko and Morishima, Shigeo},
  title = {ChitChatGuide: Conversational Interaction Using Large Language Models for Assisting People with Visual Impairments to Explore a Shopping Mall},
  journal = {Proceedings of the ACM on Human-Computer Interaction},
  year = {2024},
  volume = {8},
  number = {MHCI},
  pages = {1--25},
  doi = {10.1145/3676492}
}
Start for free →