Multi-Modal System for Walking Safety for the Visually Impaired: Multi-Object Detection and Natural Language Generation
Applied Sciences · 29.08.2024 · 14 (17) · 7643
DOI: 10.3390/app14177643 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 29.08.2024. Verfasst von Jekyung Lee und 2 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Education of the Republic of Korea.
Abstract (Originalsprache)
This study introduces a system for visually impaired individuals in a walking environment. It combines object recognition using YOLOv5 and cautionary sentence generation with KoAlpaca. The system employs image data augmentation for diverse training data and GPT for natural language training. Furthermore, the implementation of the system on a single board was followed by a comprehensive comparative analysis with existing studies. Moreover, a pilot test involving visually impaired and healthy individuals was conducted to validate the system’s practical applicability and adaptability in real-world walking environments. Our pilot test results indicated an average usability score of 4.05. Participants expressed some dissatisfaction with the notification conveying time and online implementation, but they highly praised the system’s object detection range and accuracy. The experiments demonstrated that using QLoRA enables more efficient training of larger models, which is associated with improved model performance. Our study makes a significant contribution to the literature because the proposed system enables real-time monitoring of various environmental conditions and objects in pedestrian environments using AI.
Förderung
- Ministry of Education of the Republic of Korea
Zitieren
APA
Lee, J., Cha, K., & Lee, M. (2024). Multi-Modal System for Walking Safety for the Visually Impaired: Multi-Object Detection and Natural Language Generation. Applied Sciences, 14(17), 7643. https://doi.org/10.3390/app14177643
Vancouver
Lee J, Cha K, Lee M. Multi-Modal System for Walking Safety for the Visually Impaired: Multi-Object Detection and Natural Language Generation. Applied Sciences. 2024;14(17):7643. doi:10.3390/app14177643
BibTeX
@article{Lee2024,
author = {Lee, Jekyung and Cha, Kyung-Ae and Lee, Miran},
title = {Multi-Modal System for Walking Safety for the Visually Impaired: Multi-Object Detection and Natural Language Generation},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {17},
pages = {7643},
doi = {10.3390/app14177643}
}
EUVIMED