EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Wearable Visually Impaired Assistive System Based on Semantic Vision SLAM for Grasping Operation

Fei Fei, Sifan Xian, Ruonan Yang, Changcheng Wu iD, Xiong Lu iD

Sensors · 02.06.2024 · 24 (11) · 3593

DOI: 10.3390/s24113593 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 02.06.2024. Verfasst von Fei Fei und 4 weiteren Autor:innen. Beteiligte Einrichtung: College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, China. Hauptthemen laut MEDLINE: Wearable Electronic Devices, Persons with Visual Disabilities, Algorithms. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: China postdoctoral science foundation, Natural Science Foundation of the Jiangsu Higher Education Institutions of China, Graduate Research and Practical Innovation Program at Nanjing University of Aeronautics and Astronautics.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Because of the absence of visual perception, visually impaired individuals encounter various difficulties in their daily lives. This paper proposes a visual aid system designed specifically for visually impaired individuals, aiming to assist and guide them in grasping target objects within a tabletop environment. The system employs a visual perception module that incorporates a semantic visual SLAM algorithm, achieved through the fusion of ORB-SLAM2 and YOLO V5s, enabling the construction of a semantic map of the environment. In the human–machine cooperation module, a depth camera is integrated into a wearable device worn on the hand, while a vibration array feedback device conveys directional information of the target to visually impaired individuals for tactile interaction. To enhance the system’s versatility, a Dobot Magician manipulator is also employed to aid visually impaired individuals in grasping tasks. The performance of the semantic visual SLAM algorithm in terms of localization and semantic mapping was thoroughly tested. Additionally, several experiments were conducted to simulate visually impaired individuals’ interactions in grasping target objects, effectively verifying the feasibility and effectiveness of the proposed system. Overall, this system demonstrates its capability to assist and guide visually impaired individuals in perceiving and acquiring target objects.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Wearable Electronic DevicesPersons with Visual DisabilitiesAlgorithmsHumansHand StrengthSelf-Help DevicesVisual PerceptionSemanticsMale

Stichwörter der Autor:innen

human–computer interactionsemantic mappingvisual SLAMvisually impaired assistance

Autor:innen und Institutionen

  • College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211100, China.

ORCID-Profile: Changcheng Wu, Xiong Lu

Förderung

  • China postdoctoral science foundation
  • Natural Science Foundation of the Jiangsu Higher Education Institutions of China
  • Graduate Research and Practical Innovation Program at Nanjing University of Aeronautics and Astronautics
  • Experimental Technology Research and Development Project at Nanjing University of Aeronautics and Astronautics
  • the Natural Science Foundation of Jiangsu Province
  • the Fundamental Research Funds for the Central Universities (Nanjing University of Aeronautics and Astronautics)
  • Natural Science Foundation of Jiangsu Province

Zitieren

APA

Fei, F., Xian, S., Yang, R., Wu, C., & Lu, X. (2024). A Wearable Visually Impaired Assistive System Based on Semantic Vision SLAM for Grasping Operation. Sensors, 24(11), 3593. https://doi.org/10.3390/s24113593

Vancouver

Fei F, Xian S, Yang R, Wu C, Lu X. A Wearable Visually Impaired Assistive System Based on Semantic Vision SLAM for Grasping Operation. Sensors. 2024;24(11):3593. doi:10.3390/s24113593

BibTeX
@article{Fei2024,
  author = {Fei, Fei and Xian, Sifan and Yang, Ruonan and Wu, Changcheng and Lu, Xiong},
  title = {A Wearable Visually Impaired Assistive System Based on Semantic Vision SLAM for Grasping Operation},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {11},
  pages = {3593},
  doi = {10.3390/s24113593}
}
Kostenlos starten →