Real-time sign language recognition based on YOLO algorithm
Neural Computing and Applications · 2024-05 · 36 (14) · 7609-7624
DOI: 10.1007/s00521-024-09503-6 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neural Computing and Applications, online erschienen am 15.02.2024. Verfasst von Melek Alaftekin und 2 weiteren Autor:innen. Die Arbeit wurde bisher 67-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Igdır University.
Abstract (Originalsprache)
This study focuses on real-time hand gesture recognition in the Turkish sign language detection system. YOLOv4-CSP based on convolutional neural network (CNN), a state-of-the-art object detection algorithm, is used to provide real-time and high-performance detection. The YOLOv4-CSP algorithm is created by adding CSPNet to the neck of the original YOLOv4 to improve network performance. A new object detection model has been proposed by optimizing the YOLOv4-CSP algorithm in order to provide more efficient detection in Turkish sign language. The model uses CSPNet throughout the network to increase the learning ability of the network. However, Proposed YOLOv4-CSP has a learning model with Mish activation function, complete intersection of union (CIoU) loss function and transformer block added. The Proposed YOLOv4-CSP algorithm has faster learning with transfer learning than previous versions. This allows the proposed YOLOv4-CSP algorithm to perform a faster restriction and recognition of static hand signals simultaneously. To evaluate the speed and detection performance of the proposed YOLOv4-CSP model, it is compared with previous YOLO series, which offers real-time detection, as well. YOLOv3, YOLOv3-SPP, YOLOv4-CSP and proposed YOLOv4-CSP models are trained with a labeled dataset consisting of numbers in Turkish Sign language, and their performances on the hand signals recognitions are compared. With the proposed method, 98.95% precision, 98.15% recall, 98.55 F1 score and 99.49% mAP results are obtained in 9.8 ms. The proposed method for detecting numbers in Turkish sign language outperforms other algorithms with both real-time performance and accurate hand sign prediction, regardless of background.
Förderung
- Igdır University
Zitieren
APA
Alaftekin, M., Pacal, I., & Cicek, K. (2024). Real-time sign language recognition based on YOLO algorithm. Neural Computing and Applications, 36(14), 7609-7624. https://doi.org/10.1007/s00521-024-09503-6
Vancouver
Alaftekin M, Pacal I, Cicek K. Real-time sign language recognition based on YOLO algorithm. Neural Computing and Applications. 2024;36(14):7609-7624. doi:10.1007/s00521-024-09503-6
BibTeX
@article{Alaftekin2024,
author = {Alaftekin, Melek and Pacal, Ishak and Cicek, Kenan},
title = {Real-time sign language recognition based on YOLO algorithm},
journal = {Neural Computing and Applications},
year = {2024},
volume = {36},
number = {14},
pages = {7609--7624},
doi = {10.1007/s00521-024-09503-6}
}
EUVIMED