Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech
Sensors · 13.07.2023 · 23 (14) · 6383
DOI: 10.3390/s23146383 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 13.07.2023. Verfasst von Nurzada Amangeldy und 5 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Information Technologies, L.N. Gumilyov Eurasian National University, Astana 010000, Kazakhstan. Hauptthemen laut MEDLINE: Speech, Speech Perception. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Science Committee of the Ministry of Education and Science of the Republic of Kazakhstan.
Abstract (Originalsprache)
This article is devoted to solving the problem of converting sign language into a consistent text with intonation markup for subsequent voice synthesis of sign phrases by speech with intonation. The paper proposes an improved method of continuous recognition of sign language, the results of which are transmitted to a natural language processor based on analyzers of morphology, syntax, and semantics of the Kazakh language, including morphological inflection and the construction of an intonation model of simple sentences. This approach has significant practical and social significance, as it can lead to the development of technologies that will help people with disabilities to communicate and improve their quality of life. As a result of the cross-validation of the model, we obtained an average test accuracy of 0.97 and an average val_accuracy of 0.90 for model evaluation. We also identified 20 sentence structures of the Kazakh language with their intonational model.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Faculty of Information Technologies, L.N. Gumilyov Eurasian National University, Astana 010000, Kazakhstan.
- Department of Computer Science, Lublin University of Technology, 36B Nadbystrzycka Str., 20-618 Lublin, Poland.
ORCID-Profile: Nurzada Amangeldy, Aru Ukenova, Marek Milosz, Saule Kudubayeva
Förderung
- Science Committee of the Ministry of Education and Science of the Republic of Kazakhstan
Zitieren
APA
Amangeldy, N., Ukenova, A., Bekmanova, G., Razakhova, B., Milosz, M., & Kudubayeva, S. (2023). Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech. Sensors, 23(14), 6383. https://doi.org/10.3390/s23146383
Vancouver
Amangeldy N, Ukenova A, Bekmanova G, Razakhova B, Milosz M, Kudubayeva S. Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech. Sensors. 2023;23(14):6383. doi:10.3390/s23146383
BibTeX
@article{Amangeldy2023,
author = {Amangeldy, Nurzada and Ukenova, Aru and Bekmanova, Gulmira and Razakhova, Bibigul and Milosz, Marek and Kudubayeva, Saule},
title = {Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech},
journal = {Sensors},
year = {2023},
volume = {23},
number = {14},
pages = {6383},
doi = {10.3390/s23146383}
}
EUVIMED