EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech

Nurzada Amangeldy iD, Aru Ukenova iD, Gulmira Bekmanova, Bibigul Razakhova, Marek Milosz iD, Saule Kudubayeva iD

Sensors · 13.07.2023 · 23 (14) · 6383

DOI: 10.3390/s23146383 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 13.07.2023. Verfasst von Nurzada Amangeldy und 5 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Information Technologies, L.N. Gumilyov Eurasian National University, Astana 010000, Kazakhstan. Hauptthemen laut MEDLINE: Speech, Speech Perception. Die Arbeit wurde bisher 32-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Science Committee of the Ministry of Education and Science of the Republic of Kazakhstan.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This article is devoted to solving the problem of converting sign language into a consistent text with intonation markup for subsequent voice synthesis of sign phrases by speech with intonation. The paper proposes an improved method of continuous recognition of sign language, the results of which are transmitted to a natural language processor based on analyzers of morphology, syntax, and semantics of the Kazakh language, including morphological inflection and the construction of an intonation model of simple sentences. This approach has significant practical and social significance, as it can lead to the development of technologies that will help people with disabilities to communicate and improve their quality of life. As a result of the cross-validation of the model, we obtained an average test accuracy of 0.97 and an average val_accuracy of 0.90 for model evaluation. We also identified 20 sentence structures of the Kazakh language with their intonational model.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SpeechSpeech PerceptionHumansSign LanguageQuality of LifeLanguage

Stichwörter der Autor:innen

intonational speech synthesislong short-term memorynatural language processingsign language recognitionspatiotemporal features

Autor:innen und Institutionen

  • Faculty of Information Technologies, L.N. Gumilyov Eurasian National University, Astana 010000, Kazakhstan.
  • Department of Computer Science, Lublin University of Technology, 36B Nadbystrzycka Str., 20-618 Lublin, Poland.

ORCID-Profile: Nurzada Amangeldy, Aru Ukenova, Marek Milosz, Saule Kudubayeva

Förderung

  • Science Committee of the Ministry of Education and Science of the Republic of Kazakhstan

Zitieren

APA

Amangeldy, N., Ukenova, A., Bekmanova, G., Razakhova, B., Milosz, M., & Kudubayeva, S. (2023). Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech. Sensors, 23(14), 6383. https://doi.org/10.3390/s23146383

Vancouver

Amangeldy N, Ukenova A, Bekmanova G, Razakhova B, Milosz M, Kudubayeva S. Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech. Sensors. 2023;23(14):6383. doi:10.3390/s23146383

BibTeX
@article{Amangeldy2023,
  author = {Amangeldy, Nurzada and Ukenova, Aru and Bekmanova, Gulmira and Razakhova, Bibigul and Milosz, Marek and Kudubayeva, Saule},
  title = {Continuous Sign Language Recognition and Its Translation into Intonation-Colored Speech},
  journal = {Sensors},
  year = {2023},
  volume = {23},
  number = {14},
  pages = {6383},
  doi = {10.3390/s23146383}
}
Kostenlos starten →