Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors
Sensors · 19.11.2024 · 24 (22) · 7368
DOI: 10.3390/s24227368 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 19.11.2024. Verfasst von Yuya Igarashi und 2 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Information Science and Engineering, Ritsumeikan University, 2-150 Iwakuracho, Ibarakishi 567-8570, Japan. Hauptthemen laut MEDLINE: Wearable Electronic Devices, Speech. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: KDDI Foundation, Japan Society for the Promotion of Science, KDDI Foundation 2023 Research.
Abstract (Originalsprache)
As eyewear devices such as smart glasses become more common, it is important to provide input methods that can be used at all times for such situations and people. Silent speech interaction (SSI) has the potential to be useful as a hands-free input method for various situations and people, including those who have difficulty with voiced speech. However, previous methods have involved sensor devices that are difficult to use anytime and anywhere. We propose a method for SSI that involves using an eyewear device equipped with infrared distance sensors. The proposed method measures facial skin movements associated with speech from the infrared distance sensor mounted on an eyewear device and recognizes silent speech commands by applying machine learning to time series sensor data. The proposed method was applied to a prototype system including a sensor device consisting of eyewear and ear-mounted microphones to measure the movements of the cheek, jaw joint, and jaw. Evaluations 1 and 2 showed that five speech commands could be recognized with an F value of 0.90 and ten longer speech commands with an F value of 0.83. Evaluation 3 showed how the recognition accuracy changes with the combination of sensor points. Evaluation 4 examined whether the proposed method can be used for a larger number of speech commands with 21 commands by using deep learning LSTM and a combination of DTW and kNN. Evaluation 5 examined the recognition accuracy in some situations affecting recognition accuracy such as re-attaching devices and walking. These results show the feasibility of the proposed method for a simple hands-free input interface, such as with media players and voice assistants. Our study provides the first wearable sensing method that can easily apply SSI functions to eyewear devices.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Graduate School of Information Science and Engineering, Ritsumeikan University, 2-150 Iwakuracho, Ibarakishi 567-8570, Japan.
- Digital Spirit Teck, Kusatsu 525-8577, Japan.
ORCID-Profile: Kyosuke Futami, Kazuya Murao
Förderung
- KDDI Foundation
- Japan Society for the Promotion of Science
- KDDI Foundation 2023 Research
- JSPS (Japan Society for the Promotion of Science) KAKENHI
Zitieren
APA
Igarashi, Y., Futami, K., & Murao, K. (2024). Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors. Sensors, 24(22), 7368. https://doi.org/10.3390/s24227368
Vancouver
Igarashi Y, Futami K, Murao K. Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors. Sensors. 2024;24(22):7368. doi:10.3390/s24227368
BibTeX
@article{Igarashi2024,
author = {Igarashi, Yuya and Futami, Kyosuke and Murao, Kazuya},
title = {Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors},
journal = {Sensors},
year = {2024},
volume = {24},
number = {22},
pages = {7368},
doi = {10.3390/s24227368}
}
EUVIMED