EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors

Yuya Igarashi, Kyosuke Futami iD, Kazuya Murao iD

Sensors · 19.11.2024 · 24 (22) · 7368

DOI: 10.3390/s24227368 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 19.11.2024. Verfasst von Yuya Igarashi und 2 weiteren Autor:innen. Beteiligte Einrichtung: Graduate School of Information Science and Engineering, Ritsumeikan University, 2-150 Iwakuracho, Ibarakishi 567-8570, Japan. Hauptthemen laut MEDLINE: Wearable Electronic Devices, Speech. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: KDDI Foundation, Japan Society for the Promotion of Science, KDDI Foundation 2023 Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As eyewear devices such as smart glasses become more common, it is important to provide input methods that can be used at all times for such situations and people. Silent speech interaction (SSI) has the potential to be useful as a hands-free input method for various situations and people, including those who have difficulty with voiced speech. However, previous methods have involved sensor devices that are difficult to use anytime and anywhere. We propose a method for SSI that involves using an eyewear device equipped with infrared distance sensors. The proposed method measures facial skin movements associated with speech from the infrared distance sensor mounted on an eyewear device and recognizes silent speech commands by applying machine learning to time series sensor data. The proposed method was applied to a prototype system including a sensor device consisting of eyewear and ear-mounted microphones to measure the movements of the cheek, jaw joint, and jaw. Evaluations 1 and 2 showed that five speech commands could be recognized with an F value of 0.90 and ten longer speech commands with an F value of 0.83. Evaluation 3 showed how the recognition accuracy changes with the combination of sensor points. Evaluation 4 examined whether the proposed method can be used for a larger number of speech commands with 21 commands by using deep learning LSTM and a combination of DTW and kNN. Evaluation 5 examined the recognition accuracy in some situations affecting recognition accuracy such as re-attaching devices and walking. These results show the feasibility of the proposed method for a simple hands-free input interface, such as with media players and voice assistants. Our study provides the first wearable sensing method that can easily apply SSI functions to eyewear devices.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Wearable Electronic DevicesSpeechHumansEarInfrared RaysSmart GlassesSpeech Recognition SoftwareMachine Learning

Stichwörter der Autor:innen

eyewearhands-free inputinfrared distance sensorsilent speech interactionskin motion sensing

Autor:innen und Institutionen

  • Graduate School of Information Science and Engineering, Ritsumeikan University, 2-150 Iwakuracho, Ibarakishi 567-8570, Japan.
  • Digital Spirit Teck, Kusatsu 525-8577, Japan.

ORCID-Profile: Kyosuke Futami, Kazuya Murao

Förderung

  • KDDI Foundation
  • Japan Society for the Promotion of Science
  • KDDI Foundation 2023 Research
  • JSPS (Japan Society for the Promotion of Science) KAKENHI

Zitieren

APA

Igarashi, Y., Futami, K., & Murao, K. (2024). Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors. Sensors, 24(22), 7368. https://doi.org/10.3390/s24227368

Vancouver

Igarashi Y, Futami K, Murao K. Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors. Sensors. 2024;24(22):7368. doi:10.3390/s24227368

BibTeX
@article{Igarashi2024,
  author = {Igarashi, Yuya and Futami, Kyosuke and Murao, Kazuya},
  title = {Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {22},
  pages = {7368},
  doi = {10.3390/s24227368}
}
Kostenlos starten →