EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A perceptual similarity space for speech based on self-supervised speech representations

Bronya R Chernyak, Ann R Bradlow iD, Joseph Keshet, Matthew Goldrick

The Journal of the Acoustical Society of America · 01.06.2024 · 155 (6) · 3915-3929

DOI: 10.1121/10.0026358 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Journal of the Acoustical Society of America, online erschienen am 21.06.2024. Verfasst von Bronya R Chernyak und 3 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Electrical & Computer Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel. Hauptthemen laut MEDLINE: Speech Perception, Speech Intelligibility, Speech Acoustics. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: United States-Israel Binational Science Foundation, National Science Foundation Graduate Research Fellowship Program.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Speech recognition by both humans and machines frequently fails in non-optimal yet common situations. For example, word recognition error rates for second-language (L2) speech can be high, especially under conditions involving background noise. At the same time, both human and machine speech recognition sometimes shows remarkable robustness against signal- and noise-related degradation. Which acoustic features of speech explain this substantial variation in intelligibility? Current approaches align speech to text to extract a small set of pre-defined spectro-temporal properties from specific sounds in particular words. However, variation in these properties leaves much cross-talker variation in intelligibility unexplained. We examine an alternative approach utilizing a perceptual similarity space acquired using self-supervised learning. This approach encodes distinctions between speech samples without requiring pre-defined acoustic features or speech-to-text alignment. We show that L2 English speech samples are less tightly clustered in the space than L1 samples reflecting variability in English proficiency among L2 talkers. Critically, distances in this similarity space are perceptually meaningful: L1 English listeners have lower recognition accuracy for L2 speakers whose speech is more distant in the space from L1 speech. These results indicate that perceptual similarity may form the basis for an entirely new speech and language analysis approach.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Speech PerceptionSpeech IntelligibilitySpeech AcousticsHumansMaleFemaleAdultYoung AdultMultilingualismRecognition, PsychologyNoise

Autor:innen und Institutionen

  • Faculty of Electrical & Computer Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel.
  • Department of Linguistics, Northwestern University, Evanston, Illinois 60208, USA.

ORCID-Profile: Ann R Bradlow

Förderung

  • United States-Israel Binational Science Foundation
  • National Science Foundation Graduate Research Fellowship Program

Zitieren

APA

Chernyak, B. R., Bradlow, A. R., Keshet, J., & Goldrick, M. (2024). A perceptual similarity space for speech based on self-supervised speech representations. The Journal of the Acoustical Society of America, 155(6), 3915-3929. https://doi.org/10.1121/10.0026358

Vancouver

Chernyak BR, Bradlow AR, Keshet J, Goldrick M. A perceptual similarity space for speech based on self-supervised speech representations. The Journal of the Acoustical Society of America. 2024;155(6):3915-3929. doi:10.1121/10.0026358

BibTeX
@article{Chernyak2024,
  author = {Chernyak, Bronya R and Bradlow, Ann R and Keshet, Joseph and Goldrick, Matthew},
  title = {A perceptual similarity space for speech based on self-supervised speech representations},
  journal = {The Journal of the Acoustical Society of America},
  year = {2024},
  volume = {155},
  number = {6},
  pages = {3915--3929},
  doi = {10.1121/10.0026358}
}
Kostenlos starten →