EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition

Geon Woo Lee iD, Hong Kook Kim iD

Sensors · 2024

DOI: 10.3390/s24082573 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Geon Woo Lee und 1 weiteren Autor:innen. Beteiligte Einrichtung: AI Graduate School, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea. Hauptthemen laut MEDLINE: Speech Recognition Software, Noise. Die Arbeit nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Institute of Information & communications Technology Planning & evaluation (IITP), Institute of Information & communications Technology Planning & evaluation (IITP) grant, funded by the Korean government (MSIT).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper addresses a joint training approach applied to a pipeline comprising speech enhancement (SE) and automatic speech recognition (ASR) models, where an acoustic tokenizer is included in the pipeline to leverage the linguistic information from the ASR model to the SE model. The acoustic tokenizer takes the outputs of the ASR encoder and provides a pseudo-label through K-means clustering. To transfer the linguistic information, represented by pseudo-labels, from the acoustic tokenizer to the SE model, a cluster-based pairwise contrastive (CBPC) loss function is proposed, which is a self-supervised contrastive loss function, and combined with an information noise contrastive estimation (infoNCE) loss function. This combined loss function prevents the SE model from overfitting to outlier samples and represents the pronunciation variability in samples with the same pseudo-label. The effectiveness of the proposed CBPC loss function is evaluated on a noisy LibriSpeech dataset by measuring both the speech quality scores and the word error rate (WER). The experimental results reveal that the proposed joint training approach using the described CBPC loss function achieves a lower WER than the conventional joint training approaches. In addition, it is demonstrated that the speech quality scores of the SE model trained using the proposed training approach are higher than those of the standalone-SE model and SE models trained using conventional joint training approaches. An ablation study is also conducted to investigate the effects of different combinations of loss functions on the speech quality scores and WER. Here, it is revealed that the proposed CBPC loss function combined with infoNCE contributes to a reduced WER and an increase in most of the speech quality scores.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Speech Recognition SoftwareNoiseHumansCluster AnalysisAlgorithmsSpeech

Stichwörter der Autor:innen

acoustic tokenizercontrastive lossjoint trainingnoise-robust speech recognitionself-supervised learningspeech enhancement

Autor:innen und Institutionen

  • AI Graduate School, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea.
  • School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea.
  • AunionAI Co., Ltd., Gwangju 61005, Republic of Korea.

ORCID-Profile: Geon Woo Lee, Hong Kook Kim

Förderung

  • Institute of Information & communications Technology Planning & evaluation (IITP)
  • Institute of Information & communications Technology Planning & evaluation (IITP) grant, funded by the Korean government (MSIT)

Zitieren

APA

Lee, G. W., & Kim, H. K. (2024). Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition. Sensors. https://doi.org/10.3390/s24082573

Vancouver

Lee GW, Kim HK. Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition. Sensors. 2024. doi:10.3390/s24082573

BibTeX
@article{Lee2024,
  author = {Lee, Geon Woo and Kim, Hong Kook},
  title = {Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24082573}
}
Kostenlos starten →