Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition
Sensors · 2024
DOI: 10.3390/s24082573 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Geon Woo Lee und 1 weiteren Autor:innen. Beteiligte Einrichtung: AI Graduate School, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea. Hauptthemen laut MEDLINE: Speech Recognition Software, Noise. Die Arbeit nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Institute of Information & communications Technology Planning & evaluation (IITP), Institute of Information & communications Technology Planning & evaluation (IITP) grant, funded by the Korean government (MSIT).
Abstract (Originalsprache)
This paper addresses a joint training approach applied to a pipeline comprising speech enhancement (SE) and automatic speech recognition (ASR) models, where an acoustic tokenizer is included in the pipeline to leverage the linguistic information from the ASR model to the SE model. The acoustic tokenizer takes the outputs of the ASR encoder and provides a pseudo-label through K-means clustering. To transfer the linguistic information, represented by pseudo-labels, from the acoustic tokenizer to the SE model, a cluster-based pairwise contrastive (CBPC) loss function is proposed, which is a self-supervised contrastive loss function, and combined with an information noise contrastive estimation (infoNCE) loss function. This combined loss function prevents the SE model from overfitting to outlier samples and represents the pronunciation variability in samples with the same pseudo-label. The effectiveness of the proposed CBPC loss function is evaluated on a noisy LibriSpeech dataset by measuring both the speech quality scores and the word error rate (WER). The experimental results reveal that the proposed joint training approach using the described CBPC loss function achieves a lower WER than the conventional joint training approaches. In addition, it is demonstrated that the speech quality scores of the SE model trained using the proposed training approach are higher than those of the standalone-SE model and SE models trained using conventional joint training approaches. An ablation study is also conducted to investigate the effects of different combinations of loss functions on the speech quality scores and WER. Here, it is revealed that the proposed CBPC loss function combined with infoNCE contributes to a reduced WER and an increase in most of the speech quality scores.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- AI Graduate School, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea.
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju 61005, Republic of Korea.
- AunionAI Co., Ltd., Gwangju 61005, Republic of Korea.
ORCID-Profile: Geon Woo Lee, Hong Kook Kim
Förderung
- Institute of Information & communications Technology Planning & evaluation (IITP)
- Institute of Information & communications Technology Planning & evaluation (IITP) grant, funded by the Korean government (MSIT)
Zitieren
APA
Lee, G. W., & Kim, H. K. (2024). Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition. Sensors. https://doi.org/10.3390/s24082573
Vancouver
Lee GW, Kim HK. Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition. Sensors. 2024. doi:10.3390/s24082573
BibTeX
@article{Lee2024,
author = {Lee, Geon Woo and Kim, Hong Kook},
title = {Cluster-Based Pairwise Contrastive Loss for Noise-Robust Speech Recognition},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24082573}
}
EUVIMED