Deep Learning-Based Audio-Visual Speech Enhancement with Adaptive Noise Suppression and Perceptual Loss Optimization
Advances in Nonlinear Variational Inequalities · 28.11.2024 · 28 (2s) · 688-692
DOI: 10.52783/anvi.v28.7160 ↗Kurz gefasst
Fachartikel in der Zeitschrift Advances in Nonlinear Variational Inequalities, online erschienen am 28.11.2024. Verfasst von Ojas M. Suroo,.
Abstract (Originalsprache)
In noisy environments, traditional audio-only automatic speech recognition (ASR) systems suffer significant performance degradation. Leveraging visual cues from lip movements can enhance robustness, as demonstrated in early works on audio-visual feature fusion. This paper proposes a novel deep learning-based framework, AV-DenoiseNet, which integrates convolutional neural networks (CNNs) for visual feature extraction, recurrent neural networks (RNNs) for audio processing, and a transformer-based fusion module for multi-modal speech enhancement. Building on foundational methods like linear filtering for noisy audio enhancement and adaptive noise suppression, we incorporate recent trends such as attention mechanisms and end-to-end training. Experiments on the AVSpeech and GRID datasets show up to 35% relative reduction in word error rate (WER) at low signal-to-noise ratios (SNRs) compared to baselines. Simulation results, including spectrograms and performance curves, validate the model's efficacy for real-time applications.
Zitieren
APA
Suroo,, O. M. (2024). Deep Learning-Based Audio-Visual Speech Enhancement with Adaptive Noise Suppression and Perceptual Loss Optimization. Advances in Nonlinear Variational Inequalities, 28(2s), 688-692. https://doi.org/10.52783/anvi.v28.7160
Vancouver
Suroo, OM. Deep Learning-Based Audio-Visual Speech Enhancement with Adaptive Noise Suppression and Perceptual Loss Optimization. Advances in Nonlinear Variational Inequalities. 2024;28(2s):688-692. doi:10.52783/anvi.v28.7160
BibTeX
@article{Suroo2024,
author = {Suroo,, Ojas M.},
title = {Deep Learning-Based Audio-Visual Speech Enhancement with Adaptive Noise Suppression and Perceptual Loss Optimization},
journal = {Advances in Nonlinear Variational Inequalities},
year = {2024},
volume = {28},
number = {2s},
pages = {688--692},
doi = {10.52783/anvi.v28.7160}
}
EUVIMED