EUVIMEDEUVIMED entdecken
Crossref

Deep Learning-Based Audio-Visual Speech Enhancement with Adaptive Noise Suppression and Perceptual Loss Optimization

Ojas M. Suroo,

Advances in Nonlinear Variational Inequalities · 28.11.2024 · 28 (2s) · 688-692

DOI: 10.52783/anvi.v28.7160 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Advances in Nonlinear Variational Inequalities, online erschienen am 28.11.2024. Verfasst von Ojas M. Suroo,.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In noisy environments, traditional audio-only automatic speech recognition (ASR) systems suffer significant performance degradation. Leveraging visual cues from lip movements can enhance robustness, as demonstrated in early works on audio-visual feature fusion. This paper proposes a novel deep learning-based framework, AV-DenoiseNet, which integrates convolutional neural networks (CNNs) for visual feature extraction, recurrent neural networks (RNNs) for audio processing, and a transformer-based fusion module for multi-modal speech enhancement. Building on foundational methods like linear filtering for noisy audio enhancement and adaptive noise suppression, we incorporate recent trends such as attention mechanisms and end-to-end training. Experiments on the AVSpeech and GRID datasets show up to 35% relative reduction in word error rate (WER) at low signal-to-noise ratios (SNRs) compared to baselines. Simulation results, including spectrograms and performance curves, validate the model's efficacy for real-time applications.

Zitieren

APA

Suroo,, O. M. (2024). Deep Learning-Based Audio-Visual Speech Enhancement with Adaptive Noise Suppression and Perceptual Loss Optimization. Advances in Nonlinear Variational Inequalities, 28(2s), 688-692. https://doi.org/10.52783/anvi.v28.7160

Vancouver

Suroo, OM. Deep Learning-Based Audio-Visual Speech Enhancement with Adaptive Noise Suppression and Perceptual Loss Optimization. Advances in Nonlinear Variational Inequalities. 2024;28(2s):688-692. doi:10.52783/anvi.v28.7160

BibTeX
@article{Suroo2024,
  author = {Suroo,, Ojas M.},
  title = {Deep Learning-Based Audio-Visual Speech Enhancement with Adaptive Noise Suppression and Perceptual Loss Optimization},
  journal = {Advances in Nonlinear Variational Inequalities},
  year = {2024},
  volume = {28},
  number = {2s},
  pages = {688--692},
  doi = {10.52783/anvi.v28.7160}
}
Kostenlos starten →