EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Audio-visual source separation with localization and individual control

Mohanaprasad Kothandaraman iD, Balakrishnan Ramalingam iD, Kai Sheng iD, Aman Verma, Utkarsh Dhagat iD, Pranav Parab, Siddhartha Mallavolu iD, Sankar Ganesh

PLOS One · 2025

DOI: 10.1371/journal.pone.0321856 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Mohanaprasad Kothandaraman und 7 weiteren Autor:innen. Beteiligte Einrichtung: School of Electronics Engineering (SENSE), Vellore Institute of Technology, Chennai, India. Hauptthemen laut MEDLINE: Deep Learning, Voice, Videoconferencing. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The growing reliance on video conferencing software brings significant benefits but also introduces challenges, particularly in managing audio quality. In multi-participant settings, ambient noise and interruptions can hinder speaker recognition and disrupt the flow of conversation. This work proposes an audio-visual source separation pipeline designed specifically for video conferencing and telepresence robots applications. The framework aims to isolate and enhance the speech of individual participants in noisy environments while enabling control over the volume of specific individuals captured in the video frame. The proposed pipeline comprises key components: a deep learning-based feature extractor for audio and video, an audio-guided visual attention mechanism, a module for background noise suppression and human voice separation, and Deep Multi-Resolution Network (DMRN) modules. For human voice separation, the DPRNN-TasNet, a robust deep neural network framework, is employed. Experimental results demonstrate that the methodology effectively isolates and amplifies individual participants’ speech, achieving a test accuracy of 71.88 % on both the AVE and Music 21 datasets.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Deep LearningVoiceVideoconferencingHumansNeural Networks, ComputerNoiseSpeech

Autor:innen und Institutionen

  • School of Electronics Engineering (SENSE), Vellore Institute of Technology, Chennai, India.
  • Xidian University, Xi'an, China.
  • School of Computer Science and Engineering (SCOPE), Vellore Institute of Technology, Chennai, India.

ORCID-Profile: Mohanaprasad Kothandaraman, Balakrishnan Ramalingam, Kai Sheng, Utkarsh Dhagat, Siddhartha Mallavolu

Zitieren

APA

Kothandaraman, M., Ramalingam, B., Sheng, K., Verma, A., Dhagat, U., Parab, P., Mallavolu, S., & Ganesh, S. (2025). Audio-visual source separation with localization and individual control. PLOS One. https://doi.org/10.1371/journal.pone.0321856

Vancouver

Kothandaraman M, Ramalingam B, Sheng K, Verma A, Dhagat U, Parab P, et al. Audio-visual source separation with localization and individual control. PLOS One. 2025. doi:10.1371/journal.pone.0321856

BibTeX
@article{Kothandaraman2025,
  author = {Kothandaraman, Mohanaprasad and Ramalingam, Balakrishnan and Sheng, Kai and Verma, Aman and Dhagat, Utkarsh and Parab, Pranav and Mallavolu, Siddhartha and Ganesh, Sankar},
  title = {Audio-visual source separation with localization and individual control},
  journal = {PLOS One},
  year = {2025},
  doi = {10.1371/journal.pone.0321856}
}
Kostenlos starten →