EUVIMEDEUVIMED entdecken
Crossref

Do Audio-Visual Segmentation Models Truly Segment Sounding Objects?

Jia Li, Wenjie Zhao, Ziru Huang, Yunhui Guo, Yapeng Tian

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (8) · 6172-6180

DOI: 10.1609/aaai.v40i8.37542 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jia Li und 4 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Unlike traditional visual segmentation, audio-visual segmentation (AVS) requires the model not only to identify and segment objects but also to determine whether they are sound sources. Recent AVS approaches have achieved impressive performance on standard benchmarks. Yet, an important question remains: Do these models genuinely integrate audio-visual cues to segment sounding objects? Our study reveals a fundamental bias in current methods: they tend to generate segmentation masks based predominantly on visual salience, irrespective of the audio context, resulting in unreliable predictions when sounds are absent or irrelevant. To address this challenge, we introduce AVSBench-Robust, a comprehensive benchmark incorporating diverse negative audio scenarios, including silence, noise, and off-screen sounds. We also propose a simple yet effective approach combining balanced training with negative samples and classifier-guided similarity learning. Our extensive experiments show that while state-of-the-art AVS methods consistently fail under negative audio conditions, our approach achieves remarkable improvements in both standard metrics and robustness measures, maintaining near-perfect false positive rates while preserving high-quality segmentation performance.

Zitieren

APA

Li, J., Zhao, W., Huang, Z., Guo, Y., & Tian, Y. (2026). Do Audio-Visual Segmentation Models Truly Segment Sounding Objects?. Proceedings of the AAAI Conference on Artificial Intelligence, 40(8), 6172-6180. https://doi.org/10.1609/aaai.v40i8.37542

Vancouver

Li J, Zhao W, Huang Z, Guo Y, Tian Y. Do Audio-Visual Segmentation Models Truly Segment Sounding Objects?. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(8):6172-6180. doi:10.1609/aaai.v40i8.37542

BibTeX
@article{Li2026,
  author = {Li, Jia and Zhao, Wenjie and Huang, Ziru and Guo, Yunhui and Tian, Yapeng},
  title = {Do Audio-Visual Segmentation Models Truly Segment Sounding Objects?},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {8},
  pages = {6172--6180},
  doi = {10.1609/aaai.v40i8.37542}
}
Kostenlos starten →