General-Purpose Audio-Visual Sounding Object Localization Based on Semi-Automatic Annotation
IEEE Signal Processing Letters · 2026 · 33 · 2949-2953
DOI: 10.1109/lsp.2026.3712286 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Signal Processing Letters, 2026. Verfasst von Shun Sakurai und 3 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
This paper presents general-purpose audio-visual sounding object localization (AV-SOL) for handling a wide variety of sound events in diverse real-world scenarios. AV-SOL is a task to detect and localize sound-producing objects in a video frame. A major challenge in building a general-purpose AV-SOL model lies in the limitations of existing datasets, which fail to provide both large scale and category diversity due to expensive manual annotation. To address this problem, we propose a semi-automatic approach that substantially reduces annotation costs while maintaining category diversity. The key idea is to leverage existing large-scale audio datasets that provide sound-event annotations for video recordings without their spatial locations. We manually annotate the spatial locations of sounding objects in video frames and automatically align them with the corresponding sound-event categories. The resulting dataset contains six million annotated frames covering 436 categories, which are 83.5 and 1.9 times larger than the existing largest and most diverse datasets, respectively. Experimental results demonstrate that pre-training an AV-SOL model on our dataset consistently outperforms pre-training on the existing datasets across multiple benchmarks.
Zitieren
APA
Sakurai, S., Bando, Y., Imoto, K., & Onishi, M. (2026). General-Purpose Audio-Visual Sounding Object Localization Based on Semi-Automatic Annotation. IEEE Signal Processing Letters, 33, 2949-2953. https://doi.org/10.1109/lsp.2026.3712286
Vancouver
Sakurai S, Bando Y, Imoto K, Onishi M. General-Purpose Audio-Visual Sounding Object Localization Based on Semi-Automatic Annotation. IEEE Signal Processing Letters. 2026;33:2949-2953. doi:10.1109/lsp.2026.3712286
BibTeX
@article{Sakurai2026,
author = {Sakurai, Shun and Bando, Yoshiaki and Imoto, Keisuke and Onishi, Masaki},
title = {General-Purpose Audio-Visual Sounding Object Localization Based on Semi-Automatic Annotation},
journal = {IEEE Signal Processing Letters},
year = {2026},
volume = {33},
pages = {2949--2953},
doi = {10.1109/lsp.2026.3712286}
}
EUVIMED