EUVIMEDEUVIMED entdecken
Crossref

SoundBrush: Sound as a Brush for Visual Scene Editing

Kim Sung-Bin, Kim Jun-Seong, Junseok Ko, Yewon Kim, Tae-Hyun Oh

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (7) · 7167-7175

DOI: 10.1609/aaai.v39i7.32770 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Kim Sung-Bin und 4 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We propose SoundBrush, a model that uses sound as a brush to edit and manipulate visual scenes. We extend the generative capabilities of the Latent Diffusion Model (LDM) to incorporate audio information for editing visual scenes. Inspired by existing image-editing works, we frame this task as a supervised learning problem and leverage various off-the-shelf models to construct a sound-paired visual scene editing dataset for training. This richly generated dataset enables SoundBrush to learn to map audio features into the textual space of the LDM, allowing for visual scene editing guided by diverse in-the-wild sound. Unlike existing methods, SoundBrush can accurately manipulate the overall scenery or even insert sounding objects to best match the input sound semantics while preserving the original content. Furthermore, by integrating with novel view synthesis techniques, our framework can be extended to edit 3D scenes, facilitating sound-driven 3D scene manipulation.

Zitieren

APA

Sung-Bin, K., Jun-Seong, K., Ko, J., Kim, Y., & Oh, T. (2025). SoundBrush: Sound as a Brush for Visual Scene Editing. Proceedings of the AAAI Conference on Artificial Intelligence, 39(7), 7167-7175. https://doi.org/10.1609/aaai.v39i7.32770

Vancouver

Sung-Bin K, Jun-Seong K, Ko J, Kim Y, Oh T. SoundBrush: Sound as a Brush for Visual Scene Editing. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(7):7167-7175. doi:10.1609/aaai.v39i7.32770

BibTeX
@article{SungBin2025,
  author = {Sung-Bin, Kim and Jun-Seong, Kim and Ko, Junseok and Kim, Yewon and Oh, Tae-Hyun},
  title = {SoundBrush: Sound as a Brush for Visual Scene Editing},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {7},
  pages = {7167--7175},
  doi = {10.1609/aaai.v39i7.32770}
}
Kostenlos starten →