Screen-Space Vortex-Aware Sound Synthesis for Ocean Waves via Particle-Based and Learning-Based Mapping
IEEE Access · 2026 · 14 · 49885-49907
DOI: 10.1109/access.2026.3678874 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Jong-Hyun Kim. Die Arbeit nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation of Korea (NRF) grant funded by Korean Government.
Abstract (Originalsprache)
This paper presents an audio–visual mapping framework for synthesizing ocean wave sounds that are temporally aligned with scene dynamics, derived from the motion of foam particles in particle-based fluid simulations. To efficiently handle large-scale particle data, the proposed method employs dynamic K-means clustering that adapts to variations in particle count, and performs example-based sound matching and volume control using cluster-level velocity- and density-based features. To overcome the limitations of scale-oriented mappings that fail to capture viewpoint changes and structural differences in flow patterns, we introduce a screen-space vortex representation. Specifically, the physical quantities of visible foam particles are projected onto a screen-space grid from the camera viewpoint, and a vortex (2D vorticity) field is computed from the projected 2D velocity field. This vortex feature is combined with velocity–density descriptors to improve the discriminability and temporal stability of data-driven sound matching. Furthermore, to preserve the interpretability of rule-based mappings while compensating for nonlinear and viewpoint-dependent relationships, we introduce a learning-based embedding as a solver extension that maps cluster- and vortex-level features into a latent space. Experimental results across various ocean wave scenarios (wave initiation, reflection, moving objects, spinning emitters, and propellers) demonstrate that the proposed approach produces differentiated auditory responses depending on the viewpoint, even under identical simulations, and significantly improves the stability of sound synthesis and audio–visual coherence compared to fixed clustering and scale-based mappings.
Förderung
- National Research Foundation of Korea (NRF) grant funded by Korean Government
Zitieren
APA
Kim, J. (2026). Screen-Space Vortex-Aware Sound Synthesis for Ocean Waves via Particle-Based and Learning-Based Mapping. IEEE Access, 14, 49885-49907. https://doi.org/10.1109/access.2026.3678874
Vancouver
Kim J. Screen-Space Vortex-Aware Sound Synthesis for Ocean Waves via Particle-Based and Learning-Based Mapping. IEEE Access. 2026;14:49885-49907. doi:10.1109/access.2026.3678874
BibTeX
@article{Kim2026,
author = {Kim, Jong-Hyun},
title = {Screen-Space Vortex-Aware Sound Synthesis for Ocean Waves via Particle-Based and Learning-Based Mapping},
journal = {IEEE Access},
year = {2026},
volume = {14},
pages = {49885--49907},
doi = {10.1109/access.2026.3678874}
}
EUVIMED