Analysis of a dataset for evaluating semantic relevance between text and audio
The Journal of the Acoustical Society of America · 01.10.2025 · 158 (4_Supplement) · A136-A136
DOI: 10.1121/10.0040371 ↗Kurz gefasst
Fachartikel in der Zeitschrift The Journal of the Acoustical Society of America, erschienen am 01.10.2025. Verfasst von Yusuke Kanamori und 4 weiteren Autor:innen.
Abstract (Originalsprache)
In text-to-audio generation (TTA), the relevance between input text and generated audio is an important metric. Prior work created the RELATE dataset, an open-source dataset for the subjective evaluation of relevance, by collecting 11-point scores across three perspectives: REL (overall relevance), IS (inclusion of sound event), and OS (order of sound event). In conventional work, only the REL score was analyzed based on text attributes such as complexity and audio attributes like audio categories (e.g., Animal, Music). This study expands upon that research by comprehensively analyzing all three scores (REL, IS, and OS). Text and audio attribute analyses, consistent with those previously applied to the REL score, were conducted for the IS and OS scores. Additionally, pairwise correlation coefficients among all three scores were computed. Analysis of text and audio attributes indicated that nearly identical trends for REL and IS scores. A moderate correlation of 0.56 was observed between IS and REL scores. On the other hand, IS and OS scores demonstrated a high correlation of 0.78. Furthermore, REL and OS scores showed a moderate correlation of 0.52. These results suggest that their independent assessment is necessary to ensure an appropriate evaluation of relevance.
Zitieren
APA
Kanamori, Y., Okamoto, Y., Takamichi, S., Saito, Y., & Saruwatari, H. (2025). Analysis of a dataset for evaluating semantic relevance between text and audio. The Journal of the Acoustical Society of America, 158(4_Supplement), A136-A136. https://doi.org/10.1121/10.0040371
Vancouver
Kanamori Y, Okamoto Y, Takamichi S, Saito Y, Saruwatari H. Analysis of a dataset for evaluating semantic relevance between text and audio. The Journal of the Acoustical Society of America. 2025;158(4_Supplement):A136-A136. doi:10.1121/10.0040371
BibTeX
@article{Kanamori2025,
author = {Kanamori, Yusuke and Okamoto, Yuki and Takamichi, Shinnosuke and Saito, Yuki and Saruwatari, Hiroshi},
title = {Analysis of a dataset for evaluating semantic relevance between text and audio},
journal = {The Journal of the Acoustical Society of America},
year = {2025},
volume = {158},
number = {4_Supplement},
pages = {A136--A136},
doi = {10.1121/10.0040371}
}
EUVIMED