Science of music-based citizen science: How seeing influences hearing
PLOS One · 2025
DOI: 10.1371/journal.pone.0325019 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Daniel Bedoya und 3 weiteren Autor:innen. Beteiligte Einrichtung: Laboratoire de Mécanique des Structures et des Systèmes Couplés (LMSSC), Conservatoire National des Arts et Métiers (CNAM), Paris, France. Hauptthemen laut MEDLINE: Citizen Science, Music, Hearing, Auditory Perception. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: HORIZON EUROPE European Research Council, Sorbonne Université, École Normale Supérieure.
Abstract (Originalsprache)
Citizen science engages volunteers to contribute data to scientific projects, often through visual annotation tasks. Hearing based activities are rare and less well understood. Having high quality annotations of performed music structures is essential for reliable algorithmic analysis of recorded music with applications ranging from music information retrieval to music therapy. Music annotations typically begin with an aural input combined with a variety of visual representations, but the impact of the visuals and aural inputs on the annotations are not known. Here, we present a study where participants annotate music segmentation boundaries of variable strengths given only visuals (audio waveform or piano roll) or only audio or both visuals and audio simultaneously. Participants were presented with the set of 33 contrasting theme and variations extracted from a through-recorded performance of Beethoven’s 32 Variations in C minor, WoO 80, under differing audiovisual conditions. Their segmentation boundaries were visualized using boundary credence profiles and compared using the unbalanced optimal transport distance, which tracks boundary weights and penalizes boundary removal, and compared to the F-measure. Compared to annotations derived from audio/visual (cross-modal) input (considered as the gold standard for our study), boundary annotations derived from visual (unimodal) input were closer than those derived from audio (unimodal) input. The presence of visuals led to larger peaks in boundary credence profiles, marking clearer global segmentations, while audio helped resolve discrepancies and capture subtle segmentation cues. We conclude that audio and visual inputs can be used as cognitive scaffolding to enhance results in large-scale citizen science annotation of music media and to support data analysis and interpretation. In summary, visuals provide cues for big structures, but complex structural nuances are better discerned by ear.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Laboratoire de Mécanique des Structures et des Systèmes Couplés (LMSSC), Conservatoire National des Arts et Métiers (CNAM), Paris, France.
- Institut de Recherche Mathématique Avancée IRMA, University of Strasbourg, Strasbourg, France.
- Department of Engineering and School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom.
ORCID-Profile: Daniel Bedoya, Elaine Chew
Förderung
- HORIZON EUROPE European Research Council
- Sorbonne Université
- École Normale Supérieure
Zitieren
APA
Bedoya, D., Lascabettes, P., Fyfe, L., & Chew, E. (2025). Science of music-based citizen science: How seeing influences hearing. PLOS One. https://doi.org/10.1371/journal.pone.0325019
Vancouver
Bedoya D, Lascabettes P, Fyfe L, Chew E. Science of music-based citizen science: How seeing influences hearing. PLOS One. 2025. doi:10.1371/journal.pone.0325019
BibTeX
@article{Bedoya2025,
author = {Bedoya, Daniel and Lascabettes, Paul and Fyfe, Lawrence and Chew, Elaine},
title = {Science of music-based citizen science: How seeing influences hearing},
journal = {PLOS One},
year = {2025},
doi = {10.1371/journal.pone.0325019}
}
EUVIMED