EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Science of music-based citizen science: How seeing influences hearing

Daniel Bedoya iD, Paul Lascabettes, Lawrence Fyfe, Elaine Chew iD

PLOS One · 2025

DOI: 10.1371/journal.pone.0325019 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Daniel Bedoya und 3 weiteren Autor:innen. Beteiligte Einrichtung: Laboratoire de Mécanique des Structures et des Systèmes Couplés (LMSSC), Conservatoire National des Arts et Métiers (CNAM), Paris, France. Hauptthemen laut MEDLINE: Citizen Science, Music, Hearing, Auditory Perception. Die Arbeit nennt 34 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: HORIZON EUROPE European Research Council, Sorbonne Université, École Normale Supérieure.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Citizen science engages volunteers to contribute data to scientific projects, often through visual annotation tasks. Hearing based activities are rare and less well understood. Having high quality annotations of performed music structures is essential for reliable algorithmic analysis of recorded music with applications ranging from music information retrieval to music therapy. Music annotations typically begin with an aural input combined with a variety of visual representations, but the impact of the visuals and aural inputs on the annotations are not known. Here, we present a study where participants annotate music segmentation boundaries of variable strengths given only visuals (audio waveform or piano roll) or only audio or both visuals and audio simultaneously. Participants were presented with the set of 33 contrasting theme and variations extracted from a through-recorded performance of Beethoven’s 32 Variations in C minor, WoO 80, under differing audiovisual conditions. Their segmentation boundaries were visualized using boundary credence profiles and compared using the unbalanced optimal transport distance, which tracks boundary weights and penalizes boundary removal, and compared to the F-measure. Compared to annotations derived from audio/visual (cross-modal) input (considered as the gold standard for our study), boundary annotations derived from visual (unimodal) input were closer than those derived from audio (unimodal) input. The presence of visuals led to larger peaks in boundary credence profiles, marking clearer global segmentations, while audio helped resolve discrepancies and capture subtle segmentation cues. We conclude that audio and visual inputs can be used as cognitive scaffolding to enhance results in large-scale citizen science annotation of music media and to support data analysis and interpretation. In summary, visuals provide cues for big structures, but complex structural nuances are better discerned by ear.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Citizen ScienceMusicHearingAuditory PerceptionVisual PerceptionHumansMaleFemaleAdultYoung AdultAcoustic Stimulation

Autor:innen und Institutionen

  • Laboratoire de Mécanique des Structures et des Systèmes Couplés (LMSSC), Conservatoire National des Arts et Métiers (CNAM), Paris, France.
  • Institut de Recherche Mathématique Avancée IRMA, University of Strasbourg, Strasbourg, France.
  • Department of Engineering and School of Biomedical Engineering and Imaging Sciences, King's College London, London, United Kingdom.

ORCID-Profile: Daniel Bedoya, Elaine Chew

Förderung

  • HORIZON EUROPE European Research Council
  • Sorbonne Université
  • École Normale Supérieure

Zitieren

APA

Bedoya, D., Lascabettes, P., Fyfe, L., & Chew, E. (2025). Science of music-based citizen science: How seeing influences hearing. PLOS One. https://doi.org/10.1371/journal.pone.0325019

Vancouver

Bedoya D, Lascabettes P, Fyfe L, Chew E. Science of music-based citizen science: How seeing influences hearing. PLOS One. 2025. doi:10.1371/journal.pone.0325019

BibTeX
@article{Bedoya2025,
  author = {Bedoya, Daniel and Lascabettes, Paul and Fyfe, Lawrence and Chew, Elaine},
  title = {Science of music-based citizen science: How seeing influences hearing},
  journal = {PLOS One},
  year = {2025},
  doi = {10.1371/journal.pone.0325019}
}
Kostenlos starten →