EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

The Co‐Structuring of Gesture‐Vocal Dynamics: An Exploration in Karnatak Music Performance

Lara Pearson iD, Thomas Nuttall iD, Wim Pouw iD

Cognitive Science · 2025

DOI: 10.1111/cogs.70137 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cognitive Science, 2025. Verfasst von Lara Pearson und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Music, Max Planck Institute for Empirical Aesthetics. Hauptthemen laut MEDLINE: Gestures, Music, Singing. Die Arbeit nennt 76 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Dutch Research Council (NWO), DFG priority program ViCom grant.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In music performance contexts, vocalists tend to gesture with hand and upper body movements as they sing. But how does this gesturing relate to the sung phrases, and how do singers’ gesturing styles differ from each other? In this study, we present a quantitative analysis and visualization pipeline that characterizes the multidimensional co‐structuring of body movements and vocalizations in vocal performers. We apply this to a dataset of performances within the Karnatak music tradition of South India, including audio and motion tracking data of 44 performances by three expert Karnatak vocalists, openly published with this report. Our results show that time‐varying features of head and hand gestures tend to be more similar when the concurrent vocal time‐varying features are also more similar. While for each performer we find clear co‐structuring of sound and movement, they each show their own characteristic salient dimensions (e.g., hand position, head acceleration) through which movement co‐structures with singing. Our time‐series analyses thereby provide a computational approach to characterizing individual vocalists’ unique multimodal vocal‐gesture co‐structuring profiles. We also show that co‐structuring clearly reduces degrees of freedom of the multimodal performance such that motifs that sound alike tend to co‐structure with gestures that move alike. The current method can be applied to any multimodally ensembled signals in both human and nonhuman communication, to determine co‐structuring profiles and explore any reduction in degrees of freedom. In the context of Karnatak singing performance, the current analysis is an important starting point for further experimental study of gesture‐vocal synergies.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

GesturesMusicSingingHumansIndiaMaleFemaleAdultMovement

Stichwörter der Autor:innen

Data dashboardDynamic time warpingGestureMotion trackingMultimodalitySouth Indian musicVocal performance

Autor:innen und Institutionen

  • Department of Music, Max Planck Institute for Empirical Aesthetics.
  • Institute of Musicology, Faculty of Arts and Humanities, University of Cologne.
  • Music Technology Group, Pompeu Fabra University.
  • Department of Computational Cognitive Science, Research Center for Cognitive Science and Artificial Intelligence, Tilburg University.
  • Donders Institute for Brain, Cognition, and Behaviour, Radboud University.

ORCID-Profile: Lara Pearson, Thomas Nuttall, Wim Pouw

Förderung

  • Dutch Research Council (NWO)
  • DFG priority program ViCom grant

Zitieren

APA

Pearson, L., Nuttall, T., & Pouw, W. (2025). The Co‐Structuring of Gesture‐Vocal Dynamics: An Exploration in Karnatak Music Performance. Cognitive Science. https://doi.org/10.1111/cogs.70137

Vancouver

Pearson L, Nuttall T, Pouw W. The Co‐Structuring of Gesture‐Vocal Dynamics: An Exploration in Karnatak Music Performance. Cognitive Science. 2025. doi:10.1111/cogs.70137

BibTeX
@article{Pearson2025,
  author = {Pearson, Lara and Nuttall, Thomas and Pouw, Wim},
  title = {The Co‐Structuring of Gesture‐Vocal Dynamics: An Exploration in Karnatak Music Performance},
  journal = {Cognitive Science},
  year = {2025},
  doi = {10.1111/cogs.70137}
}
Kostenlos starten →