EUVIMEDSkopri EUVIMED
Open access · CC BY 4.0

The Co‐Structuring of Gesture‐Vocal Dynamics: An Exploration in Karnatak Music Performance

Lara Pearson iD, Thomas Nuttall iD, Wim Pouw iD

Cognitive Science · 2025

DOI: 10.1111/cogs.70137 ↗

At a glance

Journal article in Cognitive Science, 2025. Written by Lara Pearson and 2 co-authors. Institution: Department of Music, Max Planck Institute for Empirical Aesthetics. Major MEDLINE topics: Gestures, Music, Singing. 76 references. The full text is freely available (open access, CC BY 4.0). Funded by: Dutch Research Council (NWO), DFG priority program ViCom grant.

AstrattSubjectsDettalji biblijografiċiCiteRelated studies

Abstract (original language)

In music performance contexts, vocalists tend to gesture with hand and upper body movements as they sing. But how does this gesturing relate to the sung phrases, and how do singers’ gesturing styles differ from each other? In this study, we present a quantitative analysis and visualization pipeline that characterizes the multidimensional co‐structuring of body movements and vocalizations in vocal performers. We apply this to a dataset of performances within the Karnatak music tradition of South India, including audio and motion tracking data of 44 performances by three expert Karnatak vocalists, openly published with this report. Our results show that time‐varying features of head and hand gestures tend to be more similar when the concurrent vocal time‐varying features are also more similar. While for each performer we find clear co‐structuring of sound and movement, they each show their own characteristic salient dimensions (e.g., hand position, head acceleration) through which movement co‐structures with singing. Our time‐series analyses thereby provide a computational approach to characterizing individual vocalists’ unique multimodal vocal‐gesture co‐structuring profiles. We also show that co‐structuring clearly reduces degrees of freedom of the multimodal performance such that motifs that sound alike tend to co‐structure with gestures that move alike. The current method can be applied to any multimodally ensembled signals in both human and nonhuman communication, to determine co‐structuring profiles and explore any reduction in degrees of freedom. In the context of Karnatak singing performance, the current analysis is an important starting point for further experimental study of gesture‐vocal synergies.

Subjects

MeSH terms (MEDLINE)

GesturesMusicSingingHumansIndiaMaleFemaleAdultMovement

Author keywords

Data dashboardDynamic time warpingGestureMotion trackingMultimodalitySouth Indian musicVocal performance

Authors and institutions

  • Department of Music, Max Planck Institute for Empirical Aesthetics.
  • Institute of Musicology, Faculty of Arts and Humanities, University of Cologne.
  • Music Technology Group, Pompeu Fabra University.
  • Department of Computational Cognitive Science, Research Center for Cognitive Science and Artificial Intelligence, Tilburg University.
  • Donders Institute for Brain, Cognition, and Behaviour, Radboud University.

ORCID profiles: Lara Pearson, Thomas Nuttall, Wim Pouw

Funding

  • Dutch Research Council (NWO)
  • DFG priority program ViCom grant

Cite

APA

Pearson, L., Nuttall, T., & Pouw, W. (2025). The Co‐Structuring of Gesture‐Vocal Dynamics: An Exploration in Karnatak Music Performance. Cognitive Science. https://doi.org/10.1111/cogs.70137

Vancouver

Pearson L, Nuttall T, Pouw W. The Co‐Structuring of Gesture‐Vocal Dynamics: An Exploration in Karnatak Music Performance. Cognitive Science. 2025. doi:10.1111/cogs.70137

BibTeX
@article{Pearson2025,
  author = {Pearson, Lara and Nuttall, Thomas and Pouw, Wim},
  title = {The Co‐Structuring of Gesture‐Vocal Dynamics: An Exploration in Karnatak Music Performance},
  journal = {Cognitive Science},
  year = {2025},
  doi = {10.1111/cogs.70137}
}
Start for free →