EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Multi‐Modal Instrument Performances (MMIP): A Musical Database

T. Kyriakou, A. Aristidou, P. Charalambous

Computer Graphics Forum · 2025-05 · 44 (2)

DOI: 10.1111/cgf.70025 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computer Graphics Forum, online erschienen am 10.04.2025. Verfasst von T. Kyriakou und 2 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Horizon 2020 Framework Programme, Deputy Ministry of Research, Innovation and Digital Policy.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Musical instrument performances are multimodal creative art forms that integrate audiovisual elements, resulting from musicians' interactions with instruments through body movements, finger actions, and facial expressions. Digitizing such performances for archiving, streaming, analysis, or synthesis requires capturing every element that shapes the overall experience, which is crucial for preserving the performance's essence. In this work, following current trends in large‐scale dataset development for deep learning analysis and generative models, we introduce the Multi‐Modal Instrument Performances (MMIP) database ( https://mmip.cs.ucy.ac.cy ). This is the first dataset to incorporate synchronized high‐quality 3D motion capture data for the body, fingers, facial expressions, and instruments, along with audio, multi‐angle videos, and MIDI data. The database currently includes 3.5 hours of performances featuring three instruments: guitar, piano, and drums. Additionally, we discuss the challenges of acquiring these multi‐modal data, detailing our approach to data collection, signal synchronization, annotation, and metadata management. Our data formats align with industry standards for ease of use, and we have developed an open‐access online repository that offers a user‐friendly environment for data exploration, supporting data organization, search capabilities, and custom visualization tools. Notable features include a MIDI‐to‐instrument animation project for visualizing the instruments and a script for playing back FBX files with synchronized audio in a web environment.

Förderung

  • Horizon 2020 Framework Programme
  • Deputy Ministry of Research, Innovation and Digital Policy

Zitieren

APA

Kyriakou, T., Aristidou, A., & Charalambous, P. (2025). Multi‐Modal Instrument Performances (MMIP): A Musical Database. Computer Graphics Forum, 44(2). https://doi.org/10.1111/cgf.70025

Vancouver

Kyriakou T, Aristidou A, Charalambous P. Multi‐Modal Instrument Performances (MMIP): A Musical Database. Computer Graphics Forum. 2025;44(2). doi:10.1111/cgf.70025

BibTeX
@article{Kyriakou2025,
  author = {Kyriakou, T. and Aristidou, A. and Charalambous, P.},
  title = {Multi‐Modal Instrument Performances (MMIP): A Musical Database},
  journal = {Computer Graphics Forum},
  year = {2025},
  volume = {44},
  number = {2},
  doi = {10.1111/cgf.70025}
}
Kostenlos starten →