EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Hybrid Approach to Optical Music Recognition With Object Detection and Multimodal LLMs

Gustavo Henrique Romão, Hygor Santiago Lara, Jesuliana Nascimento Ulysses, Jorge Nei Brito

Revista Eletrônica de Iniciação Científica em Computação · 13.03.2026 · 24 · 134-137

DOI: 10.5753/reic.2026.6891 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Revista Eletrônica de Iniciação Científica em Computação, online erschienen am 13.03.2026. Verfasst von Gustavo Henrique Romão und 3 weiteren Autor:innen. Die Arbeit nennt 11 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This research introduces a hybrid methodology for Optical Music Recognition (OMR), integrating multimodal language models (LLMs) with contemporary object detection approaches. For clef identification, Gemini 2.0 Flash was employed, capitalizing on its visual and contextual interpretation capabilities, while YOLOv8 and YOLOv11 were adopted for processing pitch value and rhythm detection. This task distribution minimizes object detection complexity, enabling YOLO models to concentrate on precise localization and classification of musical symbols. The proposed methodology demonstrated promising outcomes in the task of recognizing digital monophonic scores, with YOLOv11 achieving a mAP50 of 0.995 in the pitch detection network when clef detection is performed through LLMs.

Zitieren

APA

Romão, G. H., Lara, H. S., Ulysses, J. N., & Brito, J. N. (2026). A Hybrid Approach to Optical Music Recognition With Object Detection and Multimodal LLMs. Revista Eletrônica de Iniciação Científica em Computação, 24, 134-137. https://doi.org/10.5753/reic.2026.6891

Vancouver

Romão GH, Lara HS, Ulysses JN, Brito JN. A Hybrid Approach to Optical Music Recognition With Object Detection and Multimodal LLMs. Revista Eletrônica de Iniciação Científica em Computação. 2026;24:134-137. doi:10.5753/reic.2026.6891

BibTeX
@article{Romo2026,
  author = {Romão, Gustavo Henrique and Lara, Hygor Santiago and Ulysses, Jesuliana Nascimento and Brito, Jorge Nei},
  title = {A Hybrid Approach to Optical Music Recognition With Object Detection and Multimodal LLMs},
  journal = {Revista Eletrônica de Iniciação Científica em Computação},
  year = {2026},
  volume = {24},
  pages = {134--137},
  doi = {10.5753/reic.2026.6891}
}
Kostenlos starten →