EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ReTrackVLM: Transformer-Enhanced Multi-Object Tracking with Cross-Modal Embeddings and Zero-Shot Re-Identification Integration

Ertugrul Bayraktar

Applied Sciences · 12.02.2025 · 15 (4) · 1907

DOI: 10.3390/app15041907 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 12.02.2025. Verfasst von Ertugrul Bayraktar. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 73 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-object tracking (MOT) is an important task in computer vision, particularly in complex, dynamic environments with crowded scenes and frequent occlusions. Traditional tracking methods often suffer from identity switches (IDSws) and fragmented tracks (FMs), which limits their ability to maintain consistent object trajectories. In this paper, we present a novel framework, called ReTrackVLM, that integrates multimodal embedding from a visual language model (VLM) with a zero-shot re-identification (ReID) module to enhance tracking accuracy and robustness. ReTrackVLM leverages the rich semantic information from VLMs to distinguish objects more effectively, even under challenging conditions, while the zero-shot ReID mechanism enables robust identity matching without additional training. The system also includes a motion prediction module, powered by Kalman filtering, to handle object occlusions and abrupt movements. We evaluated ReTrackVLM on several widely used MOT benchmarks, including MOT15, MOT16, MOT17, MOT20, and DanceTrack. Our approach achieves state-of-the-art results, with improvements of 1.5% MOTA and a reduction of 10. 3% in IDSws compared to existing methods. ReTrackVLM also excels in tracking precision, recording a 91.7% precision on MOT17. However, in extremely dense scenes, the framework faces challenges with slight increases in IDSws. Despite the computational overhead of using VLMs, ReTrackVLM demonstrates the ability to track objects effectively in diverse scenarios.

Zitieren

APA

Bayraktar, E. (2025). ReTrackVLM: Transformer-Enhanced Multi-Object Tracking with Cross-Modal Embeddings and Zero-Shot Re-Identification Integration. Applied Sciences, 15(4), 1907. https://doi.org/10.3390/app15041907

Vancouver

Bayraktar E. ReTrackVLM: Transformer-Enhanced Multi-Object Tracking with Cross-Modal Embeddings and Zero-Shot Re-Identification Integration. Applied Sciences. 2025;15(4):1907. doi:10.3390/app15041907

BibTeX
@article{Bayraktar2025,
  author = {Bayraktar, Ertugrul},
  title = {ReTrackVLM: Transformer-Enhanced Multi-Object Tracking with Cross-Modal Embeddings and Zero-Shot Re-Identification Integration},
  journal = {Applied Sciences},
  year = {2025},
  volume = {15},
  number = {4},
  pages = {1907},
  doi = {10.3390/app15041907}
}
Kostenlos starten →