EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Animal Pose Tracking: 3D Multimodal Dataset and Token-based Pose Optimization

Mahir Patel, Yiwen Gu, Lucas C. Carstensen, Michael E. Hasselmo, Margrit Betke

International Journal of Computer Vision · 2023-02 · 131 (2) · 514-530

DOI: 10.1007/s11263-022-01714-5 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 23.11.2022. Verfasst von Mahir Patel und 4 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Office of Naval Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate tracking of the 3D pose of animals from video recordings is critical for many behavioral studies, yet there is a dearth of publicly available datasets that the computer vision community could use for model development. We here introduce the Rodent3D dataset that records animals exploring their environment and/or interacting with each other with multiple cameras and modalities (RGB, depth, thermal infrared). Rodent3D consists of 200 min of multimodal video recordings from up to three thermal and three RGB-D synchronized cameras (approximately 4 million frames). For the task of optimizing estimates of pose sequences provided by existing pose estimation methods, we provide a baseline model called OptiPose . While deep-learned attention mechanisms have been used for pose estimation in the past, with OptiPose , we propose a different way by representing 3D poses as tokens for which deep-learned context models pay attention to both spatial and temporal keypoint patterns. Our experiments show how OptiPose is highly robust to noise and occlusion and can be used to optimize pose sequences provided by state-of-the-art models for animal pose estimation.

Förderung

  • Office of Naval Research

Zitieren

APA

Patel, M., Gu, Y., Carstensen, L. C., Hasselmo, M. E., & Betke, M. (2023). Animal Pose Tracking: 3D Multimodal Dataset and Token-based Pose Optimization. International Journal of Computer Vision, 131(2), 514-530. https://doi.org/10.1007/s11263-022-01714-5

Vancouver

Patel M, Gu Y, Carstensen LC, Hasselmo ME, Betke M. Animal Pose Tracking: 3D Multimodal Dataset and Token-based Pose Optimization. International Journal of Computer Vision. 2023;131(2):514-530. doi:10.1007/s11263-022-01714-5

BibTeX
@article{Patel2023,
  author = {Patel, Mahir and Gu, Yiwen and Carstensen, Lucas C. and Hasselmo, Michael E. and Betke, Margrit},
  title = {Animal Pose Tracking: 3D Multimodal Dataset and Token-based Pose Optimization},
  journal = {International Journal of Computer Vision},
  year = {2023},
  volume = {131},
  number = {2},
  pages = {514--530},
  doi = {10.1007/s11263-022-01714-5}
}
Kostenlos starten →