EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

BehaveAI enables rapid detection and classification of objects and behavior from motion

Jolyon Troscianko iD, Thomas A O'Shea-Wheller, James A M Galloway iD, Kevin J Gaston

PLOS Biology · 2026

DOI: 10.1371/journal.pbio.3003632 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Biology, 2026. Verfasst von Jolyon Troscianko und 3 weiteren Autor:innen. Beteiligte Einrichtung: Centre for Ecology & Conservation, University of Exeter, Penryn, United Kingdom. Hauptthemen laut MEDLINE: Video Recording, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Natural Environment Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Here we introduce BehaveAI, a biologically inspired video analysis framework that integrates static and motion information through a novel color-from-motion encoding strategy. This method translates object movement—direction, speed, and acceleration—into color gradients, enabling both human annotators and pre-trained convolutional neural networks (CNNs) to infer motion patterns while retaining high-resolution spatial detail. Using a range of case studies, we demonstrate how the increased salience of motion information allows for the robust detection of objects that are challenging or impossible to identify reliably from static frames alone, particularly in complex natural scenes. We further demonstrate the reliable classification of different behaviors in animals and single-celled organisms. Additionally, the framework supports flexible hierarchical model structures that can separate the tasks of detection and classification for optimal efficiency, and provide individual tracking data that specifies what is present where and what it is doing in each frame. The framework makes use of the latest deep learning architecture (YOLO11), combined with a semi-supervised annotation workflow. Together with salient motion information, these features can dramatically reduce the effort required for dataset annotation such that reliable models can often be made within an hour. Moreover, smaller annotation datasets mean that model training can be achieved on conventional computers without dedicated hardware, thereby improving accessibility. The motion encoding approach is also computationally lightweight, and can run in real-time on low-end edge devices such as a Raspberry Pi. We release the framework as a free, open source, and user-friendly package.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Video RecordingImage Processing, Computer-AssistedAnimalsConvolutional Neural NetworksDetection AlgorithmsHumansMotionDeep LearningNeural Networks, ComputerAlgorithms

Autor:innen und Institutionen

  • Centre for Ecology & Conservation, University of Exeter, Penryn, United Kingdom.
  • Environment and Sustainability Institute, University of Exeter, Penryn, United Kingdom.

ORCID-Profile: Jolyon Troscianko, James A M Galloway

Förderung

  • Natural Environment Research Council

Zitieren

APA

Troscianko, J., O'Shea-Wheller, T. A., Galloway, J. A. M., & Gaston, K. J. (2026). BehaveAI enables rapid detection and classification of objects and behavior from motion. PLOS Biology. https://doi.org/10.1371/journal.pbio.3003632

Vancouver

Troscianko J, O'Shea-Wheller TA, Galloway JAM, Gaston KJ. BehaveAI enables rapid detection and classification of objects and behavior from motion. PLOS Biology. 2026. doi:10.1371/journal.pbio.3003632

BibTeX
@article{Troscianko2026,
  author = {Troscianko, Jolyon and O'Shea-Wheller, Thomas A and Galloway, James A M and Gaston, Kevin J},
  title = {BehaveAI enables rapid detection and classification of objects and behavior from motion},
  journal = {PLOS Biology},
  year = {2026},
  doi = {10.1371/journal.pbio.3003632}
}
Kostenlos starten →