Towards Context-Rich Automated Biodiversity Assessments: Deriving AI-Powered Insights from Camera Trap Data
Sensors · 2024
DOI: 10.3390/s24248122 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Paul Fergus und 9 weiteren Autor:innen. Beteiligte Einrichtung: School of Computer Science and Mathematics, Liverpool John Moores University, James Parsons Building, Byrom Street, Liverpool L3 3AF, UK. Hauptthemen laut MEDLINE: Biodiversity, Birds, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Camera traps offer enormous new opportunities in ecological studies, but current automated image analysis methods often lack the contextual richness needed to support impactful conservation outcomes. Integrating vision–language models into these workflows could address this gap by providing enhanced contextual understanding and enabling advanced queries across temporal and spatial dimensions. Here, we present an integrated approach that combines deep learning-based vision and language models to improve ecological reporting using data from camera traps. We introduce a two-stage system: YOLOv10-X to localise and classify species (mammals and birds) within images and a Phi-3.5-vision-instruct model to read YOLOv10-X bounding box labels to identify species, overcoming its limitation with hard-to-classify objects in images. Additionally, Phi-3.5 detects broader variables, such as vegetation type and time of day, providing rich ecological and environmental context to YOLO’s species detection output. When combined, this output is processed by the model’s natural language system to answer complex queries, and retrieval-augmented generation (RAG) is employed to enrich responses with external information, like species weight and IUCN status (information that cannot be obtained through direct visual analysis). Combined, this information is used to automatically generate structured reports, providing biodiversity stakeholders with deeper insights into, for example, species abundance, distribution, animal behaviour, and habitat selection. Our approach delivers contextually rich narratives that aid in wildlife management decisions. By providing contextually rich insights, our approach not only reduces manual effort but also supports timely decision making in conservation, potentially shifting efforts from reactive to proactive.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Computer Science and Mathematics, Liverpool John Moores University, James Parsons Building, Byrom Street, Liverpool L3 3AF, UK.
- Chester Zoo, Upton-by-Chester, Chester CH2 IEU, UK.
- Welgevonden Game Reserve, P.O. Box 433, Vaalwater 0530, South Africa.
- School of Mathematics and Statistics, Mathematical Institute, University of St Andrews, North Haugh, St Andrews KY16 9SS, UK.
- School of Biological Sciences, University of Aberdeen, Tillydrone Avenue, Aberdeen AB24 2TZ, UK.
- Astrophysics Research Institute, Liverpool John Moores University, IC2, Liverpool Science Park, 146 Brownlow Hill, Liverpool L3 5RF, UK.
ORCID-Profile: Paul Fergus, Carl Chalmers, Naomi Matthews, Stuart Nixon, André Burger, Oliver Hartley, Chris Sutherland, Xavier Lambin, Steven Longmore, Serge Wich
Zitieren
APA
Fergus, P., Chalmers, C., Matthews, N., Nixon, S., Burger, A., Hartley, O., Sutherland, C., Lambin, X., Longmore, S., & Wich, S. (2024). Towards Context-Rich Automated Biodiversity Assessments: Deriving AI-Powered Insights from Camera Trap Data. Sensors. https://doi.org/10.3390/s24248122
Vancouver
Fergus P, Chalmers C, Matthews N, Nixon S, Burger A, Hartley O, et al. Towards Context-Rich Automated Biodiversity Assessments: Deriving AI-Powered Insights from Camera Trap Data. Sensors. 2024. doi:10.3390/s24248122
BibTeX
@article{Fergus2024,
author = {Fergus, Paul and Chalmers, Carl and Matthews, Naomi and Nixon, Stuart and Burger, André and Hartley, Oliver and Sutherland, Chris and Lambin, Xavier and Longmore, Steven and Wich, Serge},
title = {Towards Context-Rich Automated Biodiversity Assessments: Deriving AI-Powered Insights from Camera Trap Data},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24248122}
}
EUVIMED