EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Being confident in confidence scores: calibration in deep learning models for camera trap image sequences

Gaspard Dussert, Simon Chamaillé‐Jammes, Stéphane Dray, Vincent Miele

Remote Sensing in Ecology and Conservation · 2025-02 · 11 (1) · 88-99

DOI: 10.1002/rse2.412 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Remote Sensing in Ecology and Conservation, online erschienen am 16.06.2024. Verfasst von Gaspard Dussert und 3 weiteren Autor:innen. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Grand Équipement National De Calcul Intensif.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In ecological studies, machine learning models are increasingly being used for the automatic processing of camera trap images. Although this automation facilitates and accelerates the identification step, the results of these models may lack interpretability and their immediate applicability to ecological downstream tasks (e.g. occupancy estimation) remains questionable. In particular, little is known about their calibration, a property that allows confidence scores to be interpreted as probabilities that model's predictions are true. Using a large and diverse European camera trap dataset, we investigate whether deep learning models for species classification in camera trap images are well calibrated. Additionally, as camera traps are often configured to take multiple photos of the same event, we also explore the calibration of predictions aggregated across sequences of images. Finally, we study the effect and the practicality of a post‐hoc calibration method, i.e. temperature scaling, for predictions made at image and sequence levels. Based on five established models and three independent test sets, we show that averaging the logits over the sequence, selecting an appropriate architecture, and optionally using temperature scaling can produce well‐calibrated models. Our findings have clear implication for, for instance, the calculation of error rates or the selection of confidence score thresholds in ecological studies making use of artificial intelligence models.

Förderung

  • Grand Équipement National De Calcul Intensif

Zitieren

APA

Dussert, G., Chamaillé‐Jammes, S., Dray, S., & Miele, V. (2025). Being confident in confidence scores: calibration in deep learning models for camera trap image sequences. Remote Sensing in Ecology and Conservation, 11(1), 88-99. https://doi.org/10.1002/rse2.412

Vancouver

Dussert G, Chamaillé‐Jammes S, Dray S, Miele V. Being confident in confidence scores: calibration in deep learning models for camera trap image sequences. Remote Sensing in Ecology and Conservation. 2025;11(1):88-99. doi:10.1002/rse2.412

BibTeX
@article{Dussert2025,
  author = {Dussert, Gaspard and Chamaillé‐Jammes, Simon and Dray, Stéphane and Miele, Vincent},
  title = {Being confident in confidence scores: calibration in deep learning models for camera trap image sequences},
  journal = {Remote Sensing in Ecology and Conservation},
  year = {2025},
  volume = {11},
  number = {1},
  pages = {88--99},
  doi = {10.1002/rse2.412}
}
Kostenlos starten →