EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Q-learning with temporal memory to navigate turbulence

Marco Rando iD, Martin James, Alessandro Verri, Lorenzo Rosasco, Agnese Seminara iD

eLife · 2025

DOI: 10.7554/elife.102906 ↗

Kurz gefasst

Fachartikel in der Zeitschrift eLife, 2025. Verfasst von Marco Rando und 4 weiteren Autor:innen. Beteiligte Einrichtung: MaLGa, Department of Computer Science, Bioengineering, Robotics and Systems Engineering, University of Genova, Genoa, Italy. Hauptthemen laut MEDLINE: Odorants, Memory, Learning, Smell. Die Arbeit nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Research Council, HORIZON EUROPE Framework Programme, NIDCD NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We consider the problem of olfactory searches in a turbulent environment. We focus on agents that respond solely to odor stimuli, with no access to spatial perception nor prior information about the odor. We ask whether navigation to a target can be learned robustly within a sequential decision making framework. We develop a reinforcement learning algorithm using a small set of interpretable olfactory states and train it with realistic turbulent odor cues. By introducing a temporal memory, we demonstrate that two salient features of odor traces, discretized in a few olfactory states, are sufficient to learn navigation in a realistic odor plume. Performance is dictated by the sparse nature of turbulent odors. An optimal memory exists which ignores blanks within the plume and activates a recovery strategy outside the plume. We obtain the best performance by letting agents learn their recovery strategy and show that it is mostly casting cross wind, similar to behavior observed in flying insects. The optimal strategy is robust to substantial changes in the odor plumes, suggesting minor parameter tuning may be sufficient to adapt to different environments.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

OdorantsMemoryLearningSmellOlfactory PerceptionAnimalsAlgorithms

Stichwörter der Autor:innen

memorynoneolfactory navigationphysics of living systemsreinforcement learningtime seriesturbulence

Autor:innen und Institutionen

  • MaLGa, Department of Computer Science, Bioengineering, Robotics and Systems Engineering, University of Genova, Genoa, Italy.
  • MalGa, Department of Civil, Chemical and Environmental Engineering, University of Genoa, Genova, Italy.

ORCID-Profile: Marco Rando, Agnese Seminara

Förderung

  • European Research Council
  • HORIZON EUROPE Framework Programme
  • NIDCD NIH HHS
  • Air Force Office of Scientific Research
  • National Science Foundation
  • Ministero dell'Istruzione, dell'Università e della Ricerca
  • NIH HHS
  • NextGenerationEU

Zitieren

APA

Rando, M., James, M., Verri, A., Rosasco, L., & Seminara, A. (2025). Q-learning with temporal memory to navigate turbulence. eLife. https://doi.org/10.7554/elife.102906

Vancouver

Rando M, James M, Verri A, Rosasco L, Seminara A. Q-learning with temporal memory to navigate turbulence. eLife. 2025. doi:10.7554/elife.102906

BibTeX
@article{Rando2025,
  author = {Rando, Marco and James, Martin and Verri, Alessandro and Rosasco, Lorenzo and Seminara, Agnese},
  title = {Q-learning with temporal memory to navigate turbulence},
  journal = {eLife},
  year = {2025},
  doi = {10.7554/elife.102906}
}
Kostenlos starten →