EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Abrupt and spontaneous strategy switches emerge in simple regularised neural networks

Anika T Löwe iD, Léo Touzo, Paul S Muhle-Karbe, Andrew M Saxe, Christopher Summerfield, Nicolas W Schuck iD

PLOS Computational Biology · 2024

DOI: 10.1371/journal.pcbi.1012505 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, 2024. Verfasst von Anika T Löwe und 5 weiteren Autor:innen. Beteiligte Einrichtung: Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany. Hauptthemen laut MEDLINE: Neural Networks, Computer, Learning. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 71 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wellcome Trust, Max-Planck-Gesellschaft, Wellcome Trust and Royal Society.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Humans sometimes have an insight that leads to a sudden and drastic performance improvement on the task they are working on. Sudden strategy adaptations are often linked to insights, considered to be a unique aspect of human cognition tied to complex processes such as creativity or meta-cognitive reasoning. Here, we take a learning perspective and ask whether insight-like behaviour can occur in simple artificial neural networks, even when the models only learn to form input-output associations through gradual gradient descent. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that included a hidden regularity to solve the task more efficiently. Our results show that only some humans discover this regularity, and that behaviour is marked by a sudden and abrupt strategy switch that reflects an aha-moment. Notably, we find that simple neural networks with a gradual learning rule and a constant learning rate closely mimicked behavioural characteristics of human insight-like switches, exhibiting delay of insight, suddenness and selective occurrence in only some networks. Analyses of network architectures and learning dynamics revealed that insight-like behaviour crucially depended on a regularised gating mechanism and noise added to gradient updates, which allowed the networks to accumulate “silent knowledge” that is initially suppressed by regularised gating. This suggests that insight-like behaviour can arise from gradual learning in simple neural networks, where it reflects the combined influences of noise, gating and regularisation. These results have potential implications for more complex systems, such as the brain, and guide the way for future insight research.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Neural Networks, ComputerLearningHumansCognitionComputational BiologyModels, NeurologicalAdultMaleDecision MakingFemaleYoung Adult

Autor:innen und Institutionen

  • Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Berlin, Germany.
  • Max Planck UCL Centre for Computational Psychiatry and Ageing Research, Berlin, Germany.
  • Institute of Psychology, Universität Hamburg, Hamburg, Germany.
  • Laboratoire de Physique de l'Ecole Normale Supérieure, CNRS, ENS, Université PSL, Sorbonne Université, Université Paris Cité, Paris, France.
  • Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom.
  • School of Psychology, University of Birmingham, Birmingham, United Kingdom.

ORCID-Profile: Anika T Löwe, Nicolas W Schuck

Förderung

  • Wellcome Trust
  • Max-Planck-Gesellschaft
  • Wellcome Trust and Royal Society
  • Gatsby Charitable Foundation
  • HORIZON EUROPE European Research Council
  • European Union
  • BMBF

Zitieren

APA

Löwe, A. T., Touzo, L., Muhle-Karbe, P. S., Saxe, A. M., Summerfield, C., & Schuck, N. W. (2024). Abrupt and spontaneous strategy switches emerge in simple regularised neural networks. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1012505

Vancouver

Löwe AT, Touzo L, Muhle-Karbe PS, Saxe AM, Summerfield C, Schuck NW. Abrupt and spontaneous strategy switches emerge in simple regularised neural networks. PLOS Computational Biology. 2024. doi:10.1371/journal.pcbi.1012505

BibTeX
@article{Lwe2024,
  author = {Löwe, Anika T and Touzo, Léo and Muhle-Karbe, Paul S and Saxe, Andrew M and Summerfield, Christopher and Schuck, Nicolas W},
  title = {Abrupt and spontaneous strategy switches emerge in simple regularised neural networks},
  journal = {PLOS Computational Biology},
  year = {2024},
  doi = {10.1371/journal.pcbi.1012505}
}
Kostenlos starten →