EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Resimulation-based self-supervised learning for pretraining physics foundation models

P. Harris, J. Krupa, M. Kagan, B. Maier, N. Woodward

Physical Review D · 21.02.2025 · 111 (3)

DOI: 10.1103/physrevd.111.032010 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Physical Review D, online erschienen am 21.02.2025. Verfasst von P. Harris und 4 weiteren Autor:innen. Die Arbeit wurde bisher 27-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Alexander von Humboldt-Stiftung, U.S. Department of Energy, National Science Foundation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Self-supervised learning (SSL) is at the core of training modern large machine learning models, providing a scheme for learning powerful representations that can be used in a variety of downstream tasks. However, SSL strategies must be adapted to the type of training data and downstream tasks required. We propose resimulation-based self-supervised representation learning (RS3L), a novel simulation-based SSL strategy that employs a method of to drive data augmentation for contrastive learning in the physical sciences, particularly, in fields that rely on stochastic simulators. By intervening in the middle of the simulation process and rerunning simulation components downstream of the intervention, we generate multiple realizations of an event, thus producing a set of augmentations covering all physics-driven variations available in the simulator. Using experiments from high-energy physics, we explore how this strategy may enable the development of a foundation model; we show how RS3L pretraining enables powerful performance in downstream tasks such as discrimination of a variety of objects and uncertainty mitigation. In addition to our results, we make the RS3L dataset publicly available for further studies on how to improve SSL strategies.

Förderung

  • Alexander von Humboldt-Stiftung
  • U.S. Department of Energy
  • National Science Foundation
  • Institute for Artificial Intelligence and Fundamental Interactions

Zitieren

APA

Harris, P., Krupa, J., Kagan, M., Maier, B., & Woodward, N. (2025). Resimulation-based self-supervised learning for pretraining physics foundation models. Physical Review D, 111(3). https://doi.org/10.1103/physrevd.111.032010

Vancouver

Harris P, Krupa J, Kagan M, Maier B, Woodward N. Resimulation-based self-supervised learning for pretraining physics foundation models. Physical Review D. 2025;111(3). doi:10.1103/physrevd.111.032010

BibTeX
@article{Harris2025,
  author = {Harris, P. and Krupa, J. and Kagan, M. and Maier, B. and Woodward, N.},
  title = {Resimulation-based self-supervised learning for pretraining physics foundation models},
  journal = {Physical Review D},
  year = {2025},
  volume = {111},
  number = {3},
  doi = {10.1103/physrevd.111.032010}
}
Kostenlos starten →