EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

AstroCLIP: a cross-modal foundation model for galaxies

Liam Parker, Francois Lanusse, Siavash Golkar, Leopoldo Sarra, Miles Cranmer, Alberto Bietti, Michael Eickenberg, Geraud Krawezik, Michael McCabe, Rudy Morel, Ruben Ohana, Mariel Pettee und 3 weitere

Monthly Notices of the Royal Astronomical Society · 14.06.2024 · 531 (4) · 4990-5011

DOI: 10.1093/mnras/stae1450 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Monthly Notices of the Royal Astronomical Society, online erschienen am 21.06.2024. Verfasst von Liam Parker und 14 weiteren Autor:innen. Die Arbeit wurde bisher 79-mal zitiert (Crossref) und nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: U.S. Department of Energy Office of Science.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present AstroCLIP, a single, versatile model that can embed both galaxy images and spectra into a shared, physically meaningful latent space. These embeddings can then be used – without any model fine-tuning – for a variety of downstream tasks including (1) accurate in-modality and cross-modality semantic similarity search, (2) photometric redshift estimation, (3) galaxy property estimation from both images and spectra, and (4) morphology classification. Our approach to implementing AstroCLIP consists of two parts. First, we embed galaxy images and spectra separately by pre-training separate transformer-based image and spectrum encoders in self-supervised settings. We then align the encoders using a contrastive loss. We apply our method to spectra from the Dark Energy Spectroscopic Instrument and images from its corresponding Legacy Imaging Survey. Overall, we find remarkable performance on all downstream tasks, even relative to supervised baselines. For example, for a task like photometric redshift prediction, we find similar performance to a specifically trained ResNet18, and for additional tasks like physical property estimation (stellar mass, age, metallicity, and specific-star-formation rate), we beat this supervised baseline by 19 per cent in terms of R2. We also compare our results with a state-of-the-art self-supervised single-modal model for galaxy images, and find that our approach outperforms this benchmark by roughly a factor of two on photometric redshift estimation and physical property prediction in terms of R2, while remaining roughly in-line in terms of morphology classification. Ultimately, our approach represents the first cross-modal self-supervised model for galaxies, and the first self-supervised transformer-based architectures for galaxy images and spectra.

Förderung

  • U.S. Department of Energy Office of Science

Zitieren

APA

Parker, L., Lanusse, F., Golkar, S., Sarra, L., Cranmer, M., Bietti, A., Eickenberg, M., Krawezik, G., McCabe, M., Morel, R., Ohana, R., Pettee, M., Blancard, B. R., Cho, K., & Ho, S. (2024). AstroCLIP: a cross-modal foundation model for galaxies. Monthly Notices of the Royal Astronomical Society, 531(4), 4990-5011. https://doi.org/10.1093/mnras/stae1450

Vancouver

Parker L, Lanusse F, Golkar S, Sarra L, Cranmer M, Bietti A, et al. AstroCLIP: a cross-modal foundation model for galaxies. Monthly Notices of the Royal Astronomical Society. 2024;531(4):4990-5011. doi:10.1093/mnras/stae1450

BibTeX
@article{Parker2024,
  author = {Parker, Liam and Lanusse, Francois and Golkar, Siavash and Sarra, Leopoldo and Cranmer, Miles and Bietti, Alberto and Eickenberg, Michael and Krawezik, Geraud and McCabe, Michael and Morel, Rudy and Ohana, Ruben and Pettee, Mariel and Blancard, Bruno Régaldo-Saint and Cho, Kyunghyun and Ho, Shirley},
  title = {AstroCLIP: a cross-modal foundation model for galaxies},
  journal = {Monthly Notices of the Royal Astronomical Society},
  year = {2024},
  volume = {531},
  number = {4},
  pages = {4990--5011},
  doi = {10.1093/mnras/stae1450}
}
Kostenlos starten →