AstroCLIP: a cross-modal foundation model for galaxies
Monthly Notices of the Royal Astronomical Society · 14.06.2024 · 531 (4) · 4990-5011
DOI: 10.1093/mnras/stae1450 ↗Kurz gefasst
Fachartikel in der Zeitschrift Monthly Notices of the Royal Astronomical Society, online erschienen am 21.06.2024. Verfasst von Liam Parker und 14 weiteren Autor:innen. Die Arbeit wurde bisher 79-mal zitiert (Crossref) und nennt 62 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: U.S. Department of Energy Office of Science.
Abstract (Originalsprache)
We present AstroCLIP, a single, versatile model that can embed both galaxy images and spectra into a shared, physically meaningful latent space. These embeddings can then be used – without any model fine-tuning – for a variety of downstream tasks including (1) accurate in-modality and cross-modality semantic similarity search, (2) photometric redshift estimation, (3) galaxy property estimation from both images and spectra, and (4) morphology classification. Our approach to implementing AstroCLIP consists of two parts. First, we embed galaxy images and spectra separately by pre-training separate transformer-based image and spectrum encoders in self-supervised settings. We then align the encoders using a contrastive loss. We apply our method to spectra from the Dark Energy Spectroscopic Instrument and images from its corresponding Legacy Imaging Survey. Overall, we find remarkable performance on all downstream tasks, even relative to supervised baselines. For example, for a task like photometric redshift prediction, we find similar performance to a specifically trained ResNet18, and for additional tasks like physical property estimation (stellar mass, age, metallicity, and specific-star-formation rate), we beat this supervised baseline by 19 per cent in terms of R2. We also compare our results with a state-of-the-art self-supervised single-modal model for galaxy images, and find that our approach outperforms this benchmark by roughly a factor of two on photometric redshift estimation and physical property prediction in terms of R2, while remaining roughly in-line in terms of morphology classification. Ultimately, our approach represents the first cross-modal self-supervised model for galaxies, and the first self-supervised transformer-based architectures for galaxy images and spectra.
Förderung
- U.S. Department of Energy Office of Science
Zitieren
APA
Parker, L., Lanusse, F., Golkar, S., Sarra, L., Cranmer, M., Bietti, A., Eickenberg, M., Krawezik, G., McCabe, M., Morel, R., Ohana, R., Pettee, M., Blancard, B. R., Cho, K., & Ho, S. (2024). AstroCLIP: a cross-modal foundation model for galaxies. Monthly Notices of the Royal Astronomical Society, 531(4), 4990-5011. https://doi.org/10.1093/mnras/stae1450
Vancouver
Parker L, Lanusse F, Golkar S, Sarra L, Cranmer M, Bietti A, et al. AstroCLIP: a cross-modal foundation model for galaxies. Monthly Notices of the Royal Astronomical Society. 2024;531(4):4990-5011. doi:10.1093/mnras/stae1450
BibTeX
@article{Parker2024,
author = {Parker, Liam and Lanusse, Francois and Golkar, Siavash and Sarra, Leopoldo and Cranmer, Miles and Bietti, Alberto and Eickenberg, Michael and Krawezik, Geraud and McCabe, Michael and Morel, Rudy and Ohana, Ruben and Pettee, Mariel and Blancard, Bruno Régaldo-Saint and Cho, Kyunghyun and Ho, Shirley},
title = {AstroCLIP: a cross-modal foundation model for galaxies},
journal = {Monthly Notices of the Royal Astronomical Society},
year = {2024},
volume = {531},
number = {4},
pages = {4990--5011},
doi = {10.1093/mnras/stae1450}
}
EUVIMED