EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Towards an astronomical foundation model for stars with a transformer-based model

Henry W Leung, Jo Bovy

Monthly Notices of the Royal Astronomical Society · 28.10.2023 · 527 (1) · 1494-1520

DOI: 10.1093/mnras/stad3015 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Monthly Notices of the Royal Astronomical Society, online erschienen am 05.10.2023. Verfasst von Henry W Leung und 1 weiteren Autor:innen. Die Arbeit wurde bisher 46-mal zitiert (Crossref) und nennt 79 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NSERC, Alfred P. Sloan Foundation, U.S. Department of Energy.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Rapid strides are currently being made in the field of artificial intelligence using transformer-based models like Large Language Models (LLMs). The potential of these methods for creating a single, large, versatile model in astronomy has not yet been explored. In this work, we propose a framework for data-driven astronomy that uses the same core techniques and architecture as used by LLMs. Using a variety of observations and labels of stars as an example, we build a transformer-based model and train it in a self-supervised manner with cross-survey data sets to perform a variety of inference tasks. In particular, we demonstrate that a single model can perform both discriminative and generative tasks even if the model was not trained or fine-tuned to do any specific task. For example, on the discriminative task of deriving stellar parameters from Gaia XP spectra, we achieve an accuracy of 47 K in Teff, 0.11 dex in log g, and 0.07 dex in [M/H], outperforming an expert XGBoost model in the same setting. But the same model can also generate XP spectra from stellar parameters, inpaint unobserved spectral regions, extract empirical stellar loci, and even determine the interstellar extinction curve. Our framework demonstrates that building and training a single foundation model without fine-tuning using data and parameters from multiple surveys to predict unmeasured observations and parameters is well within reach. Such ‘Large Astronomy Models’ trained on large quantities of observational data will play a large role in the analysis of current and future large surveys.

Förderung

  • NSERC
  • Alfred P. Sloan Foundation
  • U.S. Department of Energy

Zitieren

APA

Leung, H. W., & Bovy, J. (2023). Towards an astronomical foundation model for stars with a transformer-based model. Monthly Notices of the Royal Astronomical Society, 527(1), 1494-1520. https://doi.org/10.1093/mnras/stad3015

Vancouver

Leung HW, Bovy J. Towards an astronomical foundation model for stars with a transformer-based model. Monthly Notices of the Royal Astronomical Society. 2023;527(1):1494-1520. doi:10.1093/mnras/stad3015

BibTeX
@article{Leung2023,
  author = {Leung, Henry W and Bovy, Jo},
  title = {Towards an astronomical foundation model for stars with a transformer-based model},
  journal = {Monthly Notices of the Royal Astronomical Society},
  year = {2023},
  volume = {527},
  number = {1},
  pages = {1494--1520},
  doi = {10.1093/mnras/stad3015}
}
Kostenlos starten →