Learning Policies for Autonomous Earth-Observing Satellite Scheduling over Semi-Markov Decision Processes
Journal of Aerospace Information Systems · 2025-09 · 22 (9) · 789-799
DOI: 10.2514/1.i011649 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Aerospace Information Systems, 2025-09. Verfasst von Mark Andrew Stephenson und 2 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 28 Literaturangaben. Gefördert von: NASA Headquarters, Air Force Research Laboratory.
Abstract (Originalsprache)
Advances in request scheduling algorithms are necessary to fully utilize the capabilities of agile Earth-observing satellites. While typical current approaches preplan sequences of image collections for a static set of requests, onboard autonomous methods that select requests to fulfill on-the-fly can adapt to opportunistic events and reduce the burden on operators. Prior approaches that utilized reinforcement learning for autonomy considered fixed-duration decision intervals, which limited performance over high-density requests and oversimplified mission operations. In this paper, reinforcement learning is used to derive policies that make decisions at variable intervals to improve performance by scheduling requests in a mission-like manner. The resulting policies perform competitively against optimal global schedules in a flight-like evaluation environment. This is achieved by training on a new semi-Markov decision process (sMDP) formulation of the problem, leveraging step-duration information when learning. Ablation studies over new sMDP-specific training algorithm modifications and observation spaces are performed to learn a highly performant policy. Benefits of the method, including the implicit ability to account for resource constraints, are demonstrated. Ultimately, variable-decision-interval reinforcement-learning-based policies are established as a viable method for autonomous Earth-observing satellite scheduling under mission-realistic conditions, providing a closed-loop and computationally inexpensive alternative to traditional preplanning methods.
Förderung
- NASA Headquarters
- Air Force Research Laboratory
Zitieren
APA
Stephenson, M. A., Mantovani, L. Q., & Schaub, H. (2025). Learning Policies for Autonomous Earth-Observing Satellite Scheduling over Semi-Markov Decision Processes. Journal of Aerospace Information Systems, 22(9), 789-799. https://doi.org/10.2514/1.i011649
Vancouver
Stephenson MA, Mantovani LQ, Schaub H. Learning Policies for Autonomous Earth-Observing Satellite Scheduling over Semi-Markov Decision Processes. Journal of Aerospace Information Systems. 2025;22(9):789-799. doi:10.2514/1.i011649
BibTeX
@article{Stephenson2025,
author = {Stephenson, Mark Andrew and Mantovani, Lorenzzo Quevedo and Schaub, Hanspeter},
title = {Learning Policies for Autonomous Earth-Observing Satellite Scheduling over Semi-Markov Decision Processes},
journal = {Journal of Aerospace Information Systems},
year = {2025},
volume = {22},
number = {9},
pages = {789--799},
doi = {10.2514/1.i011649}
}
EUVIMED