EUVIMEDEUVIMED entdecken
Crossref

Structure Detection for Contextual Reinforcement Learning

Tianyue Zhou, Jung-Hoon Cho, Cathy Wu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (34) · 29009-29016

DOI: 10.1609/aaai.v40i34.40137 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Tianyue Zhou und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Contextual Reinforcement Learning (CRL) tackles the problem of solving a set of related Contextual Markov Decision Processes (CMDPs) that vary across different context variables. Traditional approaches---independent training and multi-task learning---struggle with either excessive computational costs or negative transfer. A recently proposed multi-policy approach, Model-Based Transfer Learning (MBTL), has demonstrated effectiveness by strategically selecting a few tasks to train and zero-shot transfer. However, CMDPs encompass a wide range of problems, exhibiting structural properties that vary from problem to problem. As such, different task selection strategies are suitable for different CMDPs. In this work, we introduce Structure Detection MBTL (SD-MBTL), a generic framework that dynamically identifies the underlying generalization structure of CMDP and selects an appropriate MBTL algorithm. For instance, we observe Mountain structure in which generalization performance degrades from the training performance of the target task as the context difference increases. We thus propose M/GP-MBTL, which detects the structure and adaptively switches between a Gaussian Process-based approach and a clustering-based approach. Extensive experiments on synthetic data and CRL benchmarks—covering continuous control, traffic control, and agricultural management—show that M/GP-MBTL surpasses the strongest prior method by 12.49% on the aggregated metric. These results highlight the promise of online structure detection for guiding source task selection in complex CRL environments.

Zitieren

APA

Zhou, T., Cho, J., & Wu, C. (2026). Structure Detection for Contextual Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(34), 29009-29016. https://doi.org/10.1609/aaai.v40i34.40137

Vancouver

Zhou T, Cho J, Wu C. Structure Detection for Contextual Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(34):29009-29016. doi:10.1609/aaai.v40i34.40137

BibTeX
@article{Zhou2026,
  author = {Zhou, Tianyue and Cho, Jung-Hoon and Wu, Cathy},
  title = {Structure Detection for Contextual Reinforcement Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {34},
  pages = {29009--29016},
  doi = {10.1609/aaai.v40i34.40137}
}
Kostenlos starten →