EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

HeteroGrad-DT: Gradient Flow Transformer for heterogeneous multi-task offline reinforcement learning

Ye Wang, Donghui Ma, Jiang Li, Lihong Guo iD

PLOS One · 11.11.2026 · 21 (11) · e0362298

DOI: 10.1371/journal.pone.0362298 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, online erschienen am 11.11.2026. Verfasst von Ye Wang und 3 weiteren Autor:innen. Beteiligte Einrichtung: Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin, China. Hauptthemen laut MEDLINE: Reinforcement Machine Learning, Robotics. Die Arbeit nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Offline reinforcement learning enables policy learning from static datasets without online exploration, making it particularly suitable for robotic control under safety and data constraints. Decision Transformer recasts offline reinforcement learning as conditional sequence modeling. Extending it to heterogeneous multi-task settings is challenging because task representations and dynamics differ, while joint training can suffer from destructive gradient interference. To address these challenges, we present HeteroGrad-DT, an integrated framework that employs task-specific tokenizers and decoupled prediction heads to retain task-native input and output dimensionality at the model interfaces while learning a shared sequence backbone. It further introduces a Gradient Flow Transformer that periodically coordinates cross-task updates using fixed-dimensional task statistics, compact summaries of shared-backbone gradients accumulated within the current synchronization interval, and an alignment-based coordination objective, rather than relying only on stepwise projection or static regularization. Experiments on the DeepMind Control Suite with up to 30 tasks show that HeteroGrad-DT improves average normalized return over representative multi-task methods. In the 30-task setting, it achieves a 32.3% relative improvement over standard training with a shared backbone and an 8.6% gain over a representative gradient-conflict mitigation baseline using the same task-aware interfaces. Ablation, mechanism, and sensitivity analyses further support complementary benefits from task-aware interfaces and gradient coordination in the evaluated settings.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Reinforcement Machine LearningRoboticsAlgorithmsHumans

Autor:innen und Institutionen

  • Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun, Jilin, China.
  • University of Chinese Academy of Sciences, Beijing, China.

ORCID-Profile: Lihong Guo

Zitieren

APA

Wang, Y., Ma, D., Li, J., & Guo, L. (2026). HeteroGrad-DT: Gradient Flow Transformer for heterogeneous multi-task offline reinforcement learning. PLOS One, 21(11), e0362298. https://doi.org/10.1371/journal.pone.0362298

Vancouver

Wang Y, Ma D, Li J, Guo L. HeteroGrad-DT: Gradient Flow Transformer for heterogeneous multi-task offline reinforcement learning. PLOS One. 2026;21(11):e0362298. doi:10.1371/journal.pone.0362298

BibTeX
@article{Wang2026,
  author = {Wang, Ye and Ma, Donghui and Li, Jiang and Guo, Lihong},
  title = {HeteroGrad-DT: Gradient Flow Transformer for heterogeneous multi-task offline reinforcement learning},
  journal = {PLOS One},
  year = {2026},
  volume = {21},
  number = {11},
  pages = {e0362298},
  doi = {10.1371/journal.pone.0362298}
}
Kostenlos starten →