Soft Conflict-Resolution Decision Transformer for Offline Multi-Task Reinforcement Learning
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (31) · 26462-26470
DOI: 10.1609/aaai.v40i31.39853 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Shudong Wang und 6 weiteren Autor:innen.
Abstract (Originalsprache)
Multi-task reinforcement learning (MTRL) seeks to learn a unified policy for diverse tasks, but often suffers from gradient conflicts across tasks. Existing masking-based methods attempt to mitigate such conflicts by assigning task-specific parameter masks. However, our empirical study shows that coarse-grained binary masks have the problem of over-suppressing key conflicting parameters, hindering knowledge sharing across tasks. Moreover, different tasks exhibit varying conflict levels, yet existing methods use a one-size-fits-all fixed sparsity strategy to keep training stability and performance, which proves inadequate. These limitations hinder the model’s generalization and learning efficiency. To address these issues, we propose SoCo-DT, a Soft Conflict-resolution method based by parameter importance. By leveraging Fisher information, mask values are dynamically adjusted to retain important parameters while suppressing conflicting ones. In addition, we introduce a dynamic sparsity adjustment strategy based on the Interquartile Range (IQR), which constructs task-specific thresholding schemes using the distribution of conflict and harmony scores during training. To enable adaptive sparsity evolution throughout training, we further incorporate an asymmetric cosine annealing schedule to continuously update the threshold. Experimental results on the Meta-World benchmark show that SoCo-DT outperforms the state-of-the-art method by 7.6% on MT50 and by 10.5% on the suboptimal dataset, demonstrating its effectiveness in mitigating gradient conflicts and improving overall multi-task performance.
Zitieren
APA
Wang, S., Wang, X., Zhang, C., Pang, S., Gui, H., Ji, W., & Liao, X. (2026). Soft Conflict-Resolution Decision Transformer for Offline Multi-Task Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(31), 26462-26470. https://doi.org/10.1609/aaai.v40i31.39853
Vancouver
Wang S, Wang X, Zhang C, Pang S, Gui H, Ji W, et al. Soft Conflict-Resolution Decision Transformer for Offline Multi-Task Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(31):26462-26470. doi:10.1609/aaai.v40i31.39853
BibTeX
@article{Wang2026,
author = {Wang, Shudong and Wang, Xinfei and Zhang, Chenhao and Pang, Shanchen and Gui, Haiyuan and Ji, Wenhao and Liao, Xiaojian},
title = {Soft Conflict-Resolution Decision Transformer for Offline Multi-Task Reinforcement Learning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {31},
pages = {26462--26470},
doi = {10.1609/aaai.v40i31.39853}
}
EUVIMED