Adaptive Hard Parameter Sharing Method Based on Multi-Task Deep Learning
Mathematics · 14.11.2023 · 11 (22) · 4639
DOI: 10.3390/math11224639 ↗Kurz gefasst
Fachartikel in der Zeitschrift Mathematics, online erschienen am 14.11.2023. Verfasst von Hongxia Wang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 16 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the National Social Science Fund of China, the Jiangsu Province Self Science, “Qinglan project” of Colleges and Universities of Jiangsu Province.
Abstract (Originalsprache)
Multi-task learning (MTL) improves the performance achieved on each task by exploiting the relevant information between tasks. At present, most of the mainstream deep MTL models are based on hard parameter sharing mechanisms, which can reduce the risk of model overfitting. However, negative knowledge transfer may occur, which hinders the performance improvement achieved for each task. In this paper, for situations when multiple tasks are jointly trained, we propose the adaptive hard parameter sharing method. On the basis of the adaptive hard parameter sharing method, the number of nodes in the network is dynamically updated by setting a continuous gradient difference-based sign threshold and a warm-up training iteration threshold through the relationships between the parameters and the loss function. After each task fully utilizes the shared information, adaptive nodes are used to further optimize each task, reducing the impact of negative migration. By using simulation studies and instance analyses, we demonstrate theoretical proof that the performance of the proposed method is better than that of the competing method.
Förderung
- the National Social Science Fund of China
- the Jiangsu Province Self Science
- “Qinglan project” of Colleges and Universities of Jiangsu Province
- Postgraduate Research and Practice Innovation Program of Jiangsu Province
Zitieren
APA
Wang, H., Jin, X., Du, Y., Zhang, N., & Hao, H. (2023). Adaptive Hard Parameter Sharing Method Based on Multi-Task Deep Learning. Mathematics, 11(22), 4639. https://doi.org/10.3390/math11224639
Vancouver
Wang H, Jin X, Du Y, Zhang N, Hao H. Adaptive Hard Parameter Sharing Method Based on Multi-Task Deep Learning. Mathematics. 2023;11(22):4639. doi:10.3390/math11224639
BibTeX
@article{Wang2023,
author = {Wang, Hongxia and Jin, Xiao and Du, Yukun and Zhang, Nan and Hao, Hongxia},
title = {Adaptive Hard Parameter Sharing Method Based on Multi-Task Deep Learning},
journal = {Mathematics},
year = {2023},
volume = {11},
number = {22},
pages = {4639},
doi = {10.3390/math11224639}
}
EUVIMED