Task-Wise Winner Taking Continual Learning
Neural Computation · 05.11.2026 · 1-39
DOI: 10.1162/neco.a.1600 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neural Computation, online erschienen am 05.11.2026. Verfasst von Juhyeon Kim und 3 weiteren Autor:innen.
Abstract (Originalsprache)
Recent research on artificial intelligence systems has been exploring ways to enable models to continuously learn, adapt, and evolve to changing tasks instead of being limited to one task. For this purpose, continual learning has been actively studied, which enables the model to learn many tasks. A major concern of continual learning is to overcome catastrophic forgetting, the phenomenon in which a model forgets tasks it has previously learned. Diverse approaches have been attempted to solve this difficulty: adding some samples from previous tasks, adding nodes or layers for incoming tasks, or regularizing the learning on the new task to preserve the optimal parameters of the previous task. The last approach maintains the initial structure of the model and does not necessarily require data sampling or structural expansion of the model. This provides advantages over issues such as data storage and management problems, increased model complexity and the related problems. However, there are some inevitable difficulties because the number of parameters is limited. First, there is still catastrophic forgetting due to parameter updates or overwrites as one task proceeds to another. Second, the performance of previous tasks inevitably degrades as the model continues to learn and adapt to new tasks. Third, given a set of tasks, if the model parameters are not well distributed across tasks, it cannot learn about the new tasks. To overcome these difficulties, we propose a method named as task-wise winner-taking continual learning (TWC) which improves the existing representative regularization method, elastic weight consolidation (EWC). TWC regularizes parameters task-wisely, recovers degeneration of performance, and efficiently distributes parameters to tasks. In this letter, we present experimental results showing that TWC performs better compared to the up-to-date methods in various situations. The highlights of the letter are that we use Fisher information in a task-wise manner to ensure performance for multiple tasks, recover performance of previously trained tasks with a recovering factor, and use winner-taking regularization to better arrange weights across multiple tasks.
Zitieren
APA
Kim, J., Park, J., Kim, J., & Shin, H. (2026). Task-Wise Winner Taking Continual Learning. Neural Computation, 1-39. https://doi.org/10.1162/neco.a.1600
Vancouver
Kim J, Park J, Kim J, Shin H. Task-Wise Winner Taking Continual Learning. Neural Computation. 2026:1-39. doi:10.1162/neco.a.1600
BibTeX
@article{Kim2026,
author = {Kim, Juhyeon and Park, Jinju and Kim, Jae-Hoon and Shin, Hyunjung},
title = {Task-Wise Winner Taking Continual Learning},
journal = {Neural Computation},
year = {2026},
pages = {1--39},
doi = {10.1162/neco.a.1600}
}
EUVIMED