We are very excited to join forces with MLCommons and OctoML.ai! Contact Grigori Fursin for more details!

Activity Regularization for Continual Learning

lib:2f998c0ca12c9e09 (v1.0.0)

Authors: Quang H. Pham,Steven C. H. Hoi
Where published: ICLR 2019 5
Document:  PDF  DOI 
Abstract URL: https://openreview.net/forum?id=S1G_cj05YQ

While deep neural networks have achieved remarkable successes, they suffer the well-known catastrophic forgetting issue when switching from existing tasks to tackle a new one. In this paper, we study continual learning with deep neural networks that learn from tasks arriving sequentially. We first propose an approximated multi-task learning framework that unifies a family of popular regularization based continual learning methods. We then analyze the weakness of existing approaches, and propose a novel regularization method named “Activity Regularization” (AR), which alleviates forgetting meanwhile keeping model’s plasticity to acquire new knowledge. Extensive experiments show that our method outperform state-of-the-art methods and effectively overcomes catastrophic forgetting.

Relevant initiatives  

Related knowledge about this paper Reproduced results (crowd-benchmarking and competitions) Artifact and reproducibility checklists Common formats for research projects and shared artifacts Reproducibility initiatives


Please log in to add your comments!
If you notice any inapropriate content that should not be here, please report us as soon as possible and we will try to remove it within 48 hours!