Class Incremental Continual Learning for Computer Vision

Authors

Rushali Grandhe, Mohamed bin Zayed University of Artificial IntelligenceFollow

Document Type

Dissertation

Abstract

Humans have the ability to learn continually throughout their lives. Continual learning, a subset of machine learning, attempts to replicate humans' lifelong learning ability in neural networks. This work investigates the class incremental form of continual learning for various vision-based applications such as image classification and object detection. Each learning phase in a class incremental learning (CIL) scenario introduces groups of classes to a model, where the goal is to learn a unified model that is performant across all classes seen thus far. Performance of different architectures such as Convolutional Neural Networks (CNNs) and Vision Transformers (ViT) is studied when used in a continual learning setting. A hybrid ViT is suitably adapted for continual learning by using an interpretability based distillation mechanism which maintains the configuration of spatial attention maps as learning progresses. Eventually this aids in reducing catastrophic forgetting by forcing the model to focus on the most discriminative regions in an image. When combined with other methods such as logit adjustments to combat bias, dubbed as D3Former, it shows considerable improvements across many datasets. Furthermore, the inherent ability of ViTs to process images as patches is explored in order to reduce exemplar memory by storing relevant patches instead of images. Besides, continual learning with CNNs is studied from the perspective of utilizing acquired knowledge in order to provide a better initialization for learning new knowledge. Apart from image classification, continual learning for object detection is explored in an open-world setting where unknown classes are present in the data. The continual learning behaviour is evaluated by improving the unknown identifiable property using post-processing and proposal sampling strategies.

First Page

i

Last Page

56

Publication Date

1-12-2022

Comments

Thesis submitted to the Deanship of Graduate and Postdoctoral Studies

In partial fulfillment of the requirements for the M.Sc degree in Computer Vision

Advisors: Dr. Salman Khan, Dr. Fahad Khan

Online access for MBZUAI patrons

Recommended Citation

R. Grandhe, "Class Incremental Continual Learning for Computer Vision", M.S. Thesis, Computer Vision, MBZUAI, Abu Dhabi, UAE, 2022.