Anomaly detection in video via self-supervised and multi-task learning

Authors

Mariana Iuliana Georgescu, Universitatea din Bucuresti
Antonio Barbalau, Universitatea din Bucuresti
Radu Tudor Ionescu, Universitatea din Bucuresti
Fahad Shahbaz Khan, Mohamed bin Zayed University of Artificial IntelligenceFollow
Marius Popescu, Universitatea din Bucuresti
Mubarak Shah, University of Central Florida

Document Type

Conference Proceeding

Publication Title

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Abstract

Anomaly detection in video is a challenging computer vision problem. Due to the lack of anomalous events at training time, anomaly detection requires the design of learning methods without full supervision. In this paper, we approach anomalous event detection in video through self-supervised and multi-task learning at the object level. We first utilize a pre-trained detector to detect objects. Then, we train a 3D convolutional neural network to produce discriminative anomaly-specific information by jointly learning multiple proxy tasks: three self-supervised and one based on knowledge distillation. The self-supervised tasks are: (i) discrimination of forward/backward moving objects (arrow of time), (ii) discrimination of objects in consecutive/intermittent frames (motion irregularity) and (iii) reconstruction of object-specific appearance information. The knowledge distillation task takes into account both classification and detection information, generating large prediction discrepancies between teacher and student models when anomalies occur. To the best of our knowledge, we are the first to approach anomalous event detection in video as a multi-task learning problem, integrating multiple self-supervised and knowledge distillation proxy tasks in a single architecture. Our lightweight architecture outperforms the state-of-the-art methods on three benchmarks: Avenue, ShanghaiTech and UCSD Ped2. Additionally, we perform an ablation study demonstrating the importance of integrating self-supervised learning and normality-specific distillation in a multi-task learning setting.

First Page

12737

Last Page

12747

DOI

10.1109/CVPR46437.2021.01255

Publication Date

1-1-2021

Keywords

training time, anomaly detection, learning methods, anomalous event detection, object level, 3D convolutional neural network, discriminative anomaly-specific information, multiple proxy tasks, object-specific appearance information, detection information, multitask learning problem, knowledge distillation proxy tasks, self-supervised learning, computer vision problem, anomalous events, Avenue benchmark, ShanghaiTech benchmark, UCSD Ped2 benchmark

Comments

IR Deposit conditions: non-described

Open Access version available on CVF:

Recommended Citation

M.I. Georgescu, A. Bărbălău, R.T. Ionescu, F.S. Khan, M. Popescu and M. Shah, "Anomaly detection in video via self-supervised and multi-task learning," in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2021, pp. 12737-12747, doi: 10.1109/CVPR46437.2021.01255.