Fixing localization errors to improve image classification

Authors

Guolei Sun, ETH Zürich
Salman Khan, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Wen Li, University of Electronic Science and Technology of China
Hisham Cholakkal, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Fahad Shahbaz Khan, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Luc Van Gool, ETH Zürich

Document Type

Conference Proceeding

Publication Title

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Abstract

Deep neural networks are generally considered black-box models that offer less interpretability for their decision process. To address this limitation, Class Activation Map (CAM) provides an attractive solution that visualizes class-specific discriminative regions in an input image. The remarkable ability of CAMs to locate class discriminating regions has been exploited in weakly-supervised segmentation and localization tasks. In this work, we explore a new direction towards the possible use of CAM in deep network learning process. We note that such visualizations lend insights into the workings of deep CNNs and could be leveraged to introduce additional constraints during the learning stage. Specifically, the CAMs for negative classes (negative CAMs) often have false activations even though those classes are absent from an image. Thereby, we propose a loss function that seeks to minimize peaks within the negative CAMs, called ‘Homogeneous Negative CAM’ loss. This way, in an effort to fix localization errors, our loss provides an extra supervisory signal that helps the model to better discriminate between similar classes. Our designed loss function is easy to implement and can be readily integrated into existing DNNs. We evaluate it on a number of classification tasks including large-scale recognition, multi-label classification and fine-grained recognition. Our loss provides better performance compared to other loss functions across the studied tasks. Additionally, we show that the proposed loss function provides higher robustness against adversarial attacks and noisy labels.

First Page

271

Last Page

287

DOI

10.1007/978-3-030-58595-2_17

Publication Date

11-20-2020

Keywords

neural networks, Class Activation Map

Comments

IR deposit conditions:

• OA version (pathway a)
• Accepted version
• 12 month embargo
• Must link to published article
• Set statement to accompany deposit

Recommended Citation

G. Sun, S. Khan, W. Li, H. Cholakkal, F. S. Khan, and L. van Gool, “Fixing localization errors to improve image classification,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Aug. 2020, vol. 12370 LNCS, pp. 271–287. doi: 10.1007/978-3-030-58595-2_17.