Towards partial supervision for generic object counting in natural scenes

Authors

Hisham Cholakkal, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Guolei Sun, Mohamed Bin Zayed University of Artificial Intelligence
Salman Khan, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Fahad Shahbaz Khan, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Ling Shao, ETH Zürich & Inception Institute of Artificial IntelligenceFollow
Luc Van Gool, Mohamed Bin Zayed University of Artificial Intelligence

Document Type

Article

Publication Title

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

Generic object counting in natural scenes is a challenging computer vision problem. Existing approaches either rely on instance-level supervision or absolute count information to train a generic object counter. We introduce a partially supervised setting that significantly reduces the supervision level required for generic object counting. We propose two novel frameworks, named lower-count (LC) and reduced lower-count (RLC), to enable object counting under this setting. Our frameworks are built on a novel dual-branch architecture that has an image classification and a density branch. Our LC framework reduces the annotation cost due to multiple instances in an image by using only lower-count supervision for all object categories. Our RLC framework further reduces the annotation cost arising from large numbers of object categories in a dataset by only using lower-count supervision for a subset of categories and class-labels for the remaining ones. The RLC framework extends our dual-branch LC framework with a novel weight modulation layer and a category-independent density map prediction. Experiments are performed on COCO, Visual Genome and PASCAL 2007 datasets. Our frameworks perform on par with state-of-the-art approaches using higher levels of supervision. Additionally, we demonstrate the applicability of our LC supervised density map for image-level supervised instance segmentation.

First Page

1604

Last Page

1622

DOI

10.1109/TPAMI.2020.3021025

Publication Date

3-1-2022

Keywords

Generic object counting, object localization, reduced supervision, weakly supervised instance segmentation

Comments

IR deposit conditions:

• OA version (pathway a)
• No embargo
• When accepted for publication, set statement to accompany deposit (see policy)
• Must link to publisher version with DOI

Recommended Citation

H. Cholakkal, G. Sun, S. Khan, F. S. Khan, L. Shao and L. Van Gool, "Towards partial supervision for generic object counting in natural scenes," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 3, pp. 1604-1622, 1 March 2022, doi: 10.1109/TPAMI.2020.3021025.