OW-DETR: Open-world detection transformer

Authors

Akshita Gupta, Inception Institute of Artificial IntelligenceFollow
Sanath Narayan, Inception Institute of Artificial Intelligence
K.J. Joseph, GoogleFollow
Salman Khan, Australian National University & Mohamed Bin Zayed University of Artificial IntelligenceFollow
Fahad Shahbaz Khan, Mohamed Bin Zayed University of Artificial Intelligence & Linköping UniversityFollow
Mubarak Shah, University of Central FloridaFollow

Document Type

Article

Publication Title

arXiv

Abstract

Open-world object detection (OWOD) is a challenging computer vision problem, where the task is to detect a known set of object categories while simultaneously identifying unknown objects. Additionally, the model must incrementally learn new classes that become known in the next training episodes. Distinct from standard object detection, the OWOD setting poses significant challenges for generating quality candidate proposals on potentially unknown objects, separating the unknown objects from the background and detecting diverse unknown objects. Here, we introduce a novel end-to-end transformer-based framework, OW-DETR, for open-world object detection. The proposed OW-DETR comprises three dedicated components namely, attention-driven pseudo-labeling, novelty classification and objectness scoring to explicitly address the aforementioned OWOD challenges. Our OW-DETR explicitly encodes multi-scale contextual information, possesses less inductive bias, enables knowledge transfer from known classes to the unknown class and can better discriminate between unknown objects and background. Comprehensive experiments are performed on two benchmarks: MS-COCO and PASCAL VOC. The extensive ablations reveal the merits of our proposed contributions. Further, our model outperforms the recently introduced OWOD approach, ORE, with absolute gains ranging from 1.8% to 3.3% in terms of unknown recall on the MS-COCO benchmark. In the case of incremental object detection, OW-DETR outperforms the state-of-the-art for all settings on the PASCAL VOC benchmark. Our codes and models will be publicly released. © 2021, CC BY-NC-SA.

DOI

doi.org/10.48550/arXiv.2112.01513

Publication Date

12-2-2021

Keywords

Computer vision, Knowledge management, Object detection, Computer vision problems, Contextual information, End to end, Inductive bias, Labelings, Learn+, Multi-scales, Object categories, Open world, Unknown objects, Object recognition, Computer Vision and Pattern Recognition (cs.CV)

Comments

Preprint: arXiv

Archived with thanks to arXiv

Preprint License: CC BY-NC-SA 4.0

Uploaded 24 March 2022

Recommended Citation

A. Gupta, S. Narayan, K.J. Joseph, S. Khan, F.S. Khan, and M. Shah, "OW-DETR: Open-world detection transformer", 2021, arXiv:2112.01513