Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection

Authors

Long Li, Northwestern Polytechnical University
Junwei Han, Northwestern Polytechnical University
Ni Zhang, Northwestern Polytechnical University
Nian Liu, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Salman Khan, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Hisham Cholakkal, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Rao Muhammad Anwer, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Fahad Shahbaz Khan, Mohamed Bin Zayed University of Artificial IntelligenceFollow

Document Type

Conference Proceeding

Publication Title

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Abstract

Most previous co-salient object detection works mainly focus on extracting co-salient cues via mining the consistency relations across images while ignore explicit exploration of background regions. In this paper, we propose a Discriminative co-saliency and background Mining Transformer framework (DMT) based on several economical multi-grained correlation modules to explicitly mine both co-saliency and background information and effectively model their discrimination. Specifically, we first propose a region-to-region correlation module for introducing inter-image relations to pixel-wise segmentation features while maintaining computational efficiency. Then, we use two types of pre-defined tokens to mine co-saliency and background information via our proposed contrast-induced pixel-to-token correlation and co-saliency token-to-token correlation modules. We also design a token-guided feature refinement module to enhance the discriminability of the segmentation features under the guidance of the learned tokens. We perform iterative mutual promotion for the segmentation feature extraction and token construction. Experimental results on three benchmark datasets demonstrate the effectiveness of our proposed method. The source code is available at: https://github.com/dragonlee258079/DMT.

First Page

7247

Last Page

7256

DOI

10.1109/CVPR52729.2023.00700

Publication Date

8-22-2023

Keywords

Low-level vision, Economics, Computer vision, Correlation, Computational modeling, Source coding, Object detection, Transformers, Object Detection, Co-salient Objects, Benchmark Datasets, Discriminative Features, Background Regions, Segmentation Feature, Image Features, Feature Maps, Function Prediction, Spatial Dimensions, Attention Mechanism, Background Characteristics, Matrix Multiplication, Semantic Segmentation, Explicit Model, Vanilla, Attention Map, Channel Attention, Feature Enhancement, Decoder Layer, Salient Object, Salient Object Detection, Informant Consensus, Decoder Features, Transformation Operations, Distillation Process, Segmentation Results, Background Model, Characteristics Of Groups, Element-wise Multiplication

Comments

Open Access version by CVF

Uploaded: 04 June 2024

Recommended Citation

L. Li et al., "Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection," 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 2023, pp. 7247-7256, doi: 10.1109/CVPR52729.2023.00700.

Additional Links

https://doi.org/10.1109/CVPR52729.2023.00700