Robust Perception and Precise Segmentation for Scribble-Supervised RGB-D Saliency Detection

Authors

Long Li, Northwestern Polytechnical University
Junwei Han, Northwestern Polytechnical University
Nian Liu, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Salman Khan, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Hisham Cholakkal, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Rao Muhammad Anwer, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Fahad Shahbaz Khan, Mohamed Bin Zayed University of Artificial IntelligenceFollow

Document Type

Article

Publication Title

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

This paper proposes a scribble-based weakly supervised RGB-D salient object detection (SOD) method to relieve the annotation burden from pixel-wise annotations. In view of the ensuing performance drop, we summarize two natural deficiencies of the scribbles and try to alleviate them, which are the weak richness of the pixel training samples (WRPS) and the poor structural integrity of the salient objects (PSIO). WRPS hinders robust saliency perception learning, which can be alleviated via model design for robust feature learning and pseudo labels generation for training sample enrichment. Specifically, we first design a dynamic searching process module as a meta operation to conduct multi-scale and multi-modal feature fusion for the robust RGB-D SOD model construction. Then, a dual-branch consistency learning mechanism is proposed to generate enough pixel training samples for robust saliency perception learning. PSIO makes direct structural learning infeasible since scribbles can not provide integral structural supervision. Thus, we propose an edge-region structure-refinement loss to recover the structural information and make precise segmentation. We deploy all components and conduct ablation studies on two baselines to validate their effectiveness and generalizability. Experimental results on eight datasets show that our method outperforms other scribble-based SOD models and achieves comparable performance with fully supervised state-of-the-art methods.

First Page

479

Last Page

496

DOI

10.1109/TPAMI.2023.3324807

Publication Date

1-1-2024

Keywords

RGB-D salient object detection, weakly supervised learning

Comments

IR Deposit conditions:

OA version (pathway a) Accepted version

No embargo

When accepted for publication, set statement to accompany deposit (see policy)

Must link to publisher version with DOI

Publisher copyright and source must be acknowledged

Recommended Citation

L. Li et al., "Robust Perception and Precise Segmentation for Scribble-Supervised RGB-D Saliency Detection," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 1, pp. 479-496, Jan. 2024, doi: 10.1109/TPAMI.2023.3324807

Additional Links

https://doi.org/10.1109/TPAMI.2023.3324807