: Edge-Aware Multimodal Transformer for RGB-D Salient Object Detection
Document Type
Article
Publication Title
IEEE Transactions on Neural Networks and Learning Systems
Abstract
RGB-D salient object detection (SOD) has gained tremendous attention in recent years. In particular, transformer has been employed and shown great potential. However, existing transformer models usually overlook the vital edge information, which is a major issue restricting the further improvement of SOD accuracy. To this end, we propose a novel edge-aware RGB-D SOD transformer, called, which explicitly models the edge information in a dual-band decomposition framework. Specifically, we employ two parallel decoder networks to learn the high-frequency edge and low-frequency body features from the low-and high-level features extracted from a two-steam multimodal backbone network, respectively. Next, we propose a cross-attention complementarity exploration module to enrich the edge/body features by exploiting the multimodal complementarity information. The refined features are then fed into our proposed color-hint guided fusion module for enhancing the depth feature and fusing the multimodal features. Finally, the resulting features are fused using our deeply supervised progressive fusion module, which progressively integrates edge and body features for predicting saliency maps. Our model explicitly considers the edge information for accurate RGB-D SOD, overcoming the limitations of existing methods and effectively improving the performance. Extensive experiments on benchmark datasets demonstrate that is an effective RGB-D SOD framework that outperforms the current state-of-the-art models, both quantitatively and qualitatively. A further extension to RGB-T SOD demonstrates the promising potential of our model in various kinds of multimodal SOD tasks.
DOI
10.1109/TNNLS.2024.3358858
Publication Date
1-1-2024
Keywords
Computational modeling, Decoding, Edge-aware model, Feature extraction, Image edge detection, multimodal learning, Object detection, salient object detection (SOD), Task analysis, transformer, Transformers
Recommended Citation
G. Chen et al., ": Edge-Aware Multimodal Transformer for RGB-D Salient Object Detection," IEEE Transactions on Neural Networks and Learning Systems, Jan 2024.
The definitive version is available at https://doi.org/10.1109/TNNLS.2024.3358858