Instance-Level Relative Saliency Ranking With Graph Reasoning

Authors

Nian Liu, Inception Institute of Artificial Intelligence
Long Li, Northwestern Polytechnical University
Wangbo Zhao, Northwestern Polytechnical University
Junwei Han, Northwestern Polytechnical University
Ling Shao, Inception Institute of Artificial Intelligence & Mohamed bin Zayed University of Artificial Intelligence

Document Type

Article

Publication Title

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

Conventional salient object detection models cannot differentiate the importance of different salient objects. Recently, two works have been proposed to detect saliency ranking by assigning different degrees of saliency to different objects. However, one of these models cannot differentiate object instances and the other focuses more on sequential attention shift order inference. In this paper, we investigate a practical problem setting that requires simultaneously segment salient instances and infer their relative saliency rank order. We present a novel unified model as the first end-to-end solution, where an improved Mask R-CNN is first used to segment salient instances and a saliency ranking branch is then added to infer the relative saliency. For relative saliency ranking, we build a new graph reasoning module by combining four graphs to incorporate the instance interaction relation, local contrast, global contrast, and a high-level semantic prior, respectively. A novel loss function is also proposed to effectively train the saliency ranking branch. Besides, a new dataset and an evaluation metric are proposed for this task, aiming at pushing forward this field of research. Finally, experimental results demonstrate that our proposed model is more effective than previous methods. We also show an example of its practical usage on adaptive image retargeting.

First Page

8321

Last Page

8337

DOI

10.1109/TPAMI.2021.3107872

Publication Date

8-26-2021

Keywords

global context, graph neural network, image retargeting, instance segmentation, local context, Saliency detection

Comments

IR Deposit conditions:

OA version (pathway a) Accepted version

No embargo

When accepted for publication, set statement to accompany deposit (see policy)

Must link to publisher version with DOI

Publisher copyright and source must be acknowledged

Recommended Citation

N. Liu, L. Li, W. Zhao, J. Han and L. Shao, "Instance-Level Relative Saliency Ranking With Graph Reasoning," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 11, pp. 8321-8337, 1 Nov. 2022, doi: 10.1109/TPAMI.2021.3107872.