Disentangled Capsule Routing for Fast Part-Object Relational Saliency

Authors

Yi Liu, Changzhou University
Dingwen Zhang, Hefei Comprehensive National Science Center
Nian Liu, Mohamed Bin Zayed University of Artificial IntelligenceFollow
Shoukun Xu, Changzhou University
Jungong Han, Aberystwyth University

Document Type

Article

Publication Title

IEEE Transactions on Image Processing

Abstract

Recently, the Part-Object Relational (POR) saliency underpinned by the Capsule Network (CapsNet) has been demonstrated to be an effective modeling mechanism to improve the saliency detection accuracy. However, it is widely known that the current capsule routing operations have huge computational complexity, which seriously limited the usability of the POR saliency models in real-time applications. To this end, this paper takes an early step towards a fast POR saliency inference by proposing a novel disentangled part-object relational network. Concretely, we disentangle horizontal routing and vertical routing from the original omnidirectional capsule routing, thus generating Disentangled Capsule Routing (DCR). This mechanism enjoys two advantages. On one hand, DCR that disentangles orthogonal 1D (i.e., vertical and horizontal) routing greatly reduces parameters and routing complexity, resulting in much faster inference than omnidirectional 2D routing adopted by existing CapsNets. On the other hand, thanks to the light POR cues explored by DCR, we could conveniently integrate the part-object routing process to different feature layers in CNN, rather than just applying it to the small-scaled one as in previous works. This helps to increase saliency inference accuracy. Compared to previous POR saliency detectors, DPORTNet infers visual saliency (5 ~ 9) × faster, and is more accurate. DPORTNet is available under the open-source license at https://github.com/liuyi1989/DCR.

First Page

6719

Last Page

6732

DOI

10.1109/TIP.2022.3215887

Publication Date

10-25-2022

Keywords

capsule network, disentangled capsule routing, multi-level information integration, part-object relationship, Salient object detection

Comments

IR Deposit conditions:

OA version (pathway a) Accepted version

No embargo

When accepted for publication, set statement to accompany deposit (see policy)

Must link to publisher version with DOI

Publisher copyright and source must be acknowledged

Recommended Citation

Y. Liu, D. Zhang, N. Liu, S. Xu and J. Han, "Disentangled Capsule Routing for Fast Part-Object Relational Saliency," in IEEE Transactions on Image Processing, vol. 31, pp. 6719-6732, 2022, doi: 10.1109/TIP.2022.3215887.