Multi-stage progressive image restoration

Authors

Syed Waqas Zamir, Inception Institute of Artificial Intelligence
Aditya Arora, Inception Institute of Artificial IntelligenceFollow
Salman Khan, Mohamed bin Zayed University of Artificial IntelligenceFollow
Munawar Hayat, Monash University
Fahad Shahbaz Khan, Mohamed bin Zayed University of Artificial IntelligenceFollow
Ming Hsuan Yang, UC Merced
Ling Shao, Inception Institute of Artificial IntelligenceFollow

Document Type

Conference Proceeding

Publication Title

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Abstract

Image restoration tasks demand a complex balance between spatial details and high-level contextualized information while recovering images. In this paper, we propose a novel synergistic design that can optimally balance these competing goals. Our main proposal is a multi-stage architecture, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps. Specifically, our model first learns the contextualized features using encoder-decoder architectures and later combines them with a high-resolution branch that retains local information. At each stage, we introduce a novel per-pixel adaptive design that leverages in-situ supervised attention to reweight the local features. A key ingredient in such a multi-stage architecture is the information exchange between different stages. To this end, we propose a two-faceted approach where the information is not only exchanged sequentially from early to late stages, but lateral connections between feature processing blocks also exist to avoid any loss of information. The resulting tightly interlinked multi-stage architecture, named as MPRNet, delivers strong performance gains on ten datasets across a range of tasks including image deraining, deblurring, and denoising. The source code and pre-trained models are available at https://github.com/swz30/MPRNet.

First Page

14816

Last Page

14826

DOI

10.1109/CVPR46437.2021.01458

Publication Date

1-1-2021

Keywords

image denoising, MPRNet, high-level contextualized information, complex balance, multistage progressive image restoration, image deraining, feature processing blocks, information exchange, local information, high-resolution branch, encoder-decoder architectures, contextualized features, restoration functions, multistage architecture, image deblurring

Comments

IR Deposit conditions: non-described

Open Access version available on CVF:

Recommended Citation

S. W. Zamir et al., "Multi-stage progressive image restoration," in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2021, pp. 14816-14826, doi: 10.1109/CVPR46437.2021.01458.