Burst image restoration and enhancement

Authors

Akshay Dudhane, Mohamed bin Zayed University of Artificial IntelligenceFollow
Syed Waqas Zamir, Inception Institute of Artificial Intelligence
Salman Khan, Mohamed bin Zayed University of Artificial Intelligence & Australian National UniversityFollow
Fahad Shahbaz Khan, Mohamed bin Zayed University of Artificial Intelligence & Linköping University, SwedenFollow
Ming-Hsuan Yang, University of California & Yonsei University & Google Research

Document Type

Conference Proceeding

Publication Title

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition

Abstract

Modern handheld devices can acquire burst image sequence in a quick succession. However, the individual acquired frames suffer from multiple degradations and are misaligned due to camera shake and object motions. The goal of Burst Image Restoration is to effectively combine complimentary cues across multiple burst frames to generate high-quality outputs. Towards this goal, we develop a novel approach by solely focusing on the effective information exchange between burst frames, such that the degradations get filtered out while the actual scene details are preserved and enhanced. Our central idea is to create a set of pseudo-burst features that combine complimentary information from all the input burst frames to seamlessly exchange information. However, the pseudo-burst cannot be successfully created unless the individual burst frames are properly aligned to discount inter-frame movements. Therefore, our approach initially extracts pre-processed features from each burst frame and matches them using an edge-boosting burst alignment module. The pseudo-burst features are then created and enriched using multi-scale contextual information. Our final step is to adaptively aggregate information from the pseudo-burst features to progressively increase resolution in multiple stages while merging the pseudo-burst features. In comparison to existing works that usually follow a late fusion scheme with single-stage upsampling, our approach performs favorably, delivering state-of-the-art performance on burst super-resolution, burst low-light image enhancement and burst denoising tasks. The source code and pre-trained models are available at https://github.com/akshaydudhane16/BIPNet. © 2022 IEEE.

First Page

5749

Last Page

5758

DOI

10.1109/CVPR52688.2022.00567

Publication Date

9-27-2022

Keywords

Low-level vision, Scene analysis and understanding

Comments

Open Access version, provided by Computer Vision Foundation.

Recommended Citation

A. Dudhane, S. W. Zamir, S. Khan, F. S. Khan and M. -H. Yang, "Burst Image Restoration and Enhancement," 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 2022, pp. 5749-5758, doi: 10.1109/CVPR52688.2022.00567.