Learning digital camera pipeline for extreme low-light imaging

Authors

Syed Waqas Zamir, Inception Institute of Artificial Intelligence
Aditya Arora, Inception Institute of Artificial IntelligenceFollow
Salman Khan, Mohamad bin Zayed University of Artificial IntelligenceFollow
Fahad Shahbaz Khan, Mohamad bin Zayed University of Artificial IntelligenceFollow
Ling Shao, Inception Institute of Artificial IntelligenceFollow

Document Type

Article

Publication Title

Neurocomputing

Abstract

In low-light conditions, a conventional camera imaging pipeline produces sub-optimal images that are usually dark and noisy due to a low photon count and low signal-to-noise ratio (SNR). We present a data-driven approach that learns the desired properties of well-exposed images and reflects them in images that are captured in extremely low ambient light environments, thereby significantly improving the visual quality of these low-light images. The recent works on this problem only consider a pixel-level loss metric that ignores perceptual quality and thus generate outputs susceptible to visual artifacts. To address this problem, we propose a new loss function that exploits the characteristics of both pixel-wise and perceptual metrics, enabling our deep neural network to learn the camera processing pipeline to transform the short-exposure, low-light RAW sensor data to well-exposed sRGB images. The results show that our method outperforms the state-of-the-art according to psychophysical tests as well as pixel-wise standard metrics and recent learning-based perceptual image quality measures. In essence, the proposed model can potentially replace the conventional digital camera pipeline for the specific case of extreme low-light imaging.

First Page

37

Last Page

47

DOI

10.1016/j.neucom.2021.04.076

Publication Date

9-10-2021

Keywords

Digital camera pipeline, Learning-based ISP modeling, Low-light image enhancement, RAW to sRGB mapping

Comments

IR deposit conditions:

• OA version (pathway b)
• Accepted version
• 24 month embargo
• Must link to publisher version with DOI

Recommended Citation

S. W. Zamir, A. Arora, S. Khan, F. S. Khan, and L. Shao, “Learning digital camera pipeline for extreme low-light imaging,” Neurocomputing, vol. 452, pp. 37–47, Sep. 2021, doi: 10.1016/J.NEUCOM.2021.04.076.