Abstract
Benefiting from the recent real image dataset, learning-
based approaches have achieved good performance for
real-image denoising. To further improve the performance
for Bayer raw data denoising, this paper introduces two
new networks, which are multi-scale residual dense
network (MRDN) and multi-scale residual dense cascaded
U-Net with block-connection (MCU-Net). Both networks
are built upon a newly designed multi-scale residual dense
block (MRDB), and MCU-Net uses MRDB to connect the
encoder and decoder of the U-Net. To better exploit the
multi-scale feature of the images, the MRDB adds another
branch of atrous spatial pyramid pooling (ASPP) based on
residual dense block (RDB). Compared to the skip
connection, the block-connection using MRDB can
adaptively transform the features of the encoder and
transfer them to the decoder of the U-Net. In addition, a
novel noise permutation algorithm is introduced to avoid
model overfitting. The superior performance of these new
networks in removing noise within Bayer images has been
demonstrated by comparison results on the SIDD
benchmark, and the top ranking of SSIM in the NTIRE 2020
Challenge on Real Image Denoising - Track1: rawRGB.
1. Introduction
As a fundamental topic of image processing, image
denoising removes the presence of noise, reconstructs the
structural content details, and then generates high-quality
images. As a key component in many practical applications
and commercialization products, such as cameras and
smartphones, the research of image denoising attracts
attentions from both academic and industry. Traditional
image denoising research mainly focuses on removing
noise within sRGB data. In a recent decade, due to a better
understanding about noise, it is revealed that denoising
within Bayer raw data will be much more efficient than
denoising within sRGB data. As an example in Fig. 1, for
an image signal processing (ISP) pipeline within cameras
to render sRGB images from Bayer raw sensor data, a
simple salt noise within a Bayer image will alter other
neighbor pixels in the sRGB image if bypassing its
denoising process. That’s the reason why camera designers
think it is simpler and more efficient to remove noise in the
Bayer image than reconstructing corrupted pixels in the
sRGB image. Hence, this paper focuses on the Bayer image
denoising to solve the problem at its early stage.
Most developed sRGB denoising algorithms can be
applied in removing the noise within the Bayer data.
Compared to traditional handcrafted methods, such as non-
local mean [1] and BM3D [2], convolutional neural
network (CNN) started a new chapter for the research of
image denoising. Recently, the learning-based image
denoising methods have achieved remarkable performance
thanks to the large image datasets and well-studied deep
learning techniques [3]-[8].
Real Image Denoising Based on Multi-Scale Residual Dense Block and Cascaded
U-Net with Block-Connection
Long Bao
*
, Zengli Yang
*
, Shuangquan Wang, Dongwoon Bai, Jungwon Lee
SOC R&D, Samsung Semiconductor, Inc.
{long.bao, zengli.y, shuangquan.w, dongwoon.bai, jungwon2.lee}@samsung.com
*
These authors have equal contributions.
Noisy Bayer Image
(One salt noise at center)
Rendered Noise-Free sRGB
Image
Rendered Noisy sRGB
Image
Figure 1: Example to show how an image signal processing (ISP)
pipeline (bypassing denoising) makes the denoising problem
much more complex and difficult in the sRGB data: one salt noisy
pixel at the center of Bayer image will lead to several noisy pixels
in the sRGB image.