Signal Processing: Image Communication 96 (2021) 116250

Contents lists available at ScienceDirect

Signal Processing: Image Communication

journal homepage: www.elsevier.com/locate/image

UIEC^2-Net: CNN-based underwater image enhancement using two color

space✩
Yudong Wang, Jichang Guo ∗, Huan Gao, Huihui Yue
School of Electrical and Information Engineering, Tianjin University, Tianjin, 300072, China

ARTICLE INFO ABSTRACT

Keywords: Underwater image enhancement has attracted much attention due to the rise of marine resource development
Underwater image enhancement in recent years. Benefit from the powerful representation capabilities of Convolution Neural Networks(CNNs),
HSV color space multiple underwater image enhancement algorithms based on CNNs have been proposed in the past few
Deep learning
years. However, almost all of these algorithms employ RGB color space setting, which is insensitive to
image properties such as luminance and saturation. To address this problem, we proposed Underwater Image
Enhancement Convolution Neural Network using 2 Color Space (UICE^2-Net) that efficiently and effectively
integrate both RGB Color Space and HSV Color Space in one single CNN. To our best knowledge, this method
is the first one to use HSV color space for underwater image enhancement based on deep learning. UIEC^2-Net
is an end-to-end trainable network, consisting of three blocks as follow: a RGB pixel-level block implements
fundamental operations such as denoising and removing color cast, a HSV global-adjust block for globally
adjusting underwater image luminance, color and saturation by adopting a novel neural curve layer, and an
attention map block for combining the advantages of RGB and HSV block output images by distributing weight
to each pixel. Experimental results on synthetic and real-world underwater images show that the proposed
method has good performance in both subjective comparisons and objective metrics. The code is available at
https://github.com/BIGWangYuDong/UWEnhancement.

1. Introduction underwater image formation model [7], which is inaccurate because it

assume many parameters. This leads to get not good enough results for
Nowadays, exploration and utilization of marine resources has be- some types of underwater images and is only suitable for underwater
come the strategy center in international community. It is very impor- scenes under certain circumstances. In the past few decades, deep
tant to make use of these images because underwater images carry learning technology has developed rapidly, which has brought a series
plenty of information. However, due to the impact of backscatter in of breakthroughs to various of computer vision and image processing
far distances, light selective absorption and scattering in water, the tasks [8–15]. Affected by deep learning, many researchers began to use
raw underwater images usually suffer from low-quality, including low
CNN-based and GAN-based algorithms to enhance underwater image.
contrast and brightness, color deviations, blurry details and uneven
The goal of CNN-based algorithms is to be faithful to the original un-
bright speck. The purpose of underwater image enhancement is to
derwater image, while the core of GAN-based algorithms is to improve
obtain higher quality and clearer underwater images, so as to make
the perceptual quality of the entire underwater image. However, the
more effective use of image information. It can make full use of images
current underwater image enhancement still fails to observe attractive
information and has been widely used to promote numerous engineer-
ing and high-level research tasks such as underwater fish detection, performance. This might be attributable to the fact that most deep
shipwreck detection, underwater archaeology, etc. learning-based underwater image enhancement algorithms almost all
Early underwater image enhancement methods often directly use used RGB Color Space. The RGB color space can deal with the scatter-
traditional techniques(e.g. histogram equalization based methods, and ing problem and improve the underwater image issues such as color
physical-based methods [1,2]). However, these algorithms have cer- deviations, some issues of quality degradation such as low contrast,
tain limitations: the classic in-air image enhancement methods [3,4] saturation and brightness cannot get further improvement due to the
are often not outstanding enough in processing underwater image. RGB color space cannot directly reflect some important parameters of
Most physical model-based methods [5,6] are based on a simplified the image.

✩ The work described in this paper was fully supported by a grant from the National Natural Science Foundation of China (No. 61771334).
∗ Corresponding author.
E-mail addresses: [email protected] (Y. Wang), [email protected] (J. Guo).

https://doi.org/10.1016/j.image.2021.116250
Received 31 July 2020; Received in revised form 10 March 2021; Accepted 23 March 2021
Available online 16 April 2021
0923-5965/© 2021 Elsevier B.V. All rights reserved.
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

• Two color space: We apply a differentiable transformations of

the image in HSV color spaces. To our best knowledge, this
method is the first to use HSV color space for underwater image
enhancement based on deep learning.

2. Relate work

Underwater image enhancement based on deep learning: Ex-

ploring the underwater world has become a popular direction in recent
years [20,21]. Underwater image enhancement as an important pre-
treatment process aim to improve the visual quality of low-quality
images. A variety of methods based on deep learning have been pro-
posed and can be organized into two main categories, which are
Fig. 1. Samples of real-world underwater image enhancement by UIEC^2-Net. Top row: CNN-based algorithms and GAN-based algorithms. Moreover, the deep
raw underwater images from DUIE datasets [16]; Bottom row: the corresponding results
of our model.
learning-based underwater image enhancement can be divided into 5
methods [22] based on their architectural difference: encoder–decoder
models [23–25], which is a famous architecture in many low-level tasks
to advance image quality; designed a modular or block has the same
In this paper, to make up for the above shortcomings of RGB
structure and are repeatedly applied in the network to improve the
color space based enhancement methods, we propose UIEC^2-Net, a
feature extraction capability of the network [26,27]; designed multiple
novel CNN-based underwater image enhancement methods, using both
branch aims to learn different features through separate branches [16,
RGB Color Space and HSV Color Space. The samples of real-world
28,29]; improving the performance of enhancement and restoration by
underwater image enhancement by UIEC^2-Net is shown in Fig. 1.
predicted the depth map or transmission map of the underwater im-
UIEC^2-Net contains three blocks as follow: a RGB pixel-level block for
age [30–32]; employed multiple generators to inference the improved
simple and basic operations such as denoising and removing color cast,
image [33–35].
a HSV global-adjust block for globally adjust and fine-tune underwater
Underwater image dataset: Deep learning-based methods always
image such as saturation and brightness, and an attention map block
need to be driven by large datasets. Unlike other low-level vision
for combining the advantages of RGB pixel-level block and HSV global-
tasks such as image super-resolution [36], image denoising [37], image
adjust block output images. Similar to [17], UIEC^2-Net includes global
deblurring [38] and image dehazing [39], which can use plenty of
image transformations to adjust colors, brightness, saturation, etc. Same
synthetic degraded images and high-quality counterparts for training,
as [16], the RGB pixel-level block is a plain fully CNN, which we do not
it is difficult to synthesize realistic underwater images for deep learn-
use skip connection or downpooling. Although most of the processing ing training, due to the complex underwater image formation models
results of the RGB block is already doing very well, there are still exists are affected by many factors(e.g. turbidity and lighting conditions).
some problem such as insufficient texture details and the background Therefore, it is necessary to synthesize underwater images, using GAN
color cast cannot restored well. Inspired by [18], we applied a HSV to synthesize low-quality underwater images is a common way to
global-adjust block to enhance the properties of underwater images obtain paired image, Li et al. [32] proposed a GAN-based method to
through the learned global adjustment curves in HSV color space, which synthesize underwater images from RGB-D in-air images and Fabbri
more expressively adjust underwater image properties and make the et al. [25] used Cyclegan [27,40] to degrade the underwater im-
underwater image enhancement effect better, no matter on visual effect age quality. Meanwhile, using underwater image formation model has
or scores of image quality evaluation standard. We simplified the model gradually attracted attention recently. Blasinski [41] provided a three-
of [18], but it still shows good performance in underwater image en- parameter underwater image formation model [42] for underwater
hancement. However, because the H-channel of HSV color space is very image simulation. Anwar et al. [26] incorporate a new underwater im-
sensitive, when HSV color space is used to process some underwater age synthesis method that simulates 10 different categories underwater
images, this may cause color distortion in some areas of the image. To images using NYU-v2 indoor dataset [43], which is used in related
solve the problem and further improve the quality of enhanced image, research [24]. Although the synthetic image is similar to the real-world
attention map block is used to combine the advantages of the output underwater image, there still exists a gap between them. Other methods
results of the above two blocks and avoid the related disadvantages include Duarte et al. [44] simulated underwater image degradation
at the same time. Therefore, attention map block has been proposed using milk, chlorophyll, or green tea in a tank to get paired-dataset.
to learn the weight of each pixel for the output of each block. This is More recently, Li et al. constructed a real-world underwater image
similar to [19], but we are pixel-level instead of channel-level for better enhancement dataset, including 950 underwater images, 890 of which
results. Our contributions in this paper are three-fold: have the correspond reference images. These potential reference images
are produced by 12 enhancement methods, and voted by 50 volunteers
• An end-to-end convolution neural network for underwater to select the final references.
image enhancement: The convolution neural network architec- Underwater image enhancement using multiple color spaces:
ture incorporating a RGB pixel-level block for simple and basic Using multiple color spaces for image enhancement is a popular re-
operations such as denoising and removing color cast, a HSV search direction in recent years. However, underwater image enhance-
global-adjust block for adjusting global color cast and fine-tuning ment methods using multiple color spaces are all focus on conventional
underwater image such as saturation and brightness, and a at- algorithms. Iqbal et al. [45,46] used histogram stretching process on
tention map block for getting better underwater enhanced image. two different color spaces. They first use RGB color space to correct
UIEC^2-Net parameters are much less than deep CNN models, underwater image contrast and then use HSI color space to further im-
meanwhile, our model has better generalization ability and gets prove the image quality. Hitam et al. [3] used CLAHE on RGB and HSV
better results on real-world underwater image datasets. color spaces and then combined Euclidean Norm to obtain enhanced
• HSV global-adjust block: We introduce a simple neural network image, the results shows that the image quality can be improved by
curve layer, that learns a piece-wise linear scaling curve to adjust enhancing the contrast. Ghani et al. [47] combined global and local
image properties in HSV color space, especially the saturation and contrast correction by using RGB and HSV two color spaces to enhance
brightness of the underwater image. underwater image quality. In 2015, Ghani et al. [48] used RGB and

2
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

Fig. 2. An overview of the proposed UIEC^2-Net architecture. UIEC^2-Net consists of three blocks trained end-to-end: a RGB pixel-level block for simple and basic processing, a HSV
global-adjust block that leverages neural curve layers for globally refine image property (saturation and luminance), and a attention map block for getting better underwater enhanced
image through attention mechanism.

HSV color spaces during histogram stretching process, and in 2017, consists with eight 3 × 3 convolutional layers with stride 1, each of
Ghani et al. [49] proposed a method called Recursive Adaptive His- them using batch normalization and the first 7 layers followed by a
togram Modification (RAHM) using both RGB and HSV color space to Leaky ReLU activation, the last layer using sigmoid as the activation
improve the visual quality of the underwater images. In view of above function, its purpose is to make the output in [0,1]. RGB pixel-level
introduction, using multiple color space to enhance underwater image block produces a result with output shape 𝐻 × 𝑊 × 3, where 𝐻, 𝑊
shows better performance. Meanwhile, CNN has better enhancement is the height and width of feature maps, the number of channels is 3,
effect than traditional methods. Therefore, in this paper, we are the and the output is passed over to the HSV global-adjust block for further
first using two color spaces(RGB and HSV color space) in deep learning- processing. The performance of the RGB pixel-level block architecture
based underwater image enhancement to get higher quality images. As we proposed in this paper has reached a high level, however, we believe
far as we know, using multiple color spaces is not currently applied to that the widely used backbones for pixel-level tasks such as the U-
deep learning-based underwater image enhancement methods. Net architecture [50] and the residual network architecture [51] can
further improve the performance, we will try in the future work.
3. Proposed model HSV global-adjust block: The output of RGB pixel-level block is
transformed to HSV color space through differentiable RGB→HSV and
In this section, we first discuss the details of the proposed CNN- is used as input of HSV global-adjust block. Our proposed HSV global-
based underwater image enhancement using 2 color spaces(RGB and adjust block and parameter settings are shown at the bottom of Fig. 2,
HSV color space). Then we introduce the loss function used in UIEC^2- which consists of five 3 × 3 convolutional layers with stride 1, each of
Net. Finally, we present the RGB color space and the HSV color space them followed by a Leaky ReLU activation. There is a 2 × 2 maxpooling
conversions to permit end-to-end learning via stochastic gradient de- after the first four layers and the remaining one is not. Then we place
scent(SGD) and backpropagation. a global average pooling layer processes the shape of the feature maps
to 1 × 1 × 𝐶 and followed by a fully connected layer that regress
3.1. Network architecture the knot points of a piece-wise linear curve. The curve adjusts the
predicted image(𝐼̂𝑖 ∈ [0, 1]) by scaling pixels with the formula presented
UIEC^2-Net is an end-to-end trainable neural network that consists in Eq. (1):
of three blocks as shown in Fig. 2. The RGB pixel-level block is a CNN-
based network for simple and basic processing such as denoising and ∑
𝑀−1
𝑆(𝐼̂𝑖𝑗𝑙 ) = 𝑘0 + (𝑘𝑚+1 − 𝑘𝑚 )𝛿(𝑀 𝐼̂𝑖𝑗𝑙 − 𝑚) (1)
removing color cast, the HSV global-adjust block employing a novel 𝑚=0
neural curve layer that globally refine image properties, especially the
where
saturation and luminance, and the attention map block distribute weight
to result of RGB and HSV block at pixel-level through the attention ⎧ 0, 𝑥<0
⎪
mechanism to obtain better enhanced underwater images. 𝛿(𝑥) = ⎨ 𝑥, 0≤𝑥≤1
RGB pixel-level block: The architecture of the RGB pixel-level ⎪ 1, 𝑥>1
⎩
block and parameter settings are shown at top of Fig. 2. Because, down-
sampling may cause some problems such as missing image information, where 𝑀 is the number of predicted knot points, 𝐼̂𝑖𝑗𝑙 is the 𝑗th pixel
especially, for pixel-level computer vision tasks, it needs to be used value in the 𝑙th color channel of the 𝑖th image, 𝑘𝑚 is the value of the
in conjunction with upsampling, and needs various of convolution to knot point 𝑚. The HSV global-adjust block can be seen as a simple
offset the impact of downsampling and upsampling. We designed the matter of multiplication of a pixel’s value: the pixel value is multiplied
RGB pixel-level block as a plain fully CNN without downsampling. It by the value corresponding to the piece-wise linear curve to get the

3
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

between the UIEC^2-Net output and groundtruth image. The L1 loss

presented in Eq. (4):
‖ ‖
𝓁1 = ‖𝐼̂𝑖 − 𝐼𝑖 ‖ (4)
‖ ‖1
SSIM loss (𝑆𝑆𝐼𝑀 ): We use the SSIM loss [52] in our objective
function to impose the structure and texture similarity on the predicted
image. In this paper, we use gray images, which convert from RGB
images, to compute SSIM score, and for each pixel 𝑥, the SSIM value is
computed within a 11 × 11 image patch around the pixel. The formula
Fig. 3. Examples of learnt neural global adjustment curves in HSV color space. is as follows:
2𝜇𝐼 (𝑥)𝜇𝐼̂ (𝑥) + 𝑐1 2𝜎𝐼 𝐼̂ (𝑥) + 𝑐2
𝑆𝑆𝐼𝑀(𝑥) = ⋅ (5)
𝜇𝐼2 (𝑥) + 𝜇 2 (𝑥) + 𝑐1 𝜎𝐼2 (𝑥) + 𝜎 2 (𝑥) + 𝑐2
𝐼̂ 𝐼̂
globally refined images. Examples of the piece-wise linear curve that
where 𝜇𝐼̂ (𝑥) and 𝜎 ̂ (𝑥) correspond to the mean and standard deviation
learnt by proposed model are shown in Fig. 3. 𝐼
of the RGB patch from the predicted image, similarly, 𝜇𝐼 (𝑥) and 𝜎𝐼 (𝑥)
HSV is composed of three channels(Hue, Saturation, Value), we
are the same corresponds to groundtruth image. And 𝜎𝐼 𝐼̂ (𝑥) is cross-
using four curves to adjust and refine image properties. We adjust
covariance. We set 𝑐1 = 0.02 and 𝑐2 = 0.03. The SSIM loss is expressed
luminance by using value scaled based on value, adjust saturation by
as:
using saturation based on saturation and saturation based on hue, and
1 ∑
𝑁
refine color by using hue based on hue. The input of HSV global-adjust ( )
𝑆𝑆𝐼𝑀 = 1 − 𝑆𝑆𝐼𝑀 𝑥𝑖 (6)
block is the HSV image converted from the RGB pixel-level block’s 𝑁 𝑖=1
output. The HSV global-adjust block produces a HSV result with output
The SSIM loss is similar to L1 loss, we applied it twice during training.
shape 𝐻 × 𝑊 × 3 and the HSV image is converted back to RGB space
Perceptual loss(𝜙𝑗 ): Inspired by [53], we defined the perceptual
via a differentiable HSV to RGB conversion.
loss based on VGG network [54], which pre-trained on the ImageNet
Attention map block: The raw input image is concatenated with
dataset [55]. Due to the deep layer has a better understanding of
the output of RGB pixel-level block and HSV global-adjust block as
semantic information and can fully preserve the image content and
the input of this block, which is shown at the middle of Fig. 2. The
overall spatial structure, and the shallow layer is more sensitive to
architecture of attention-map block is similar to RGB pixel-level block,
color and texture, we select layer 4_3 from VGG-19 to make it sensitive
which consists with eight 3 × 3 convolutional layers, each of them using
to both color and semantics. The perceptual loss expressed as the
batch normalization and the first 7 layers followed by a Leaky ReLU
distance between the feature representations of predicted RGB image
activation, the last layer uses Sigmoid as activation function, which can
and groundtruth image:
set the output between 0 and 1. The output shape of attention map
block is 𝐻 × 𝑊 × 6, where the first three channels are weights of RGB 1 ∑𝑁
‖ ‖
𝜙𝑗 = ‖𝜙 (𝐼̂ ) − 𝜙𝑗 (𝐼𝑖 )‖ (7)
block output and the remaining three channels belong to HSV block 𝐶𝑗 𝐻𝑗 𝑊𝑗 𝑖=1 ‖ 𝑗 𝑖 ‖
output. Finally, multiply the attention map with the output of RGB
where 𝜙𝑗 is the 𝑗th convolutional layer of pre-trained VGG-19; 𝑁 is the
image converted form HSV image and output of RGB block respectively,
number of each batch in the training procedure; 𝐶𝑗 𝐻𝑗 𝑊𝑗 represents
and add the two results above to get high quality enhanced underwater
the 𝑗th layer dimension of the feature maps of the VGG-19 network,
image, which is the UIEC^2-Net final output.
which 𝐶𝑗 , 𝐻𝑗 , 𝑊𝑗 represent number, height and width of the feature
map, respectively. The perceptual loss computed the distance between
3.2. Loss function
the UIEC^2-Net output RGB image and groundtruth image.
Loss term weights: Each loss term has a weight hyperparameter:
The end-to-end training of UIEC^2-Net is supervised by four loss 𝜔𝓁1 , 𝜔𝑆𝑆𝐼𝑀 , 𝜔ℎ𝑠𝑣 , 𝜔𝑝𝑒𝑟𝑐 , meanwhile, the L1 and SSIM loss has two
components, which consists of two color space-specific terms. We de- hyperparameters: 𝜆𝑝𝑖𝑥𝑒𝑙 , 𝜆𝑤ℎ𝑜𝑙𝑒 . We set 𝜔𝓁1 = 1, 𝜔𝑆𝑆𝐼𝑀 = 1, 𝜔ℎ𝑠𝑣 = 1,
fined 𝐼𝑖 is the groundtruth image and 𝐼̂𝑖 is the predicted image. Our 𝜔𝑝𝑒𝑟𝑐 = 0.5, and at the first 20 epochs we set 𝜆𝑝𝑖𝑥𝑒𝑙 = 0.5, 𝜆𝑤ℎ𝑜𝑙𝑒 = 0.5,
total loss function is presented in Eq. (2): and set 𝜆𝑝𝑖𝑥𝑒𝑙 = 0.1, 𝜆𝑤ℎ𝑜𝑙𝑒 = 0.9 for the last epochs.
 =(𝜆𝑝𝑖𝑥𝑒𝑙 + 𝜆𝑤ℎ𝑜𝑙𝑒 )(𝜔𝓁1 𝓁1 + 𝜔𝑆𝑆𝐼𝑀 𝑆𝑆𝐼𝑀 )+
(2) 3.3. Color space transformations
𝜔ℎ𝑠𝑣 ℎ𝑠𝑣 + 𝜔𝑝𝑒𝑟𝑐 𝜙𝑗

where ℎ𝑠𝑣 is the HSV color space loss function, and 𝓁1 , 𝑆𝑆𝐼𝑀 , 𝜙𝑗 UIEC^2-Net relies on differentiable RGB→HSV and HSV→RGB color
are RGB color space loss functions. 𝜆𝑝𝑖𝑥𝑒𝑙 represent the weight applied space conversions to permit the end-to-end learning via backpropaga-
in RGB pixel-level block and 𝜆𝑤ℎ𝑜𝑙𝑒 is the weight applied in whole tion. The related color space conversion methods can be found on the
network. These functions and terms are defined in more detail below. OpenCV website.1
HSV loss(ℎ𝑠𝑣 ): HSV image 𝐼𝑖 is divided into three channels: hue RGB→HSV: The color space transformation contains minimum and
𝐻𝑖 ∈ [0, 2𝜋), saturation 𝑆𝑖 ∈ [0, 1], value 𝑉𝑖 ∈ [0, 1]. Inspired by [18], maximum operations, each of them can differentiable. The RGB→HSV
conversion functions shown as Eqs. (8)–(10):
we compute ℎ𝑠𝑣 in the conical HSV color space:
‖ ( )‖ 𝑉 = max(𝑅, 𝐺, 𝐵) (8)
ℎ𝑠𝑣 = ‖𝑆̂𝑖 𝑉̂𝑖 cos(𝐻̂ 𝑖 ) − 𝑆𝑖 𝑉𝑖 cos 𝐻𝑖 ‖ (3)
‖ ‖1 {
𝑉 −min(𝑅,𝐺,𝐵)
Compared to RGB, HSV has more advantages because it separates , 𝑖𝑓 𝑉 ≠ 0
𝑆= 𝑉 (9)
color into useful components(hue, saturation, value). It can globally 0, 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒
60(𝐺−𝐵)
adjust luminance through value-channel, improve saturation through ⎧ (𝑉 −min(𝑅,𝐺,𝐵))
, 𝑖𝑓 𝑉 = 𝑅
saturation-channel and refine color through hue-channel. ⎪ 120+60(𝐵−𝑅)
𝐻 =⎨ (𝑉 −min(𝑅,𝐺,𝐵))
, 𝑖𝑓 𝑉 = 𝐺 (10)
L1 loss (𝓁1 ): We use 𝐿1 loss on RGB pixels between the predicted ⎪ 240+60(𝑅−𝐺)
, 𝑖𝑓 𝑉 = 𝐵
and groundtruth images. This loss function is first applied in RGB pixel- ⎩ (𝑉 −min(𝑅,𝐺,𝐵))

level block, which computed 𝐿1 distance between the predicted RGB

1
image and groundtruth image. Then it applied to compute the distance https://docs.opencv.org/3.3.0/de/d25/imgproc_color_conversions.html.

4
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

Table 1
Full-reference image quality assessment in terms of MSE, PSNR and SSIM on synthetic
images.
Method MSE PSNR(dB) SSIM
raw 4142.5386 14.4978 0.6948
HE 3869.0180 14.2563 0.6779
WB 4246.7734 14.5174 0.6862
UDCP [56] 4809.4566 13.6778 0.6373
ULAP [57] 3725.4596 14.3739 0.6989
UWGAN [58] 4028.8004 13.3007 0.7083
UGAN [25] 2266.4678 16.2636 0.6625
UWCNN [26] 3770.7036 14.8638 0.7419
DUIENet [16] 2323.3417 16.1073 0.7734
UIEC^2-Net 1126.3743 20.5442 0.8486

4.1. Implementation details

Fig. 4. Piecewise linear functions used to convert HSV to RGB color space. For training, the input of our network are both synthetic and real-
Source: https://commons.wikimedia.org/w/index.php?curid=1116423.
world underwater images. A random set of 800 pairs of real-world
images extracted from the UIEBD dataset [16] and 1200 pairs of
synthetic images with ten types are generated from the NYU-v2 RGB-D
if 𝐻 < 0 then 𝐻 = 𝐻 + 360, 0 ≤ 𝑉 ≤ 1, 0 ≤ 𝑆 ≤ 1, 0 ≤ 𝐻 ≤ 360. dataset based on [26], both of them (2000 images in total) are used
The operation involves conditional statements, which conditioned on to train our network. We resize the input images to size 350 × 350
the values of R, G and B. However, these can be processed under the and randomly crop them to size 320 × 320. For testing, 90 real-world
pytorch2 framework, and the operations such as backpropagation can and 900 synthetic images are treated as the testing set. Compared to
be performed. training, we do not resize or randomly crop the input image. We trained
HSV→RGB: The conversion rarely applied in relevant deep learning- our model using ADAM and set the learning rate to 0.0001. The batch
based methods, due to the related formulas are hard to differentiable. size is set to 8 and the epoch is set to 50. We use Pytorch as the deep
Inspired by [18], we replaced complex formulas with piecewise linear learning framework on an Inter(R) i7-8700k CPU, 32GB RAM, and a
functions based on Fig. 4. The corresponding piecewise linear curves Nvidia GTX 1080Ti GPU.
are defined by linear segments and knot points, which makes them have
4.2. Experiment on synthetic datasets
a gradient between the points. The formulas shown as:

𝑅(𝐼̂𝑖𝑗 ) =𝑣𝑗 − ((𝑣𝑠 𝑣𝑗 )∕60)𝛿(360ℎ𝑗 − 60) (11) It is very common to train the network through synthetic under-
water images because there is no GT for underwater images, thus, We
+ ((𝑣𝑠 𝑣𝑗 )∕60)𝛿(360ℎ𝑗 − 240)
first evaluate the capacity of our method on synthetic testing set. In
𝐵(𝐼̂𝑖𝑗 ) =𝑣𝑗 (1 − 𝑠𝑗 ) + ((𝑣𝑠 𝑣𝑗 )∕60)𝛿(360ℎ𝑗 − 120) (12) Fig. 5, we present the results of underwater image enhancement on the
− ((𝑣𝑠 𝑣𝑗 )∕60)𝛿(360ℎ𝑗 − 300) synthetic underwater images from our testing set.
As shown in Fig. 5(a),we simply give examples of synthetic un-
𝐵(𝐼̂𝑖𝑗 ) =𝑣𝑗 (1 − 𝑠𝑗 ) + ((𝑣𝑠 𝑣𝑗 )∕60)𝛿(360ℎ𝑗 − 120) (13) derwater images, generated from RGB-D in-air images. Fig. 5(b)–(k)
− ((𝑣𝑠 𝑣𝑗 )∕60)𝛿(360ℎ𝑗 − 300) shows the reference image of Histogram Equalization (HE), White
Balance (WB), UDCP [56], ULAP [57], UWGAN [58], UGAN [25]
where we define 𝛿(𝑥) as in Eq. (14) ,UWCNN [26], DUIENet [16], the proposed UIEC^2-Net and the cor-
⎧ 0, responding reference image respectively. It is noted that UWCNN [26]
𝑥<0
⎪ has 10 different pre-train models, which are trained separately by 10
𝛿(𝑥) = ⎨ 𝑥, 0 ≤ 𝑥 ≤ 40 (14) types. We only choose type-1 model as a comparative experiment,
⎪ 60, 𝑥 > 60
⎩ so when the underwater images category is rich, its generalization
performance is not good. Similarly, for other methods, they have better
where ℎ𝑗 , 𝑠𝑗 , 𝑣𝑗 are the hue, saturation and value of pixel 𝑗 of image
processing results in one or two categories. However, in most cases,
𝐼𝑖 ,respectively. and both of them belongs [0, 1].
they are not good at processing color cast and image details. The meth-
ods we proposed has good generalization performance and can restore
4. Experimental results color cast better, which Fig. 5 can demonstrating its effectiveness and
robustness.
To evaluate our method, we perform qualitative (subjective com- We choose Mean Square Error (MSE), Peak Signal to Noise Ratio
parisons) and quantitative (subjective comparisons) comparisons with (PSNR) and Structural SIMilarity index (SSIM) as full-reference image
traditional methods and the recent state-of-the-art deep learning-based quality assessment to assess the processed synthetic underwater images
underwater image enhancement methods on both synthetic and real- for all of the compared methods. For MSE and PSNR metrics, the lower
world underwater images. These methods include Histogram Equaliza- MSE (higher PSNR) denotes the result is more close to the GT in terms
tion (HE), White Balance (WB), UDCP [56], ULAP [57], UWGAN [58], of image content. For SSIM, the higher SSIM scores mean the result is
UGAN [25] ,UWCNN [26], DUIENet [16]. For deep learning-based more similar to the GT in terms of image structure and texture. Table 1
methods, we run the source codes with the pre-trained model param- reports the quantitative results of different methods on the synthetic
eters shared by provided by corresponding authors to produce the testing images. We highlight the top 1 performance in red, whereas
best results for an objective evaluation. In this section, we will first the second best is in blue. Regarding the SSIM response, UIEC^2-
Net is higher than the second-best performer. Similarly, our PSNR is
supplement the training details, and then analyzed the experimental
obviously higher (less erroneous as indicated by the MSE scores) than
results of synthetic and real-world underwater images.
the compared methods. This can demonstrate that our proposed method
achieves the best performance in terms of full-reference image quality
2
https://pytorch.org/. assessment on synthetic underwater images.

5
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

Fig. 5. Subjective comparisons on synthetic underwater images. From left to right are raw underwater images, the results of Histogram Equalization (HE), White Balance (WB),
UDCP [56], ULAP [57], UWGAN [58], UGAN [25], UWCNN [26], DUIENet [16], the proposed UIEC^2-Net and reference images.

Fig. 6. Subjective comparisons on bluish underwater images. From left to right are raw underwater images, the results of Histogram Equalization (HE), White Balance (WB),
UDCP [56], ULAP [57], UWGAN [58], UGAN [25], UWCNN [26], DUIENet [16], the proposed UIEC^2-Net and reference images.

Fig. 7. Subjective comparisons on greenish underwater images. From left to right are raw underwater images, the results of Histogram Equalization (HE), White Balance (WB),
UDCP [56], ULAP [57], UWGAN [58], UGAN [25], UWCNN [26], DUIENet [16], the proposed UIEC^2-Net and reference images.

Fig. 8. Subjective comparisons on low-illuminated underwater images. From left to right are raw underwater images, the results of Histogram Equalization (HE), White Balance (WB),
UDCP [56], ULAP [57], UWGAN [58], UGAN [25], UWCNN [26], DUIENet [16], the proposed UIEC^2-Net and reference images.

6
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

Fig. 9. Subjective comparisons on yellowish underwater images. From left to right are raw underwater images, the results of Histogram Equalization (HE), White Balance (WB),
UDCP [56], ULAP [57], UWGAN [58], UGAN [25], UWCNN [26], DUIENet [16], the proposed UIEC^2-Net and reference images.

Fig. 10. Subjective comparisons on shallow water images. From left to right are raw underwater images, the results of Histogram Equalization (HE), White Balance (WB), UDCP [56],
ULAP [57], UWGAN [58], UGAN [25], UWCNN [26], DUIENet [16], the proposed UIEC^2-Net and reference images.

Fig. 11. The enhanced result of only using RGB pixel-level block, without(w/o) using HSV global-adjust block and UIEC^2-Net. Top row: the result of only use RGB pixel-level
block; Middle row: the result of UIEC^2-Net not using HSV global-adjust block; Bottom row: the result of UIEC^2-Net.

4.3. Experiment on real datasets (a) of Figs. 6 and 7. Underwater images also include low-brightness sit-
uations(shown in Fig. 8), yellowish styles (shown in Fig. 9) and shallow
water images(shown in Fig. 10). As shown in Figs. 6–10, color deviation
From test dataset, we extracted five categories underwater images
seriously affects the visual quality of underwater images. Traditional
(greenish underwater images, bluish underwater images, yellowish un- methods usually consider only one type of underwater images. HE
derwater images, shallow water images, and underwater images image effectively improves the contrast but cannot remove the color cast well.
with limited illumination) to compare the generalization capability WB can improve the greenish underwater images by supplementing
of underwater enhancement methods. Note that our test dataset and red light, but the results are not good in other cases. ULAP [57] can
corresponding reference images are provided by UIEBD [16]. Due to better enhance the greenish underwater images due to a good estimate
the nature of light propagation, the red light first disappears in water, of the underwater optical attenuation but the bluish underwater image
followed by green light and then the blue light. Most underwater cannot be improved well. UDCP [56] aggravate the effect of the color
images are bluish and greenish, such as the raw underwater images in cast. Deep learning-based methods have poor generalization and low

7
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

Table 2 Table 4
Full-reference image quality assessment in terms of MSE, PSNR and SSIM on real-world Image quality assessment of with/without HSV Rtouching block and attention map
images. block.
Method MSE PSNR(dB) SSIM Method MSE PSNR(dB) SSIM UCIQE UIQM
raw 1322.1355 18.2701 0.8151 only RGB pixel-level block 397.3553 23.8228 0.92849 0.6351 4.0732
HE 1078.9476 19.5854 0.8509 w/o attention map block 393.9875 24.1794 0.92651 0.60183 4.0979
WB 1455.7350 17.9261 0.8041 UIEC^2-Net 365.5963 24.5663 0.9346 0.6193 4.0790
UDCP [56] 5829.6013 11.1646 0.5405
ULAP [57] 1517.6039 18.6789 0.8194
UWGAN [58] 1256.0906 18.6209 0.8454
UGAN [25] 558.2965 21.3031 0.7691 such a sacrifice is necessary to improved whole network preference.
UWCNN [26] 1342.7639 18.2851 0.8150
DUIENet [16] 2023.3417 16.2906 0.7884 In addition, it can achieve good results in subjective perception of
UIEC^2-Net 365.5963 24.5663 0.9346 underwater image.
Such an example is presented in Fig. 11(a), (b) demonstrate that
Table 3 after adding HSV global-adjust block, the background color cast of
No-reference image quality evaluation in terms of UCIQE, UICM, UISM, UIConM and underwater image is processed better and the saturation has been
UIQM on real-world images. improved to make the processed images more realistic. Fig. 11(c) shows
Method UCIQE UICM UISM UIConM UIQM that although adopting global-adjust block can effectively avoid the
raw 0.5044 2.5656 2.7255 0.5492 2.8407 luminance problem such as overexposure, HSV block may also over-
HE 0.5828 4.3170 3.6235 0.7017 3.7006
processing the image to reduce the contrast and saturation of the
WB 0.5429 5.1675 2.6102 0.5472 2.8731
UDCP [56] 0.5747 4.0233 2.7689 0.6169 3.1367 entire image. Meanwhile, due to the H-channel of HSV color space is
ULAP [57] 0.6014 4.9015 2.8597 0.5145 2.8224 very sensitive, HSV block may causes color distortion in some reddish
UWGAN [58] 0.5352 3.0011 3.1388 0.6169 3.2174 areas when processing the underwater images. These shows in the
UGAN [25] 0.6162 4.2476 4.2527 0.7384 4.0157
UWCNN [26] 0.5044 2.55656 2.7255 0.5492 2.8407
second row of Fig. 11(d). Attention map block can solve the problem
DUIENet [16] 0.6051 4.0727 3.8559 0.6940 3.7988 by extracting the feature information of the Raw images, results of
UIEC^2-Net 0.6193 4.9046 4.7558 0.7094 4.0790 RGB and HSV, then distributing weight of each pixel to combine the
Reference 0.6451 5.2137 3.9190 0.7116 3.8483 advantages of results from RGB and HSV block. As shown in (c),(d),
some problems with HSV blocks can be avoided when UIEC^2-Net using
attention map block can. In addition, attention map block can also
sensitivity to brightness and saturation of underwater images, and tend avoid the appearance of noise blocks, which shown in (e).
to introduce artifacts, over-enhancement and color casts. By contrast,
(k) of Figs. 6–10 shows that our proposed UIEC^2-Net effectively re-
5. Conclusion
moves the haze and color casts (especially the background color cast)
of the underwater images, adjusts underwater image properties (e.g.
brightness and saturation) and has good generalization capability in In this paper, we proposed a novel CNN-based underwater image
dealing with underwater images. In addition, our results even achieve enhancement method, using both RGB Color Space and HSV Color
better visual quality than the corresponding reference images(e.g. less Space. We first use pixel-level block based on RGB color space for
noise and better details). simple and basic enhancement operations such as removing color cast
Similarly to Section 4.2, we choose MSE, PSNR and SSIM to assess and denoising, then a global-adjust block based on HSV color space
the recovered results on real-world underwater images. We calculate is used for globally refining underwater image properties such as
the results of each method and the corresponding reference image, the
luminance and saturation. The proposed method has a good effect on
quantitative results of different methods on real-world underwater im-
removing color cast, especially for the restoration and enhancement of
ages are shown in Table 2. Our method achieves the best performance
background color, and also an greatly retain the detailed information
in terms of full-reference image quality assessment, which can prove
of the underwater image. Furthermore, our method can be use as a
our proposed method is good at handling details.
Meanwhile, we choose underwater color image quality evaluation guide for subsequent research of underwater image color correction.
(UCIQE) [59] and underwater image quality measure (UIQM) [60] as Experiments on synthetic and real-world underwater images including
No-reference Image Quality Evaluation. UCIQE evaluate underwater qualitative and quantitative assessment demonstrated the effectiveness
image quality by color density, saturation and contrast. UIQM is a com- of our method.
prehensive underwater image evaluation index, which is the weighted
sum of underwater image colorfulness measure(UICM), underwater
image sharpness measure(UISM) and underwater image conreast mea- CRediT authorship contribution statement
sure(UIConM): 𝑈 𝐼𝑄𝑀 = 𝑐1 × 𝑈 𝐼𝐶𝑀 + 𝑐2 × 𝑈 𝐼𝑆𝑀 + 𝑐3 × 𝑈 𝐼𝑄𝑀. We
set 𝑐1 = 0.0282, 𝑐2 = 0.2953, 𝑐3 = 3.5753 according to [60]. As shown in Yudong Wang: Methodology, Programming, Writing - review &
Table 3, compared with other methods, our proposed method achieve editing. Jichang Guo: Discuss ideas, Methodology, Conceptualization,
the best score in UCIQE, UISM and UIQM, and even get higher score Writing - review & editing. Huan Gao: Discuss ideas, Data Collation,
than reference image in UIQM and UISM. This also demonstrates that
Experimental Comparison, Writing - review & editing. Huihui Yue:
our algorithm has more advantages in processing details.
Discuss ideas, Experimental Comparison, Writing - review & editing.
4.4. Ablation study
Declaration of competing interest
To demonstrate the effect of HSV global-adjust block and attention
map block in our network, we compared the proposed UIEC^2-Net with
only using RGB pixel-level block and without(w/o) attention map block The authors declare that they have no known competing finan-
as an ablation study. As show in Table 4, although the use of HSV block cial interests or personal relationships that could have appeared to
and attention map block decreased performance of UCIQE and UIQM, influence the work reported in this paper.

8
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

References [29] H. Li, J. Li, W. Wang, A fusion adversarial underwater image enhancement
network with a public test dataset, 2019, arXiv preprint arXiv:1906.06819.
[1] C. Li, J. Guo, R. Cong, Y. Pang, B. Wang, Underwater image enhancement by [30] M. Hou, R. Liu, X. Fan, Z. Luo, Joint residual learning for underwater image
dehazing with minimum information loss and histogram distribution prior, IEEE enhancement, in: 2018 25th IEEE International Conference on Image Processing
Trans. Image Process. 25 (12) (2016) 5664–5677. (ICIP), IEEE, 2018, pp. 4043–4047.
[2] C. Li, J. Guo, C. Guo, R. Cong, J. Gong, A hybrid method for underwater image [31] K. Cao, Y.-T. Peng, P.C. Cosman, Underwater image restoration using deep
correction, Pattern Recognit. Lett. 94 (2017) 62–67. networks to estimate background light and scene depth, in: 2018 IEEE Southwest
[3] M.S. Hitam, E.A. Awalludin, W.N.J.H.W. Yussof, Z. Bachok, Mixture contrast Symposium on Image Analysis and Interpretation (SSIAI), IEEE, 2018, pp. 1–4.
limited adaptive histogram equalization for underwater image enhancement, in: [32] J. Li, K.A. Skinner, R.M. Eustice, M. Johnson-Roberson, Watergan: Unsupervised
2013 International Conference on Computer Applications Technology (ICCAT),
generative network to enable real-time color correction of monocular underwater
IEEE, 2013, pp. 1–5.
images, IEEE Robot. Autom. Lett. 3 (1) (2017) 387–394.
[4] Z.-u. Rahman, J. Daniel, et al., Woodell, multis. cale retinex for color image
[33] C. Li, J. Guo, C. Guo, Emerging from water: Underwater image color correction
enhancement image, lEEE Process. 3 (1996) 1003–1006.
based on weakly supervised color transfer, IEEE Signal Process. Lett. 25 (3)
[5] C. Li, J. Guo, B. Wang, R. Cong, Y. Zhang, J. Wang, Single underwater image
enhancement based on color cast removal and visibility restoration, J. Electron. (2018) 323–327.
Imaging 25 (3) (2016) 033012. [34] J. Lu, N. Li, S. Zhang, Z. Yu, H. Zheng, B. Zheng, Multi-scale adversarial network
[6] C. Li, J. Quo, Y. Pang, S. Chen, J. Wang, Single underwater image restoration for underwater image restoration, Opt. Laser Technol. 110 (2019) 105–113.
by blue-green channels dehazing and red channel correction, in: 2016 IEEE [35] X. Ye, H. Xu, X. Ji, R. Xu, Underwater image enhancement using stacked
International Conference on Acoustics, Speech and Signal Processing (ICASSP), generative adversarial networks, in: Pacific Rim Conference on Multimedia,
IEEE, 2016, pp. 1731–1735. Springer, 2018, pp. 514–524.
[7] J.Y. Chiang, Y.-C. Chen, Underwater image enhancement by wavelength [36] Z. Li, J. Yang, Z. Liu, X. Yang, G. Jeon, W. Wu, Feedback network for image
compensation and dehazing, IEEE Trans. Image Process. 21 (4) (2011) super-resolution, in: Proceedings of the IEEE Conference on Computer Vision and
1756–1769. Pattern Recognition, 2019, pp. 3867–3876.
[8] Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature 521 (7553) (2015) [37] K. Zhang, W. Zuo, Y. Chen, D. Meng, L. Zhang, Beyond a gaussian denoiser:
436–444. Residual learning of deep cnn for image denoising, IEEE Trans. Image Process.
[9] C. Li, R. Cong, J. Hou, S. Zhang, Y. Qian, S. Kwong, Nested network with two-
26 (7) (2017) 3142–3155.
stream pyramid for salient object detection in optical remote sensing images,
[38] J. Zhang, J. Pan, J. Ren, Y. Song, L. Bao, R.W. Lau, M.-H. Yang, Dynamic scene
IEEE Trans. Geosci. Remote Sens. 57 (11) (2019) 9156–9166.
deblurring using spatially variant recurrent neural networks, in: Proceedings of
[10] C. Guo, C. Li, J. Guo, R. Cong, H. Fu, P. Han, Hierarchical features driven
the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp.
residual learning for depth map super-resolution, IEEE Trans. Image Process. 28
(5) (2019) 2545–2557. 2521–2529.
[11] C. Li, C. Guo, J. Guo, Underwater image color correction based on weakly [39] H. Dong, J. Pan, L. Xiang, Z. Hu, X. Zhang, F. Wang, M.-H. Yang, Multi-scale
supervised color transfer, IEEE Signal Process. Lett. 25 (3) (2018) 323–327. boosted dehazing network with dense feature fusion, in: Proceedings of the
[12] C. Li, H. Fu, R. Cong, Z. Li, Q. Xu, NuI-Go: Recursive non-local encoder-decoder IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp.
network for retinal image non-uniform illumination removal, in: Proceedings of 2157–2167.
the 28th ACM International Conference on Multimedia, 2020, pp. 1478–1487. [40] J.-Y. Zhu, T. Park, P. Isola, A.A. Efros, Unpaired image-to-image translation using
[13] C. Li, R. Cong, Y. Piao, Q. Xu, C.C. Loy, RGB-D salient object detection with cycle-consistent adversarial networks, in: Proceedings of the IEEE International
cross-modality modulation and selection, in: European Conference on Computer Conference on Computer Vision, 2017, pp. 2223–2232.
Vision, Springer, 2020, pp. 225–241. [41] H. Blasinski, T. Lian, J. Farrell, Underwater image systems simulation, in:
[14] L. Chongyi, G. Chunle, G. Jichang, H. Ping, F. Huazhu, R. Cong, PDR-net: Imaging Systems and Applications, Optical Society of America, 2017, pp.
Perception-inspired single image dehazing network with refinement, IEEE Trans. ITh3E–3.
Multimed. 22 (3) (2020) 704–716.
[42] H. Blasinski, J. Farrell, A three parameter underwater image formation model,
[15] C. Guo, C. Li, J. Guo, C.C. Loy, J. Hou, S. Kwong, R. Cong, Zero-reference
Electron. Imaging 2016 (18) (2016) 1–8.
deep curve estimation for low-light image enhancement, in: Proceedings of the
[43] N. Silberman, D. Hoiem, P. Kohli, R. Fergus, Indoor segmentation and support
IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp.
1780–1789. inference from rgbd images, in: European Conference on Computer Vision,
[16] C. Li, C. Guo, W. Ren, R. Cong, J. Hou, S. Kwong, D. Tao, An underwater image Springer, 2012, pp. 746–760.
enhancement benchmark dataset and beyond, IEEE Trans. Image Process. 29 [44] A. Duarte, F. Codevilla, J.D.O. Gaya, S.S. Botelho, A dataset to evaluate
(2019) 4376–4389. underwater image restoration methods, in: OCEANS 2016-Shanghai, IEEE, 2016,
[17] E. Schwartz, R. Giryes, A.M. Bronstein, Deepisp: Toward learning an end-to-end pp. 1–6.
image processing pipeline, IEEE Trans. Image Process. 28 (2) (2018) 912–923. [45] K. Iqbal, R.A. Salam, A. Osman, A.Z. Talib, Underwater image enhancement using
[18] S. Moran, G. Slabaugh, DIFAR: Deep image formation and retouching, 2019, an integrated colour model, IAENG Int. J. Comput. Sci. 34 (2) (2007).
arXiv preprint arXiv:1911.13175. [46] K. Iqbal, M. Odetayo, A. James, R.A. Salam, A.Z.H. Talib, Enhancing the low
[19] J. Hu, L. Shen, G. Sun, Squeeze-and-excitation networks, in: Proceedings of quality images using unsupervised colour correction method, in: 2010 IEEE
the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. International Conference on Systems, Man and Cybernetics, IEEE, 2010, pp.
7132–7141. 1703–1709.
[20] M. Han, Z. Lyu, T. Qiu, M. Xu, A review on intelligence dehazing and color [47] A.S.A. Ghani, N.A.M. Isa, Enhancement of low quality underwater image through
restoration for underwater images, IEEE Trans. Syst. Man Cybern. Syst. (2018). integrated global and local contrast correction, Appl. Soft Comput. 37 (2015)
[21] R. Cui, L. Chen, C. Yang, M. Chen, Extended state observer-based integral sliding 332–344.
mode control for an underwater robot with unknown disturbances and uncertain [48] A.S.A. Ghani, N.A.M. Isa, Underwater image quality enhancement through
nonlinearities, IEEE Trans. Ind. Electron. 64 (8) (2017) 6785–6795. integrated color model with Rayleigh distribution, Appl. Soft Comput. 27 (2015)
[22] S. Anwar, C. Li, Diving deeper into underwater image enhancement: A survey, 219–230.
Signal Process., Image Commun. 89 (2020) 115978. [49] A.S.A. Ghani, N.A.M. Isa, Automatic system for improving underwater image
[23] X. Sun, L. Liu, Q. Li, J. Dong, E. Lima, R. Yin, Deep pixel-to-pixel network contrast and color through recursive adaptive histogram modification, Comput.
for underwater image enhancement and restoration, IET Image Process. 13 (3) Electron. Agric. 141 (2017) 181–195.
(2018) 469–474. [50] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for
[24] P.M. Uplavikar, Z. Wu, Z. Wang, All-in-One Underwater Image Enhancement biomedical image segmentation, in: International Conference on Medical Image
Using Domain-Adversarial Learning, in: CVPR Workshops, 2019, pp. 1–8. Computing and Computer-Assisted Intervention, Springer, 2015, pp. 234–241.
[25] C. Fabbri, M.J. Islam, J. Sattar, Enhancing underwater imagery using generative [51] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in:
adversarial networks, in: 2018 IEEE International Conference on Robotics and Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,
Automation (ICRA), IEEE, 2018, pp. 7159–7165. 2016, pp. 770–778.
[26] S. Anwar, C. Li, F. Porikli, Deep underwater image enhancement, 2018, arXiv [52] H. Zhao, O. Gallo, I. Frosio, J. Kautz, Loss functions for image restoration with
preprint arXiv:1807.03528. neural networks, IEEE Trans. Comput. Imag. 3 (1) (2016) 47–57.
[27] Y. Guo, H. Li, P. Zhuang, Underwater image enhancement using a multiscale [53] J. Johnson, A. Alahi, L. Fei-Fei, Perceptual losses for real-time style transfer and
dense generative adversarial network, IEEE J. Ocean. Eng. (2019). super-resolution, in: European Conference on Computer Vision, Springer, 2016,
[28] Y. Wang, J. Zhang, Y. Cao, Z. Wang, A deep CNN method for underwater image pp. 694–711.
enhancement, in: 2017 IEEE International Conference on Image Processing (ICIP), [54] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale
IEEE, 2017, pp. 1382–1386. image recognition, 2014, arXiv preprint arXiv:1409.1556.

9
Y. Wang, J. Guo, H. Gao et al. Signal Processing: Image Communication 96 (2021) 116250

[55] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, Imagenet: A large-scale [58] N. Wang, Y. Zhou, F. Han, H. Zhu, Y. Zheng, UWGAN: Underwater GAN for
hierarchical image database, in: 2009 IEEE Conference on Computer Vision and real-world underwater color restoration and dehazing, 2019, arXiv preprint
Pattern Recognition, Ieee, 2009, pp. 248–255. arXiv:1912.10269.
[56] P.L. Drews, E.R. Nascimento, S.S. Botelho, M.F.M. Campos, Underwater depth [59] M. Yang, A. Sowmya, An underwater color image quality evaluation metric, IEEE
estimation and image restoration based on single images, IEEE Comput. Graph. Trans. Image Process. 24 (12) (2015) 6062–6071.
Appl. 36 (2) (2016) 24–35. [60] K. Panetta, C. Gao, S. Agaian, Human-visual-system-inspired underwater image
[57] W. Song, Y. Wang, D. Huang, D. Tjondronegoro, A rapid scene depth estimation quality measures, IEEE J. Ocean. Eng. 41 (3) (2015) 541–551.
model based on underwater light attenuation prior for underwater image
restoration, in: Pacific Rim Conference on Multimedia, Springer, 2018, pp.
678–688.

10