ACCENTS Transactions on Image Processing and Computer Vision, Vol 4(11)

ISSN (Online): 2455-4707

Review Article
http://dx.doi.org/10.19101/TIPCV.2018.411002

A review on image enhancement with deep learning approach

Amanjot Kaur1* and Gagandeep2
M.Tech Research Scholar, Department of Computer Science, Punjabi University, Patiala, Punjab1
Associate Professor, Department of Computer Science, Punjabi University, Patiala, Punjab 2

©2018 ACCENTS

Abstract
Deep learning is presently a dynamic research zone in machine learning and pattern recognition society. It has
importance in an expansive zone of applications, for example, speech recognition, natural language processing, and
computer vision. The procedures created from deep learning research have just been affecting the exploration of image
enhancement. Images caught in low-light conditions more often than not experience the ill effects of low difference,
which expands the trouble of consequent PC vision errands in an incredible extent. Many applications mean to upgrade
brilliance, differentiate and decrease noisy content from the images in an on-load up constant manner. We demonstrate
that a variation of the stacked-inadequate denoising auto encoder can figure out how to adaptively improve and denoise
from artificially obscured and disruption included preparing cases.

Keywords
Deep learning, Image enhancement, Image denoising, Auto encoder, Neural networks.

1.Introduction Recently, deep learning-based methodologies

Great quality images and recordings are vital to basic increased huge footing as they are appeared to beat
robotized and human-level basic leadership for other cutting edge machine learning equipment in
errands extending from security applications, military numerous computer vision applications, including
missions, way wanting to medicinal diagnostics and object detection, image classification and scene
business recommender frameworks. Spotless, understanding. While neural systems have been
superior quality images caught by camera generally examined for images denoising errands,
frameworks give better proof to all around educated there are no current works utilizing deep systems to
strategy. Be that as it may, cost imperatives constrain both upgrade and denoise images taken in
extensive scale uses of such frameworks; therefore ineffectively enlightened situations. We approach the
reasonable sensors are normally utilized. Inadequate issue of low-light images improvement from a
lighting, still low sensor quality, deliver images noisy portrayal learning point of view utilizing deep auto
that may hinder Intelligence, surveillance and encoders prepared to pick up basic signals includes in
reconnaissance (ISR) missions and business low-light images and adaptively light up and denoise.
ventures[1]. The strategy exploits nearby way quick complexity
change like the works in with the end goal that
Noise is a major issue in estimating light. Regardless enhancements are done in respect to nearby
of how great the sensors are, there is disruption in neighbours to counteract over amplifing splendid
images, particularly in low-light conditions or in the pixels. A similar system is prepared to learn noise
dark environment. Image denoising is the procedure structures so as to deliver brighter, denoised images.
of decreasing undesired disruption in images. An
image denoising methodology takes a noisy image as 2.Related work
input and yields an image where the noise has been 2.1Contrast enhancement
decreased. There are many applications remain to There are notable contrast enhancement techniques,
profit by changes in image enhancement calculations. for example, enhancing image contrast by histogram
equalization [2]. Histogram stretching techniques and
its variants like brightness preserving bi-histogram
equalization(BBHE) and quantized bi-histogram
equalization (QBHE) gained prominence to enhance
*Author for correspondence on the artifacts of histogram equalization[3,4]. Along
16
 ACCENTS Transactions on Image Processing and Computer Vision, Vol 4(11)

these, an optimization technique, OCTM was functions like the gamma function to improve image
presented for mapping the contrast-tone of an image contrast. A low-light model is proposed and bright
with the utilization of mathematical transfer function channel prior is used to calculate transmission in our
[5]. Contrast-limiting adaptive histogram equalization low-light model [7]. In [8], LLCNN learns to
(CLAHE) has a place with the class of histogram- adaptively enhance image contrast and increase
stretching techniques and serves to point of image brightness. A special module is designed to
confinement to the degree of contrast enhancement help training and improve the performance. Also find
result of histogram equalization [6]. However, this that SSIM loss suits better for low-light image
requires weighting of some area learning as well as enhancement task. Some comparison on deep
an associated complexity increment. Accessible plans learning models shown in Table 1.
additionally investigated utilizing non-linear

Table 1 Comparison on different deep learning models and other metrics

Name Model Noise level Patch size SSIM PSNR
Zhang K et al (2017) DCNN1 =25 35x35 0.819 29.02
Lore KG et al (2017) LLNet2 =18 17x17 0.92 20.35
S-LLNet3 0.87 18.07
Tao L et al (2017) DCNN - 41x41 0.846 -
1
Denoising Convolutional Neural Network (DnCNN)
2
Low Light Net (LLNet)
3
Staged Low Light Net(S-LLNet)

2.2Image denoising Data is gone through each layer, with the yield of the
Image denoising tasks have been explored using previous layer giving contribution to the following
BM3D[9], non-linear filters[10] and KSVD[11]. layer. The main layer in a system is known as the
Using deep learning, authors in [12] exhibited the info layer, while the latter is called a yield layer.
idea of denoising autoencoders for learning features Every one of the layers between the two are
from noisy images while [13] applied convolutional mentioned to as hidden layers as appeared in the
neural networks to denoise natural images. The Figure 1. Each layer is commonly a straightforward,
network was applied for deblurring and uniform calculation containing one sort of actuation
inpainting[14]. Stacked denoising auto encoders were work. Learning can be directed, unsupervised or
used in[15] to reconstruct clean images from noisy hybrid.
images by exploiting the encoding layer of the
multilayer perceptron (MLP). In expansion, authors
in[16] implemented an adaptive multi-column
architecture to robustly denoise images by training
the model with various different kinds of noise and
testing on images with arbitrary noise levels and type.
In [17] author look over the creation of feed-forward
denoising convolutional neural networks (DnCNNs)
to grab the progress in very deep architecture,
learning algorithm, and regularization procedure into
image denoising. Specifically, residual learning and
batch normalization are used to speed up the training
process as well as boosting up the denoising
Figure 1 A feed-forward neural network
performance.
4.Image enhancement
3.Deep learning Image enhancement is the way toward altering
Deep Learning, as a branch of Machine Learning,
computerized images with the goal that the outcomes
utilizes calculations to process information and
are more reasonable for show or further image
impersonate the reasoning procedure, or to create
investigation. Image enhancement should be possible
deliberations [18]. Deep learning (DL) utilizes layers
by evacuating noise, image sharpening, or lighting up
of calculations to process information, comprehend
an image, making it less demanding to distinguish the
human discourse, and outwardly perceive objects.
17
Kaur et al.

key highlights[19]. Image enhancement algorithms the procedure utilizes an obscured, or "unsharp",
include deblurring, filtering, and contrast methods. negative image to make a mask of the original
Deep learning utilizes neural systems to learn helpful image. The unsharped mask is then joined with the
portrayals of highlights straightforwardly from image positive (original) image, constructing an image
information. Neural systems are pretrained to that is less blurred than the original. The
distinguish and remove different sorts of disruption subsequent image, in spite of the fact that clearer,
from images and improve the images. might be a less precise portrayal of the image's
subject.
4.1Methods in image enhancement  Deep neural network: Execute image processing
There are certain methods for Image Enhancement undertakings, for example, removing noise from
some of them are listed below [1]: images and constructing high-resolution images
 Histogram matching: Histogram matching is the from low-resolutions images, utilizing
change of a images with the goal that its histogram convolutional neural networks. Deep learning
coordinates a predefined histogram. The surely utilizes neural networks to learn valuable
understood histogram equalization strategy is an portrayals of highlights straightforwardly from
exceptional case in which the predetermined information. For instance, you can utilize a
histogram is consistently appropriated. pertained neural network to recognize the images
 Contrast-limited adaptive histogram and remove various type of noise from images.
equalization (CLAHE): It is used to enhance the
contrast of the grayscale image assumed as I by 4.2Types of noise
transforming. CLAHE works on small regions in Gaussian noise: It is statistical noise having a
the image called tiles, rather than the whole probability density function (PDF) equal to that of
images[1]. Contrast of every tile is enhanced, the normal distribution, which is also known as the
therefore, the histogram of the output region Gaussian distribution[20]. In other words, the values
approximately matches the histogram predefined that the noise can take on are Gaussian-distributed.
the „Distribution‟ parameter. The probability density function p of a Gaussian
 Wiener filter: Wiener filter is a filter used to random variable z is given in (1):
create a gauge of a coveted or target arbitrary
process by linear time-invariant (LTI) filtering of p(g) = (1)
√
an observed noisy process, accepting known Where z represents the gray level, the mean value
stationary signal and noise spectra, and added
and the standard deviation.
substance noise. The Wiener filter limits the mean
square error between the evaluated random process
Salt and pepper noise: is a form of noise sometimes
and the desired procedure. seen on images. It is also known as impulse
 Median filter: The median filter is a nonlinear noise[20]. This noise can be caused by sharp and
computerized filtering method, regularly used to sudden disturbances in the image signal. It presents
expel noise from a image. Such noise reduction is itself as sparsely occurring white and black pixels.
a common pre-processing step to enhance the The Probability density function of a salt and pepper
results of later processing for example, edge noise is given in (2):
recognition on an image. Median filtering is
broadly utilized as a part of digital image
processing in light of the fact that, under specific p(z) = { (2)
conditions, it preserves edges while removing
noise.
 Linear contrast adjustment: In this the contrast Where, z represents the gray level, a represents the
adjustment block changes the contrast of an image bright region and b represents the dark region.
by linearly scaling the pixel values amongst lower
and upper limits. Pixel values that are below or Poisson noise: The appearance of this noise is seen
above this range are saturated to the lower or upper due to the statistical nature of electromagnetic waves
limit value, individually. such as x-rays, visible lights and gamma rays. The x-
 Unsharp mask filtering: Unsharp masking ray and gamma ray sources emitted number of
(USM) is an image sharpening method, frequently photons per unit time [20]. These rays are injected in
accessible in digital image processing software. patient‟s body from its source, in medical x rays and
The "unsharp" of the name gets from the way that gamma rays imaging systems. These sources are
18
 ACCENTS Transactions on Image Processing and Computer Vision, Vol 4(11)

having random fluctuation of photons. Result [3] Krutsch R, Tenorio D. Histogram Equalization. Free
gathered image has spatial and temporal randomness. Semicond Inc.
This noise is also called as quantum (photon) noise or a. http://cache.freescale.com/files/dsp/doc/app_note/AN4
shot noise. This noise obeys the Poisson distribution 318.pdf. Accessed 26 March 2018.
[4] Kaur M, Kaur J, Kaur J. Survey of contrast
and is given in (3): enhancement techniques based on histogram
p(k) = (3) equalization. International Journal of Advanced
Computer Science and Applications. 2011; 2(7):137-
41.
Where, k takes value 0,1,2.., λ represents the average [5] Wu X. A linear programming approach for optimal
no. of events per interval, e=2.71 and k! is the contrast-tone mapping. IEEE Transactions on Image
factorial of k. Processing. 2011; 20(5):1262-72.
[6] Pisano ED, Zong S, Hemminger BM, DeLuca M,
4.3Challenges in image enhancement Johnston RE, Muller K, et al. Contrast limited
Image enhancement itself is a major test in image adaptive histogram equalization image processing to
improve the detection of simulated spiculations in
handling condition. An image might be low quality
dense mammograms. Journal of Digital Imaging.
since its difference is low, or it is noisy, or it is 1998; 11(4):193-200.
obscured. Numerous calculations have been figured [7] Tao L, Zhu C, Song J, Lu T, Jia H, Xie X. Low-light
to evacuate these corruptions. The test is the way to image enhancement using CNN and bright channel
evacuate corruptions without influencing the signal prior. In international conference on image processing
[21]. Noise decrease calculations include nearby 2017 (pp. 3215-9). IEEE.
averaging or smoothing which will blur the edges in [8] Tao L, Zhu C, Xiang G, Li Y, Jia H, Xie X. LLCNN:
the images. Noise levels are hard to predict. a convolutional neural network for low-light image
Therefore, removing such nosies is a challenging enhancement. In visual communications and image
processing 2017 (pp. 1-4). IEEE.
task.
[9] Dabov K, Foi A, Katkovnik V, Egiazarian K. BM3D
image denoising with shape-adaptive principal
5.Conclusion and future work component analysis. In signal processing with
Deep Learning has emerged as a new field of adaptive sparse structured representations 2009.
machine learning, and has obtained extensive [10] Chan RH, Ho CW, Nikolova M. Salt-and-pepper noise
interests in different research area. In contrast to removal by median-type noise detectors and detail-
traditional methods for image enhancement, Deep preserving regularization. IEEE Transactions on
Image Processing. 2005; 14(10):1479-85.
Learning has great advantage of inherently providing
[11] Elad M, Aharon M. Image denoising via sparse and
success in denoising the image. Still deep learning redundant representations over learned dictionaries.
algorithms need to be trained more to get more IEEE Transactions on Image Processing. 2006;
accurate results. Some possible improvements and 15(12):3736-45.
future research directions are to train the models with [12] Vincent P, Larochelle H, Bengio Y, Manzagol PA.
the combination of different noise types that are Extracting and composing robust features with
robust and adaptive in the low light situations such as denoising autoencoders. In proceedings of the
dusty and foggy sights. Training with poisson noise international conference on machine learning 2008
to simulate more realistic scenarios. (pp. 1096-103). ACM.
[13] Jain V, Seung S. Natural image denoising with
convolutional networks. In advances in neural
Acknowledgment information processing systems 2009 (pp. 769-76).
None. [14] Xie J, Xu L, Chen E. Image denoising and inpainting
with deep neural networks. In advances in neural
Conflicts of interest information processing systems 2012 (pp. 341-9).
The authors have no conflicts of interest to declare. [15] Burger HC, Schuler CJ, Harmeling S. Image
denoising: can plain neural networks compete with
References BM3D? In conference on computer vision and pattern
[1] Lore KG, Akintayo A, Sarkar S. LLNet: a deep recognition 2012 (pp. 2392-9). IEEE.
autoencoder approach to natural low-light image [16] Agostinelli F, Anderson MR, Lee H. Adaptive multi-
enhancement. Pattern Recognition. 2017; 61:650-62. column deep neural networks with application to
[2] Trahanias PE, Venetsanopoulos AN. Color image robust image denoising. In advances in neural
enhancement through 3-D histogram equalization. In information processing systems 2013 (pp. 1493-501).
international conference on image, speech and signal [17] Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond
analysis 1992 (pp. 545-8). IEEE. a Gaussian denoiser: Residual learning of deep CNN

19
Kaur et al.

for image denoising. IEEE Transactions on Image Amanjot Kaur received the B.Tech
Processing. 2017; 26(7):3142-55. degree in Computer Science and
[18] Chen XW, Lin X. Big data deep learning: challenges Engineering from MIMIT Malout,
and perspectives. IEEE Access. 2014; 2:514-25. Punjab with 76% in 2016. She is
[19] Cheng HD, Shi XJ. A simple and effective histogram Currently pursuing M.Tech from
equalization approach to image enhancement. Digital Punjabi University, Patiala. Her area of
Signal Processing. 2004; 14(2):158-70. research is Image enhancement in Deep
[20] Boyat AK, Joshi BK. A review paper: noise models in Learning.
digital image processing. Signal & Image Processing: Email: [email protected]
An International Journal. 2015; 6(2): 63-75.
[21] Huang TS, Aizawa K. Image processing: some Dr. Gagandeep received her Ph.D
challenging problems. Proceedings of the National degree in Computer Science from
Academy of Sciences. 1993; 90(21):9766-9. Punjabi University, Patiala in 2012 on
the topic “Analysis and Development
of Testing Techniques for Component-
based Software Systems”. Currently,
she is working as Associate Professor
in the Department of Computer
Science, Punjabi University, Patiala. She has more than 20
years of teaching and research experience. She is currently
involved in research work on data security, virtualization
techniques and computational offloading mechanism for
cloud and mobile computing.

20