1

Survey On: Deblur Removing Mild Blur

Using DeepLearning
Musaib Bin Jabbar
MASTER OF COMPUTER APPLICATION, RAYAT BAHRA UNIVERSITY, MOHALI PANJAB
[email protected]

Abstract — This literature paper survey provides a com-

prehensive overview of the recent advancements in image
deblurring techniques. Image deblurring is a significant task in
--- (1)
image processing that aims to restore sharpness and clarity to
blurry images. The paper covers various approaches to image
deblurring, including traditional methods like blind and non- Here (x, y) are pixel co-ordinates and σ is standard deviation of
blind deconvolution, as well as modern approaches based on gaussian function.
image priors, deep learning, and hybrid methods. The survey Deep learning plays a significant role in image pro-
summarizes the key contributions of each method, their ad- cessing by providing powerful models that can learn and extract
vantages, and their limitations in handling various types of features from images automatically. These models, such as
blurs. The paper also discusses the evaluation metrics and Convolutional Neural Networks (CNNs), can be used for vari-
benchmark datasets used to compare and analyze the perfor- ous tasks such as image classification, object detection, seg-
mance of different methods. Furthermore, the survey highlights mentation, super-resolution, and more. They have outperformed
the current challenges in the field and suggests future research traditional image processing techniques and have become the
directions to improve the effectiveness and efficiency of image state-of-the-art in many tasks.
deblurring techniques. Deep learning allows image processing algorithms to
learn complex representations of images through multiple lay-
Index Terms — image deblurring, deep learning, convolutional ers of artificial neural networks. These representations can cap-
neural networks. ture not only the basic features of an image but also high- level
abstractions, such as the presence of objects and their relation-
1. INTRODUCTIOIN ships. This capability enables deep learning models to perform
image recognition, semantic segmentation, and other computer
BLUR is caused due to various reasons like camera
vision tasks with high accuracy and robustness. Additionally,
movement while capturing the image, subject movement while deep learning algorithms can be trained end-to- end, making it
capturing the image, missed focus, insufficient depth of field easier to scale and fine-tune them to different use cases and
and lens softness. The aim of image deblurring is to remove the domains.
blur from such images in order to get the sharp and clear image In this literature paper survey, we explore various approaches to
from blur images. Image deblurring is the classical problem in image deblurring, which has been an active area of research in
image processing. An Image deblurring in image processing is a computer vision and image processing. We start by introducing
recovering process, it recovers the sharp image from blur image the basic concepts of image deblurring and the challenges asso-
by which is caused by camera motion or object motion. From last ciated with it. We then provide an in-depth review of the state-
few years Image deblurring has attracted attention in image pro- of-the-art methods for image deblurring, including traditional
cessing and computer vision. In mathematical formulation, blur methods such as blind and non-blind deconvolution, as well as
image is denoted as Y, sharp image is denoted as S, K denotes more recent methods that leverage image priors, deep learning-
the blur kernel, N denotes the measurement of noise in image based approaches, and hybrid methods. For each method, we
and * denotes the convolution operation. discuss its underlying principles, strengths, and limitations, and
Y=S*K+N provide a critical analysis of its performance on benchmark da-
tasets. Finally, we highlight the current challenges and opportu-
Image deblurring is estimating S from Y. Parametric nities in image deblurring research, including the need for better
blur kernel is there in some circumstances like motion blur and evaluation metrics, improved handling of complex blur types,
out-of-focus blur. The parametric assumption on blur kernel can and the development of more efficient and effective algorithms.
greatly influence the efficiency of blind image deblurring. The Overall, this paper provides a valuable resource for researchers
gaussian function is mostly used in image deblurring. and practitioners in the field of image deblurring.
 2

2. RELATED WORK
Projection Networks for Super-Resolution Convolutional Neural
A. Convolutional Neural Networks Network), are based on convolutional neural networks (CNNs)
CNN (convolutional neural network) have become a popu- and can achieve state-of-the-art results on various image super-
lar choice for image processing tasks due to their ability to au- resolution benchmarks. These methods are trained on large da-
tomatically learn useful features from the image data and their tasets of high and low- resolution image pairs and use deep learn-
high accuracy in a wide range of tasks. CNNs are important ing algorithms to learn the mapping from low to high-resolution
in image deblurring because they can effectively learn the map- images. Despite the promising results of SISR methods, there are
ping between the blurred image and the sharp image. Image still some challenges that need to be addressed, such as the need
deblurring is a challenging problem because it requires the re- for high computational resources, the difficulty of obtaining large
moval of significant amounts of noise and blur, which can and diverse training datasets, and the limitations of current SR
cause loss of detail and resolution in the final deblurred image. algorithms in preserving fine details in the high-resolution output.
By using a deep learning approach, a CNN can learn the com-
plex mapping between the blurred image and the sharp image, D. Objective
allowing it to effectively remove the blur and recover the details
of the image. The convolutional layers in the network can be The objective of image deblurring using deep learning is to de-
trained to learn features that are robust to the presence of blur velop algorithms that can automatically restore clear and sharp
and noise, allowing the network to produce high-quality images from blurry ones using deep neural networks. Traditional
deblurred images. methods for image deblurring typically rely on hand-crafted fea-
Additionally, CNNs can be trained end-to-end, allowing the tures and assumptions about the underlying blur model, which
network to learn the entire mapping from the input blurred im- can limit their effectiveness and generalizability. Deep learning-
age to the output deblurred image in a single pass. This makes based methods, on the other hand, can automatically learn the
the deblurring process more efficient and eliminates the need most relevant features and blur models from a large amount of
for complex pre-processing or post-processing steps. data, making them more flexible and robust to variations in blur
In conclusion, the ability of CNNs to learn complex mappings types and image content. The key objective of image deblurring
and their high accuracy make them an important tool for image using deep learning is to train a deep neural network that can ef-
deblurring tasks. fectively capture the mapping between blurry and clear images.
This involves designing appropriate loss functions that can meas-
B. Deep Learning ure the quality of the network's output and updating the network
parameters using backpropagation-based optimization techniques.
Deep learning is a subfield of machine learning that has The ultimate goal is to develop a deep learning-based image
had significant impact in the field of image processing. In image deblurring algorithm that can produce high-quality, visually ap-
processing, deep learning algorithms such as Convolutional pealing results on a wide range of images and blur types, with
Neural Networks (CNNs) are used for tasks such as image clas- minimal manual intervention. The use of deep learning for image
sification, object detection, semantic segmentation, and image deblurring has shown promising results in recent years, and sev-
deblurring. eral state-of-the-art methods have been proposed. However, there
In image deblurring, a deep learning model is trained to remove are still many challenges and opportunities for future research in
blur from a blurred image and restore the original, sharp image. this area, including the need for more efficient and effective train-
This is done by learning a mapping from the blurred image to ing methods, better handling of complex blur types, and the de-
the sharp image, which is then used to deblur new images. velopment of methods that can handle large-scale, real-world im-
Overall, deep learning has become a powerful tool for image age datasets. To achieve the objective of image deblurring
processing tasks and is likely to continue to play an important using deep learning, there are several key steps involved in
role in this field in the future. the algorithm's design and implementation. These include
selecting appropriate network architectures and hyperpa-
C. Image Super Resolution
rameters, preparing training and validation datasets, design-
Image super-resolution (SR) is the process of increasing
the resolution of an image while preserving its content. Image ing loss functions that can capture perceptual quality, and
SR is used in a variety of applications, such as computer vision, implementing efficient optimization techniques.
image processing, and graphics. There are several methods for
image super-resolution, including traditional methods, such as 3. LITERATURE SURVEY
bicubic interpolation and single image super-resolution (SISR)
methods, which use machine learning techniques, such as deep Chih-Hung Liang, et-al [1] has proposed a method which is based
learning, to restore high resolution images from low-resolution on deblurring from raw images directly. The dataset used contains
inputs. SISR methods, such as SRCNN (Super-Resolution Con- two types of images, sRGB images and raw images and the model
volutional Neural Network) and DBSRCNN (Deep Back- is efficient to utilize unique characteristics of raw Images for
 3

deblurring process. The author found that the performance of Kusam Lata, et-al [9] has proposed the method in which images
previously present deblurring models can be improved by using are translated based on some conditions using Conditional GANs.
raw images for training the model. Here image to image translation is done to generate high quality
Mauricio Delbracio, et-al [2] has proposed blind image recon- data. Then, hyper-parameter tuning is used to analyze the perfor-
struction method for removing mild image blur from natural mance of the model. This model makes image to image mapping
images. First blur estimation is done, and the blur is removed. easier, and it also calculates loss function. In future, we can de-
The author also found that the proposed model is more efficient velop model for audio and video translation also.
than existing traditional and modern blind images restoration Xiangyu Xu, et-al [10] has proposed a method that stimulates the
methods. The proposed method is more focused on mobile image processing of digital cameras by introducing a new pipe-
phone images. line that produces realistic training data. To prevent information
Mahdi S. Hoseini, et-al [3] has proposed a image deblurring loss in raw images dual convolutional neural network is used.
method which uses one shot convolution filtering that re- The proposed model also helps in color correction by learning a
stores naturally blurred images by directly convolving with spatially variant color transformation. The experiments showed
them. In proposed method Finite Impulse Response (FIR) and that it is easier to recover images with super resolution from raw
Point Spread Function (PSF) is used. For image edge deblur- images as the information loss is less in raw images.
ring they have Gaussian low pass filters to denoise the images. Xin Yang, et-al [11] In this paper, the author has proposed a
Jiangxin Dong, et-al [4] has proposed a method for estimation Deep Recurrent Fusion Network (DRFN) for image super-
of inliers and outliers using deep convolutional neural network resolution. The method uses transposed convolution instead of
which will facilitate the image deblurring process. The model bicubic interpolation for up sampling. To reconstruct the final HR
estimates the confidence map which is used to identify the inli- images, it performs multi-level fusion of different level features
ers and outliers. The proposed method can be used for both extracted from recurrent residual blocks which makes full use of
blind and non-blind image deblurring. potential information for HR image reconstruction.
Chunzhi Gu, et-al [5] has proposed the method which does not
Jiawei Zhang, et-al [12] In this paper, the author proposed a net-
require blur kernel estimation for deblurring process, instead a
work of convolutional neural network (CNN) and recurrent neu-
pair of images blurred images and noisy image are used.
ral network (RNN) for dynamic scene deblurring. Convolutional
Blurred image is sliced into patches, then each patch in blurred
neural network is used to learn weights for RNN and to recon-
image is analyzed and computed with corresponding patch in
struct the final deblurred images. Deconvolutional operator is re-
noisy image to analyze the optical flow of both images. Gaussi-
current neural network RNN. The proposed model is faster and
an mixture model (GMM) is used to identify intensity distribu-
smaller than existing deblurring models based on CNN. The pro-
tion of each patch.
posed method computationally less expensive as compared to ex-
Yuhui Quan, et-al [6] In this paper, the author has proposed a
isting deblurring models.
non-blind deblurring approach based on CNN defined in Gabor
Kai Zhang, et-al [13] In this paper, the proposed method for im-
domain. This method utilizes optimal space frequency, strong
age denoising is a deep convolutional neural network which is
orientation selectivity and complex-valued (CV) representa-
capable of which uses residual learning and batch normalization.
tions for efficient image denoising. Performance of the pro-
The previously available methods are capable of handling the im-
posed method is tested for different noise settings in nonblind
age noise to a specified level, but the proposed method DnCNN
image deblurring. In future, the proposed method can be im-
is capable of handling the image with unknown noise level. The
proved to handle other image recovery tasks.
proposed model is focused on Gaussian noise in the images. Fu-
M. R. Mahesh Mohan, et-al [7] In this paper, the author pro-
ture work is to investigate CNN model for denoising of images
posed a deblurring method for unconstrained digital lens cam-
with complex noise.
eras. This method handles three major problems, view con-
Ying Tai, et-al [14] has proposed a deep Convolutional neural
sistency using a Coherent Fusion Module, scene inconsistency
network (CNN) model, Deep Recursive Residual Network
using Adaptive scale-space and image dependency nature blur
(DRRN) for image super-resolution. A residual unit is recursively
by using advanced ASPP module. A new dataset was also de-
learned in a recursive block which is used to learn the residual
veloped during the process.
image between the high-resolution images and low-resolution
Muhammad Asim, et-al [8] In this paper, the author has pro-
images. The experiments and analysis show that DRRN is a deep,
posed a method for blind image deblurring which uses deep
concise, and superior model than other existing models.
generative networks as prior. The two pretrained generative
Ruomei Yan, et-al [15] has proposed an efficient blur estimation
networks were used. For deblurring, sharp image and blur ker-
method for blind blurs. A pre-trained DNN is used for the train-
nel is to be found first. The proposed method is more efficient
ing purpose in a supervised manner in training stage. For blind
for large blurs and heavy noise. Its computationally expensive
blur estimation GRNN is used in parameter estimation stage. The
to train generative models on large datasets. To train this model
experiments show that the proposed method is more efficient in
we need massive amount of training data. In future, the model
blind blur estimation as compared to already present methods. In
can be upgraded to train on less training data.
addition, the proposed method is also applicable to non-
uniformly blurred images.
 4

LITERATURE SURVEY TABLE:

AUTHOR YEAR METHODOLOGY LIMITATION

Chih-Hung The proposed method is based on deblurring from raw images directly. The dataset used The proposed method is not
Liang, etc 2020 contains both raw images and sRGB images and the model is efficient to utilize unique char- efficient for images containing
acteristics of raw Images for deblurring process. non – uniform blurs.
Mauricio The author proposed blind image restoration method for removing mild blur from natural Proposed method is applicable
Delbracio, etc 2021 images. First blur estimation is done, and the blur is removed. The proposed method is more only for images of size up to
focused on mobile phone images. 12MP.
A deblurring method which uses one shot convolution filtering that restores naturally This method is not able to
Mahdi S. Ho- 2018 blurred images by directly convolving with them. In proposed method Finite Impulse Re- maintain both speed and preci-
seini, etc sponse (FIR) and Point Spread Function (PSF) is used. For image edge deblurring they have sion at same time.
Gaussian low pass filters to denoise the images.
proposed a method for estimation of inliers and outliers using deep convolutional neural Proposed method is time con-
Jiangxin Dong, 2021 network which will facilitate the image deblurring process. The model estimates the confi- suming as it involves iterative
etc dence map which is used to identify the inliers and outliers. The proposed method can be optimization process.
used for bothblind and non-blind image deblurring.
The method does not require blur kernel estimation for deblurring process, instead a pair of This method is dependent on
Chunzhi Gu, etc 2021 images blurred images and noisy image are used. Blurred image is sliced into patches, then optical flow which is its major
each patch in is analyzed and computed with corresponding patch in noisy image to analyze limitation.
the optical flow of both images. Gaussian mixture model (GMM) is used to identify intensi-
ty distribution of each patch.
A non-blind deblurring approach based on CNN defined in Gabor domain which utilizes The proposed method becomes
Yuhui Quan, etc 2021 optimal space frequency, strong orientation selectivity and complex-valued (CV) representa- less effective with the increase
tions for efficient image denoising. Performance of the method is tested for different noise of complex noise in images.
settings in nonblind image deblurring.
The author proposed a deblurring method for unconstrained digital lens cameras. This meth- The method is not applicable
M. R. Mahesh 2021 od handles three major problems, view consistency using a Coherent Fusion Module, scene for low resolution Images
Mohan, etc inconsistency using Adaptive scale-space and image dependency nature blur by using ad- which is degraded due to mo-
vanced ASPP module. A new dataset was also developed during the process. tion blur and down sampling.
Author has proposed a method for blind image deblurring which uses deep generative net- The output of this method is
2020 works as prior. The two pretrained generative networks were used. For deblurring, sharp deblurred low resolution image
Muhammad image and blur kernel is to be found first. The proposed method is more efficient for large which is the major limitation.
Asim, etc blurs and heavy noise. Its computationally expensive to train generative models on large
datasets. To train this model we need massive amount of training data.
In the proposed method images are translated based on some conditions using Conditional The method is more focused on
Kusam Lata, etc 2019 GANs. Here image to image translation is done to generate high quality data. Then, hyper- image translations and is not
parameter tuning is used to analyze the performance of the model. This model makes image applicable for audio and video
to image mapping easier, and it also calculates loss function. translations.
The author proposed a method that stimulates the image processing of digital cameras by It is applicable and is more
Xiangyu Xu, etc 2019 introducing a new pipeline that produces realistic training data. To prevent information loss efficient with raw images and it
in raw images dual convolutional neural network is used. The proposed model also helps in is not applicable for other
color correction by learning a spatially variant color transformation. formats of images.
A Deep Recurrent Fusion Network (DRFN) for image super-resolution which uses trans- The method is more concen-
Xin Yang, etc 2018 posed convolution instead of bicubic interpolation for up sampling. To reconstruct the final trated on image super resolu-
HR images, it performs multi-level fusion of different level features extracted from recurrent tion rather than image deblur-
residual blocks which makes full use of potential information for HR image reconstruction. ring.
Author has proposed a network of convolutional neural network (CNN) and recurrent neural The proposed method is not
Jiawei Zhang, 2017 network (RNN) for dynamic scene deblurring. Convolutional neural network is used to learn applicable for non-uniform
etc weights for RNN and to reconstruct the final deblurred images. Deconvolutional operator is blurs.
recurrent neural network RNN. The proposed model is faster and smaller than existing
deblurring models based on CNN.
The proposed method for image denoising is a deep convolutional neural network which is This method is not efficient for
Kai Zhang, etc 2017 capable of which uses residual learning and batch normalization. The previously available removing complex noise from
methods are capable of handling the image noise to a specified level, but the proposed meth- images.
od DnCNN is capable of handling the image with unknown noise level. The proposed model
is focused on Gaussian noise in the images.
2017 A deep Convolutional neural network (CNN) model, Deep Recursive Residual Network The proposed method works
Ying Tai, etc. (DRRN) for image super-resolution. A residual unit is recursively learned in a recursive only with a single image at a
block which is used to learn the residual image between the HR images and LR images. time.
2016 A pre-trained DNN is used for the training purpose in a supervised manner in training stage. This method is limited to blur
Ruomei Yan, For blind blur estimation GRNN is used in parameter estimation stage. The proposed method estimation only and does not
etc is more efficient in blind blur estimation as compared to already present methods. deblurs the images.
 5

5. CONCLUSION
[7] M. R. Mahesh Mohan, G. K. Nithin, and A. N. Rajagopalan, “Deep Dy-
In conclusion, this project aimed to evaluate the ef- namic Scene Deblurring for Unconstrained Dual-Lens Cameras” IEEE
fectiveness of using CNNs for image deblurring. We have TRANSACTIONS ON IMAGE PROCESSING, VOL. 30, 2021.
[8] Muhammad Asim, Fahad Shamshad, and Ali Ahmed, Member, IEEE
shown that deep learning-based approaches can achieve “Blind Image Deconvolution Using Deep Generative Priors” IEEE
state- of-the-art performance in removing non-uniform blur TRANSACTIONS ON COMPUTATIONAL IMAGING, VOL. 6, 2020.
from images. We learned how to deblur Gaussian blurred [9] Kusam Lata, Mayank Dave, Nishanth K N “Image-to-Image Translation
Using Generative Adversarial Network” 2019 IEEE Conference Record #
images using deep learning and convolutional neural net- 45616.
works. Our experiments demonstrate the potential of CNNs [10] Xiangyu Xu, Yongrui Ma, Wenxiu Sun “Towards Real Scene Super-
Resolution with Raw Images” 2019 IEEE/CVF Conference on Computer
for image deblurring and highlight the importance of careful- Vision and Pattern Recognition (CVPR).
ly designing the network architecture and training procedure. [11] Xin Yang, Haiyang Mei, Jiqing Zhang, Ke Xu, Baocai Yin, Qiang Zhang,
C

Xiaopeng Wei “DRFN: Deep Recurrent Fusion Network for Single-Image

We have also shown that combining different techniques, Super-Resolution with Large Factors” 2018 IEEE TRANSACTIONS ON
such as residual networks and skip connections, can further MULTIMEDIA.
improve the performance of CNNs for image deblurring. In [12] Jiawei Zhang, Jinshan Pan, Jimmy Ren, Yibing Song, Linchao Bao, Ryn-
son W.H. Lau, Ming-Hsuan Yang “Dynamic Scene Deblurring Using Spa-
future work, we plan to evaluate the performance of the net- tially Variant Recurrent Neural Networks” 2018 IEEE/CVF Conference on
work architecturethat we have proposed on a larger and more Computer Vision and Pattern Recognition.
[13] Kai Zhang, Wangmeng Zuo, Senior Member, IEEE, Yunjin Chen, Deyu
diverse dataset of blurred images. Meng, and Lei Zhang “Beyond a Gaussian Denoiser: Residual Learning of
Overall, this project has provided valuable insights Deep CNN for Image Denoising” IEEE TRANSACTIONS ON IMAGE
PROCESSING, VOL. 26, NO. 7, JULY 2017.
into the use of CNNs for image deblurring and highlights the
[14] Ying Tai, Jian Yang, Xiaoming Liu “Image Super-Resolution via Deep
potential for deep learning techniques to tackle complex im- Recursive Residual Network” 2017 IEEE Conference on Computer Vision
age processing problems. and Pattern Recognition.
[15] Ruomei Yan, Ling Shao “Blind Image Blur Estimation via Deep Learning”
2016 IEEE TRANSACTIONS ON IMAGE PROCESSING.
ACKNOWLEDGEMENT

We would like to express our sincere gratitude to all

those who have provided their invaluable support and assis-
tance during the course of this project.
We would like to extend our heartfelt thanks to Dr.
Kamalakshi Naganna, Professor and Head, Department of
Computer Science and Engineering, Sapthagiri College of En-
gineering and our project guide Dr. Praveen Kumar KV,
Professor, Department of Computer Science and Engineering,
Sapthagiri College of Engineering, who has provided constant
guidance, support, and encouragement throughout the project.
Their invaluable insights and suggestions have been instru-
mental in shaping this project.
Finally, we are deeply grateful to our friends and fami-
ly for their constant support and encouragement.

REFERENCES
[1] Chih-Hung Liang, Yu-An Chen, Yueh-Cheng Liu, and Winston H.
Hsu,” Raw Image Deblurring”, IEEE TRANSACTIONS ON MUL-
TIMEDIA, 2020
[2] Mauricio Delbracio, Ignacio Garcia-Dorado, Sungjoon Choi,
DamienKelly, and Peyman Milanfar, “Polyblur: Removing Mild Blur
by Polynomial Reblurring” IEEE transactions on Computional Imag-
ing,Vol 7, 2021
[3] Mahdi S. Hosseini, Member, IEEE, and Konstantinos N. Plataniotis,
“Convolutional Deblurring for Natural Imaging” IEEE TRANSAC-
TION ON IMAGE PROCESSING, SEPTEMBER 2018.
[4] Jiangxin Dong and Jinshan Pan, “Deep Outlier Handling for Image
Deblurring” IEEE TRANSACTIONS ON IMAGE PROCESSING,
VOL. 30, 2021.
[5] Chunzhi Gu, Xuequan Lu, Ying He, and Chao Zhang, “Blur Removal
via Blurred-Noisy Image Pair” IEEE TRANSACTIONS ON IMAGE
PROCESSING, VOL. 30, 2021.
[6] Yuhui Quan, Peikang Lin, Yong Xu, Yuesong Nan, and Hui Ji,
“Nonblind image deblurring via Deep Learning in Complex Field” 2021
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARN-
ING SYSTEMS.
6