Image Colorization using Deep Learning

In recent years, CNNs have emerged as the de facto standard

Abstract—We demonstrate a convolutional neural network-based for solving image classification problems, achieving error rates
system that colorizes black-and-white photographic images lower than 4% in the Image Net Challenge. CNNs contribute
accurately without the need for human intervention. Different much of their success to their ability to learn and discern colors,
network designs, objectives, color spaces, and issue formulations patterns, and shapes within images and associate them with
are investigated. The final classification-based model we design classes of objects. The Characteristics of CNN will naturally
produces colorized images that are substantially more aesthetically coordinate well along with colorized images since their
beautiful than the baseline regression-based model, confirming the constituents can connect easily with the choices of color.
validity of our methods and exposing intriguing future research
possibilities. Computer vision and machine learning approaches
have been used to investigate the automated colorization of black
and white photographs. Our study uses deep learning techniques
to generate a plausible colored photograph of ancient, historically
black and white images without requiring direct human
intervention. A wide range of datasets has been set up to formulate
the process. A CNN model will be set up to convert all the training
images from RGB color space to the Lab color space. The L Fig . 1 – Sample Input Image (Left)
channel will be used as the input to the network and train the
network to predict the ab channels, and the next input L channel
will be combined with the expected ab channels. The model is tested Fig . 2 – Sample Output Image (Right)
and run to generate the result.
II. SCOPE OF THE PROJECT
I. INTRODUCTION In this Project, the scope of our work is to –
Automated Colorization of Black and white images have been
subjecting intense research in the discipline of computer vision ● Detect and recolor a black and white image
and machine learning. Having a fascinating aesthetic AI ● Image Restoration
perspective, this feature has vast practical applications starting
from video restoration to enhancement of images along with Classic deep learning methods are based on the assumption that
improved deduction. the data are vectors to exploit basic operations such as
Here we designed a Systematic - framework - learning convolutions. While this suffices for many signals’
approach for a solution to this problem. We design and build a classification problems such as speech, image, and video
convolution neural network (CNN) that accepts a black and classification/segmentation, in various applications the data
white image as an input and generates a colorized version of have other structures. Image Colorization plays a key role in
the image as its predicted output. in Figure 1 and 2, it shows an many fields like Multimedia Retrieval and Human-Machine
example of a black and white image that undergoes through the Interaction. The objectives of the project are mainly to input a
model and the model generates a colored image of the input black and white image and then detect it and convert it into a
based on the learning of the model through images without no colored image and then display it. The dataset which is used for
further human intervention. the project is OpenCV. OpenCV is the open-source computer
that enables the system to recognize images and patterns to give
desired results. The system supports deep learning frameworks
such as Tensor Flow, Keras, etc.
 III. PROBLEM DEFINITION

The problem of the project states that to automate the colorization

of black & white pictures through a deep neural network. While
it is probably difficult to predict exactly the correct colors due to
the loss of information (a tulip for example can be of many
different colors), the recognition of items within
a photograph (plants, leaves, flowers, water, animals, etc.) should
lead to the real color. Refurbishing the misplaced synonym of a CNN gives us a collection of distributions in 𝑧̂ from the
black and white pixel is shown as the problem of selecting the
foremost color among a set of color pallete. resized input image. In 𝑌̂, 𝑧̂ is represented as annealed-
mean of the distribution corresponding to the ab pair which
IV. METHODOLOGY is written as the transformation of the original distribution
RGB Color space, as well as lab Color space, have three
channels. But it's unlikely the former, the latter encodes color . is the equation of the annealed mean
information differently:
distribution. The Loss function is defined by the neural
● The L channel encodes lightness intensity only
networks while training. During training, it aims to
● The channel encodes green-red.
minimize the loss over the training set. The output of the
● And the b channel encodes blue-yellow
̂
As the L-channel encodes the intensity only, we can opt-out to use
CNN is 𝑍 with a given input image X by transforming all
L-Channel as our grayscale input to the network. It is mandatory color images to their corresponding values in the training
from there to predict a and b channels. From L Channels as the
set. Mathematically, we have to invert the mapping H, so it
given input and ab channels as the predicted ones, we tend to form
the final image as the output. stands like . For comparing standard cross-

entropy loss along with ground truth Z and the estimate 𝑍̂ ,

The entire process can be summed up in the following steps: we use the equation

● Converting all training images from the RGB

color space to the Lab color space.
● Take input through the L channel to the network This loss function able to produce color in a dull texture
and train the network to predict the ab channels. because the distribution of colors in ImageNet is heavy
● Merge input L channel along with the predicted ab around the gray line. By nudging the algorithm to produce
channels.
vibrant colors, we change the loss function to
● Transform the lab image back into RGB.

The input image is rescaled to a particular dimension suppose

224*224. Let the rescaled grayscale input image be X. When the
image is passed through the neural network shown, it gets is the color rebalancing term, which would
transformed to by the neural network. This transformation by G rebalance the loss based on the rarity of the color
the network can be written mathematically as 𝒛̂ = 𝑮(𝑿) class. This helps us to get more vibrant and
The dimensions of 𝑧̂ is H * W * Q, where H(Height) = 56 and saturated colored pictures as output.

W(weight) = 56 of the output of the last convolution layer. 𝑧̂

contains a vector of (Q = 313) for each of the H*W pixels, where
the probability of the pixel belonging to that class is represented
by the pixel value. We aim to find a value for a single pair ab

channel for each probability distribution .

 V. TECHNOLOGY USED
Software Used -
● Spyder(Anaconda)
● TensorFlow
● OpenCV
● Keras
Hardware Used –
In all Kinds of Hardware that supports recent technology

VI. BLOCK DIAGRAM

Consider a black and white image, if we have a close look at it, it

would appear less graceful as the picture is not appealing. Also,
the color features which are accessed by the objects are lost and
found hard to accept. While transforming the picture through the
model accessed by semantics, it predicts the colors as the
prediction is free and any color photo can be used during model
training. Color prediction is multimodal which means several
(Process Flow Diagram of Image Colorization) objects can take on several colors.

VIII. CASE STUDY

(Simplified Step Diagram of Image Colorization)

VII. OUTPUT SCREEN

These images of the output Screen show the given input
image in monochrome in the left window and the given output
image in color in the right window. Colorization can be a
powerful pretext task for a self-supervised feature for
learning, acting as a cross-channel encoder.
 IX. BENEFITS OF THE PROJECT encouragement, support, and guidelines throughout the
development of the project. Words are inadequate in offering
our thanks to the other mates, teachers, and other members at
These projects will provide benefits to these applications in the GNIT, Kolkata for their encouragement and cooperation in
following sectors – carrying out this project work. The guidance and support
Astronomy - The Hubble telescope doesn't use color film. received from all the members who are contributing to this
Telescopic cameras are used to record light from the universe project were vital for the success of this project.
along with special electronic detectors which produce black and
white cosmos images. The matrix corresponding to the image is XI. REFERENCES
sent back to the earth station. Different color filters are applied
and the equivalent matrices are sent back to the earth station [1] Laporte. https://github.com/Laps, 2019.
where they are combined to form a color image. This process is [2] B.Hari, J.Hopkins for object segmentation and fine-grained
cost-effective and tedious which makes our system effective in localization, IEEE Conference
work. [3] K. Simonyan and A. Zisserman. Very deep convolutional
Archeology - Archeology Images captured a few decades ago networks for large-scale image recognition. arXiv
when there were no color cameras are still in grayscale format preprint arXiv:1409.1556, 2014.
and the process to convert them to color requires manual effort. [4] A. Olmos et al. A biologically inspired algorithm for the
Hence this application is chosen so that the process of recovery of shading and reflectance images. Perception,
colorization would contribute to further research. 33(12):1463–1473, 2004.
Electron Microscopy - An electron microscope is used to [5] Scoria. https://github.com/scoria, 2014.
observe objects that cannot be seen with the naked eyes, such as
viruses and bacteria. The electron microscope functions by
passing an electron beam and focusing it through
electromagnets; this beam is either passed through the object or
reflected off the surface of the object on a fluorescent plate.
This electron beam does not transfer color information, and
hence the images captured thereby are gray. Colors are essential
in electron microscopy as they represent vital information.
Hence, our system would make this colorization process simpler.

IX. CONCLUSION AND FUTURE WORK

Through our experiments, we have demonstrated the efficacy and
potential of using deep convolutional neural networks to colorize
black and white images. In particular, we have empirically shown
that formulating the task as a classification problem can yield
colorized images that are arguably much more aesthetically
pleasing than those generated by a baseline regression-based
model, and thus shows much promise for further development.
Our work, therefore, lays a solid foundation for future work.

Moving forward, we have identified several avenues for

improving our current system. To address the issue of color
inconsistency, we can consider incorporating segmentation to
enforce uniformity in color within segments. We can also utilize
post-processing schemes such as total variation minimization and
conditional random fields to achieve a similar end. Finally,
redesigning the system around an adversarial network may yield
improved results, since instead of focusing on minimizing the
cross-entropy loss on a per-pixel basis, the system would learn to
generate pictures that compare well with real-world images.
Based on the quality of results we have produced, the network we
have designed and built would be a prime candidate for being the
generator in such an adversarial network.

X . ACKNOWLEDGMENT
Success of any project depends largely on the encouragement
and guidelines of many others. I take this sincere opportunity to
express my gratitude to the people who have been instrumental
in the successful completion of this project work. We have great
pleasure in presenting the report on Image Colorization. We
take this opportunity to express our sincere thanks to our project
mentor Prof. Tridib Chakraborty, Assistant Professor
Department of Information Technology, GNIT for providing
the technical guidelines and suggestions regarding the line of
work. We would like to express our gratitude for her constant