Cartooniation Using White-box Technique in Machine Learning

Harshitha R,Kavya S Muttur,Prof.Jyothi Shetty

Dept. Computer Science,RV College of Engineering,Bangalore

Abstract: In this paper, we propose a the recent success of Instagram, the

solution to transforming photos of real-world popularity of simple and fun photo effects
scenes into cartoon style images, which is apps has been on the rise. The mobile
valuable and challenging in computer vision platform presents a unique arena for these
andcomputer graphics.The learning applications by connecting users with both
objectives of our method are separately based the means to capture images, and the
on each extracted representations, making computational power to perform
our frame-work controllable and sophisticated processing on these images. By
adjustable.By observing the cartoon painting the way , manually recreating real-world
behavior and consulting artists, we propose to scenes in cartoon styles is very laborious and
separately identify three white-box involves substantial artistic skills. To obtain
representations from images: the surface high-quality cartoons, artists have to draw
representation that contains a smooth surface every single line and shade each color region
of cartoon images, the structure of target scenes. Meanwhile, existing image
representation that refers to the sparse color- editing software/algorithms with standard
blocks and flatten global content in the features cannot produce satisfactory results
celluloid style workflow, and the texture for cartoonization. Therefore, specially
representation that reflects high frequency designed techniques that can automatically
texture, contours, and details in cartoon transform real-world photos to high-quality
images. A Generative Adversarial Network cartoon style images are very helpful and for
(GAN) framework is used to learn the artists, tremendous amount of time can be
extracted representations and to cartoonize saved so that they can focus on more creative
images.Experimental results show that our work. Such tools also provide a useful
method is able to generate high-quality addition to photo editing software such as
cartoon images from real-world photos and Instagram and Photoshop.We propose to
outperforms state-of-the-art methods. decompose images into several cartoon
representations, and list them as
1.Introduction
follows:Firstly, we extract the surface
Cartoons are an artistic form widely used in representation to represent the smooth
our daily life. In addition to artistic interests, surface of images.Secondly, the structure
their applications range from publication in representation is proposed to effectively seize
printed media to storytelling for children’s the global structural information and sparse
education. Like other forms of artworks, color blocks in celluloid cartoon
many famous cartoon images were created style.Thirdly, we use the texture
based on real-world scenes. However With representation to contain painted details and
edges.The separately extracted cartoon
representations enable the cartooniaztion
problem to be optimized end-to-end within a
Generative Neural Networks (GAN)
framework, making it scalable and
controllable for practical use cases and easy
to meet diversified artistic demands with
taskspecific fine-tuning. We test our method Image Smoothening: To extract smooth and
on a variety of real world photos on diverse cartoon resembling surfaces from
images, Guided filters are used. A guided
scenes in different styles.We also show that
filter is an advanced version of Bilateral
our method outperforms previous stateof-the-
filters with better near the edge behavior.
art methods through qualitative experiments,
quantitative experiments, and user studies.
The goal is simply removing/significantly
decreasing the noise and obtaining useful
2.Proposed Approach image structures. The filtering output of the
guided filter is an optimal linear transform of
Preprocessing an input image. Following the approach of
Bilateral filters it retains smoothing property
Along with the proposed three-step approach, and in addition, is free from gradient reversal
preprocessing is an important part of our artifacts.
model. It helps to smoothen the image, filter
the features, converting it to sketches, and
translating the output from a domain to
another. After implementing these related
work we can be sure that the output generated
by our model will give us the best output that
retains the highest quality features.

Super-pixel and Structure Extraction: This

method is used to divide the image into Non-photorealistic Rendering: It helps to
regions and defining a predicate for convert images into artistic styles such as
measuring the boundary between two regions. sketching, painting, and water-coloring. To
Based on the predicate segmentation, an expand its functionality we use it with Neural
algorithm is developed whose decision is Style Transfer Methods that helps to sum up
based on a greedy technique but still helps to the style of one image and another. The
satisfy global properties. After identification combined piece of code helps to mark
of contours, we implement Gradient semantic edges while segregating image
Ascent to initialize the image with rough details. But in the “White box cartoonization”
clusters and iteratively amend the clusters method a single image is utilized and learns
until convergence. Advancing our process, to cartoonist features from a set of animated
develop a cartoon-like segmentation method visuals allowing our model to produce high-
we use the Felzenszwalb algorithm that
helps us to seize global content information
and produce practically usable results for quality output on diverse cases.
celluloid style cartoon workflows.
 Image-to-Image Translation: The drawback
with GAN is, it only works for given training
data, but paired training data isn’t always
available. To overcome the drawback we
employ cycleGAN where the goal is to
translate an image from a source domain X to
a target domain Y even in absence of paired
training data.

Generative Adversarial Network: It is an

image synthesizer that helps to generate new
data using joint probability. To generate new
images it uses Generator and Discriminator.
The generator makes images and
Discriminator checks images to be real or
fake and then sends feedback to the generator
thus asking him to generate better data. The
more both networks are trained, the better
images we get. 3.Full Model

The input image is dissolved in three parts

wiz Surface Representation, Structural
Representation,and TextureRepresentation.
A GAN model with a generator G and two
discriminators Ds and Dt are introduced. The
goal of Ds is to characterize surface features
extracted from model outputs and cartoons,
whereas Dt is responsible for separate textural
information from model outputs and cartoons.
To pluck high-level features and to impose a
spatial constraint on global content between Performance Analysis
outputs and provided paired cartoons we use
pre-trained VGGNetwork. Our model is the fastest among four methods
on all devices and all resolutions, and has the
smallest model size. Especially, our model
4.Experimental Results can process a 720*1280 image on GPU within
only 17.23ms, which enables it for real-time
We implement our GAN method with High-Resolution video processing tasks. We
TensorFlow.A discriminator Ds is proposed apply our model on diverse real-world scenes,
to verify whether result and paired cartoon including natural landscape, city views,
images have similar surfaces, and regulate the people, animals, and plants.
generator G to learn the information stored in
the extracted surface representation.we
implement Felzenszwalb algorithm to
segment the areas.To impose a spatial
constraint on global content between outputs
and provided paired cartoons we use pre-
trained VGGNetwork.

Evaluation metric

In qualitative experiments, we present results

with details of four different methods and
original images, as well as qualitative
analysis. In quantitative experiments, we use
Frechet Inception Distance (FID) to evaluate
the performance by calculating the distance
between source image distribution and target
image distribution. In the user study,
candidates are asked to rate the results of
different methods between 1 to 5 in cartoon
quality and overall quality. Higher scores
mean better quality.

Experimental Dataset

Human face and landscape data are collected

for generalization on diverse scenes. The data
contains real-world photos and cartoon
images, and the test data only includes real-
world photos. All the images are resized and
cropped to 256×256.Photos are downloaded
from Internet, used for testing.
5.Conclusion [3] Zengchang Qin, Zhenbo Luo, Hua Wang,
" Autopainter: Cartoon Image Generation
Our project can generate high-quality from Sketch by Using Conditional Generative
cartoonized images and videos from real Adversarial Networks”, International
world photos/video. While processing image Conference on Image Processing, 2017
it go through different stages and modules.
Which includes the surface representation, the [4] J. Bruna, P. Sprechmann, and Y. LeCun.,
structure representation and the texture “Superresolution with deep convolutional
representation. The style of image, weight sufficient statistics” In International
,output all will be controlled by each Conference on Learning Representations
representation function.Corresponding image (ICLR), 2016
processing modules are used to extract three
representations for network training, and [5] K. Beaulieu and D. Dalisay, "Machine
output styles could be controlled by adjusting LearningMastery", Machine Learning
the weight of each representation in the loss Mastery, 2019. [Online].
function.
[6] M.-E. Nilsback and A. Zisserman,
6.Future work “Automated flower classification over a large
number of classes,” in Proceedings of the
Meanwhile, existing image editing Indian Conference on Computer Vision
software/algorithms with standard features
cannot produce satisfactory results for [7] Graphics and Image Processing, Dec
CartooniZation. Therefore, specially 2008.T.-Y. Lin, M. Maire, S. Belongie, J.
designed techniques that can automatically Hays, P. Perona, D.Ramanan, P. Dollar, and
transform real world photos to high-quality C. L. Zitnick, “Microsoftcoco: Common
cartoon style images are very helpful and for objects ´ in context,” in European conference
artists, tremendous amount of time can be on computer vision.
saved so that they can focus on more creative
work. Such tools also provide a useful [8] J.-Y. Zhu, T. Park, P. Isola, and A. A.
addition to photo editing software such as Efros, “Unpaired image to image translation
Instagram and Photoshop. using cycle-consistent adversarial networks,”
arXiv preprint arXiv:1703.10593, 2017.
References
[9] S. Benaim and L. Wolf, “One-sided
[1] Y. Chen, Y.-K. Lai, Y.-J. Liu, unsupervised domain mapping,” arXiv
"CartoonGAN:GenerativeAdversarial preprint arXiv:1706.00826,2017.
Network for photo cartoonization",
International Conference on Image [10] P. Isola, J Wang, Xinrui, and Yu, Jinze.
Processing, 2018 Learning to Cartoonize Using White-Box
Cartoon Representations. IEEE/CVF
[2] Y. Chen, Y.-K. Lai, Y.-J. Liu, Conference on Computer Vision and Pattern
"Transforming photos to comics using Recognition (CVPR)
convolutional neural networks",International
Conference on Image Processing, 2017