Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

CHAPTER 1
INTRODUCTION

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 1

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

1. INTRODUCTION
In this era, technology upgrades to its maximum with the help of creativity and
innovation, with such an ideas in the field of an ANN, the basic module is said to be
image processing stream so that most of the systems will map the inputs to its outputs
with varied uncertainty logic. The image will be considered as the digital formation
and it will be decimated to its corresponding bits. The classification of image or
video in the existing systems seems difficult due to its methodology works with the
file name search and not the content inside it. Depending upon the query given by the
user the ANN should have to classify the content with various attributes.
Our proposed algorithm deal with Deep Learning methods, in which it
confines each and every data, learns the contents by separating its features to the deep
bottom. The database itself maintains a separate individual data centre that will
contain a finite most significant amount of features. Deep learning method shows its
maximum performance to its extent and plays a smart extraction of the content from
the data, which is on process. Deep learning is one of the classifications of soft
computing phenomenon in which extraction of data from millions of segregated
images can be retrieved using this phenomenon. The retrieval performance of a
content-based image retrieval system crucially depends on the feature representation
and similarity measurement, which have been extensively studied by multimedia
researchers for decades Although a variety of techniques have been proposed, it
remains one of the most challenging problems in current content-based image
retrieval (CBIR) research, which is mainly due to the well-known“ semantic gap”
issue that exists between low-level image pixels captured by machines and high-level
semantic concept perceived by humans. From a high-level perspective, such challenge
can be rooted to the fundamental challenge of artificial intelligence (AI) that is, how
to build and train intelligent machines like human to tackle real-world tasks (Fig. 1).
Machine learning is one promising technique that attempts to address this challenge in
the long term. Recent years have witnessed some important advanced new techniques
in machine learning. Deep learning is the part of learning, which includes a family of
machine learning algorithms that attempt to model high-level abstractions in data by
employing deep architectures composed of multiple non-linear transformations [10].

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 2

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

machine

Fig. 1.1 CBIR systems with deep learning

Unlike traditional machine learning techniques that are often using “shallow”
architectures, deep learning mimics the human brain that is organized in a deep
architecture and processes information through multiple stages of transformation and
representation. By exploring deep architecture features at multiple levels of abstracts
from data automatically, deep learning methods allow a system to learn complex
functions that directly map raw sensory input datum to the output, without relying on
human-crafted features using domain knowledge. Many recent studies have reported
encouraging results for applying deep learning techniques to a variety of applications,
including speech recognition, object recognition, and natural language processing,
among others. Inspired by the successes of deep learning, in this paper, we attempt to
explore deep learning methods with application to CBIR tasks. Despite much research
attention of applying deep learning for image classification and recognition in
computer vision, there is a still limited amount of attention focusing on the CBIR
applications. In the proposed method, we investigate deep learning methods for
learning feature representations from the images and their similarity measures towards
CBIR tasks By human-centered we mean systems which consider the behavior and
needs of the human user. As noted earlier, the foundational areas of MIR were often
in computing-centric fields. However, since the primary goal is to provide effective
browsing and search tools for the user, it is clear that the design of the systems should
be human-centric. There have been several major recent initiatives in this direction
such as user understanding, experiential computing, and affective computing. One of
the most fascinating studies was done on whether organization by similarity assists
image browsing. The users were asked to illustrate a set of destination guide articles
for a travel Web site. The similarity by visual content view was compared with a text
caption similarity view. In 40 of the 54 searches, users chose to use the text caption
view with comments such as “it gave me a breakdown of the subject.” In many cases,

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 3

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

the users began with the text caption view to ensure sufficient diversity. Also, it was
noted by the users that they would want both possibilities simultaneously. In another
experiment, the visual similarity view was compared with a random set. Most users
were slightly more satisfied with the visual similarity view, but there was one user
who preferred the random images view. Specifically, the visual similarity view was
preferred in 66% of the searches.
By understanding user types, it is clear that the current work has not addressed the full
plurality of image and user types and that a broad evaluation is important. In specific
cases, there has been niche work such as the use of general purpose documentary
images by generalist and specialist users and the use of creative images by specialist
users. Other interesting studies have been done on the process of managing personal
photograph collections describe a concise analysis of methodologies for interactive
retrieval of color images which includes guidelines for selecting methods based on the
domain and the type of search goal. Some give useful insights into how users apply
the steps of indexing, filtering, browsing, and ranking in video retrieval. Usage
mining in large multimedia databases is another emerging problem. The objective is
to extract the hidden information in user behaviors on large multimedia databases.

 Machine Learning
Machine learning (ML) is the study of computer algorithms that improve
automatically through experience and by the use of data. It is seen as a part
of artificial intelligence. Machine learning algorithms build a model based on sample
data, known as "training data", in order to make predictions or decisions without
being explicitly programmed to do so. Machine learning algorithms are used in a wide
variety of applications, such as in medicine, email filtering, speech recognition,
and computer vision, where it is difficult or unfeasible to develop conventional
algorithms to perform the needed tasks.
A subset of machine learning is closely related to computational statistics,
which focuses on making predictions using computers; but not all machine learning is
statistical learning. The study of mathematical optimization delivers methods, theory
and application domains to the field of machine learning. Data mining is a related
field of study, focusing on exploratory data analysis through unsupervised learning. In

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 4

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

its application across business problems, machine learning is also referred to

as predictive analytics.
1.1 Problem Statement
The incomplete annotation issue in text based image retrieval will degrade the
retrieval performance of the searching process. Query by Image Retrieval (QBIR) has
evolved into a necessary module in which the contents of the images are extracted to
search the images from the database. But CBIR system also faces many challenging
problems because of the large volume of the database constrain, the difficulty in both
people and computer understanding the images, the difficulty of creating a query and
the issue of evaluating results properly. In our method, we explore an alternative
strategy for searching an image database in which the content is expressed in terms of
an image and its multiple features are extracted using different image feature
extraction algorithms. These features are analyzed with the features of image database
and the most similar images are retrieved using an efficient index based sorting
algorithm. Specifically, we aim to address the following open research queries:
(i) Are deep learning techniques effective for learning good feature representations
from images to tackle CBIR tasks?
(ii) How much improvement can be achieved by deep learning techniques when
compared with traditional features crafted by experts in multimedia and computer
vision?
(iii) How to apply and adapt an existing deep learning model trained in one domain to
a new content based image retrieval task in another domain effectively?
A supervised learning is mostly used for categorization and classification of
digital images. Supervised learning is dependent on labeled datasets, and in some
cases, when there are too many images, it is difficult to manage the labeling process.
To handle this problem, the authors proposed a novel weak semantic consistency
constrained (WSCC) approach for image classification. In this case, the extreme
circumstance is obtained by con- sidering each image as one class. Through this
approach, learning of exemplar classifier is used to predict weak semantic
correlations.
In case when there is no available labeled information, the images are
clustered through the weak semantic correlations and images within the one cluster
are assigned the same midlevel class. The partially labeled images are used to
constrain the process of clustering and they are assigned to various midlevel classes

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 5

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

on the basis of visual semantics. In this way, the newly assigned images are used for
classifier learning and the process is repeated till convergence. The experiments are
performed by using semisupervised and unsupervised image classification.
Searching for digital images from lager storage or databases is often required,
so content-based image retrieval (CBIR) also known as query-based image retrieval
(QBIR) is used for image retrieval. Many approaches are used to resolve this issue
such as scale-invariant transform and vector of locally aggregated descriptor. Due to
most prominent results and with a great performance of the deep convolutional neural
network (CNN), a novel term frequency-inverse document frequency (TF-IDF) using
as description vector the weighted convolutional word frequencies based on CNN is
proposed for CBIR. For this purpose, the learned filers of convolution layers of
convolution neuron model were used as a detector of the visual words, in which the
degree of the visual pattern is provided by the activation of each filter as tf part. Then
three approaches of computing the idf part are pro- posed. By providing powerful
image retrieval tech- niques with a better outcome, these approaches concatenate the
TF-IDF with CNN analysis for visual content. To prove the proposed model, the
authors conduct experiment on four image retrieval datasets and the outcomes of the
experiments show the existence of the truth of the model.
Hashing function gives a similar binary code to the similar content of the
image which maps the high-dimensional visual data into low-dimensional binary
space. This approach is basically depending upon the CNN. It is to be assumed that
the semantic labels are represented by the several latent layer attributes (binary code)
and classification also depends upon these attributes. Based on this approach, the
supervised deep hashing technique constructs a hash function from a latent layer in
the deep neurons network and the binary code is learned from the objective functions
that explained about the classification error and other desirable properties in the
binary code.
The main feature of the SSDH is that it unifies retrieval and classification in a
single model. SSDH is scalable to large-scale search, and by slight modification in the
existing deep network for classification, SSDH is simple and easily realizable.

1.2 Aim of the Project

1. To analyze encouraging results for applying deep learning techniques to a
variety of applications, including speech recognition, object recognition, and

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 6

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

natural language processing, among others. Inspired by the successes of deep

learning, in this paper, we attempt to explore deep learning methods with
application to CBIR tasks.

2. To classify and recognize in computer vision, there is a still limited amount of

attention focusing on the CBIR applications. In the proposed method, we
investigate deep learning methods for learning feature representations from
the images and their similarity measures towards CBIR tasks

1.3 Objective
To study the different techniques used in Content Based Image retrieval.
• The basic task of this project is to find similar images (using their visual features)
according to a query image within a large image database.
• Content- Based Image Retrieval (CBIR) system aims to develop techniques that
support effective searching.
• Browsing tasks performed on a large image collection by using automatically
derived image features.
• To calculate time taken for training and testing the large scale dataset.
• To use proximity function that effectively measures distance/similarity between
images with higher precision value.
• To simulate and verify the retrieval results using CBIR and verify the system
operation.
To introduce a deep learning framework for CBIR by training large-scale Deep belief
Networks for learning effective feature representations of images.
• To analyze the system performance by using precision and recall

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 7

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

CHAPTER 2
LITERATURE SURVEY

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 8

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

2. LITERATURE SURVEY
2.1. Related work
For many previous content based approaches there were many local and global
features to represent the image properties and content. Primitives and colony filter are
used for color and texture feature extractions. In their work, an image is divided into
many sub blocks and each block’s color moments are extracted with respect to the
algorithm which exists. These moments are clustered into different classes by using a
clustering algorithm and a specified color feature vector algorithm which is calculated
from the query image and the images in the image database.
The distance between each digital image will be represented by each digital
value but previously we cannot able to retrieve such an accurate value from each of
the image that we searched particularly. Some of the papers will define the mode of
communication take place between the point nodes of the image. The average
precision is 59.61. Object-based image retrieval systems retrieve images from a
database by extracting the object features in the images. In this method database
images are segmented and compare each segmented region against a region in the
query image given by the user. These types of image retrieval systems are generally
successful for objects that can be easily separated from the background and that have
distinctive colors or textures. Color is considered as one of the important low-level
visual features as the human eye can differentiate between visuals on the basis of
color. The images of the real-world object that are taken within the range of human
visual spectrum can be distinguished on the basis of differences in color. The color
feature is steady and hardly gets affected by the image translation, scale, and rotation.
Through the use of dominant color descriptor (DCD), the overall color information of
the image can be replaced by a small amount of representing colors. DCD is taken as
one of the MPEG-7 color descriptors and uses an effective, compact, and intuitive
format to narrate the indicative color distribution and feature.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 9

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 10

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Sr. Paper Title Year of Publication Journal/conference name

No
.

1 Content-based multimedia information Volume 2 ACM Transactions on Multimedia Computing,

retrieval: State of the art and challenges Issue 1 Communications, and Applications
February 2006

2 Object-Based Image Retrieval Using the IEEE Proceedings of Computer Society Conference on Computer Vision
Statistical Structure of Images 2004 and Pattern Recognition (CVPR’04)

3 A Survey on Image Retrieval Methods 2004 CIET-ECE DEPT

4 Content-Based Image Retrieval using Vol. 9, Issue 5, No 1, IJCSI International Journal of Computer Science
Color Moment and Gabor Texture September 2012 Issues,
Feature
5 Content based image retrieval using 07 February 2018 Cluster Computing
deep learning process https://doi.org/10.1007/s10586-018-1731-0
Springer
Table 1.1 Literature survey by various researcher on the basis of algorithm and implementation startegy

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 11

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Sr.No. Algorithm used for feature extraction Algorithm used for Classicification

1 In TRECVID Evaluation, there is a close connection between Semantic classification of the video shots into categories
private industry and academic research where a realistic task- such as zoom-in, zoom-out, pan, and so on. A more recent
specific test set is gathered, discussed, agreed upon, and then practitioner’s guide to video transition detection
numerous research teams attempt to provide the best video
retrieval system for the test set.
2 We represent each sub image in the HSV color space, with the Bayesian Classification
following location-dependent features: (1) half-resolution hue
intensities, (2) half-resolution saturation intensities, and (3) full-
resolution symmetric 5-3 two-level wavelet coefficients of the
value band.
3 k-means algorithm genetic algorithm method k-Nearest Neighbor and Support Vector Machine
classifiers.
4 Color moments and Gabor texture features. Gabor texture descriptors
5 Deep belief network (DBN) method of deep learning The content features extraction seems to be reliable
compared to the existing algorithms, the DBN generates a
huge data set for learning features and provides a good
classification to handle the finding of the efficient content
extraction

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 12

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

r. Algorithm/technique used for similarity measure Ranking algorithm used

No.

1 MPEG-7 standard The new color features [Lew 2001; Gevers 2001] Re-ranking algorithm using post-retrieval clustering
such as the NF, rgb, and m color spaces have specific benefits in areas
such as lighting invariance, intuitiveness, and perceptual uniformity. A
quantitative comparison of influential color models is performed
2 We used approximately 260,000 sub images randomly drawn from 1100
Corel images to learn the unconditional density. This offline process
took about one hour. We computed the partition function for
discretizing feature values from the training set using the K-means
algorithm [4], initialized with equal probability mass partitions.
3 Retrieval by color, Retrieval by Shape ,Retrieval by Texture Cross language information retrieval system document
ranking scheme
4 We calculate the similarity with combined features of color and texture -
using Canberra distance as similarity measure.
5 A high-dimensional vector is used to represent image patches extracted This algorithm usually consists of multistage nonlinear
from the input image. These vectors are composed of a set of feature transformers which are like cascading many neural
maps, and through the network, the feature map of the image patch is networks together. High level data abstractions are
learned from the training data. done with distributed representation i.e., data will be
analyzed with different dimensions and parameters.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 13

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Sr. Precison Recall

No.

1 In some cases, a general problem is reduced to a smaller niche problem -

where high accuracy and precision can be quantitatively demonstrated, but
the general problem remains largely unsolved
2 The first stage Bayesian classifier described in section 3 is able to quickly It found the use of location-sensitive features, the
eliminate the vast majority (more than 97% in our experiments) of the learning of dependence structures, and the accuracy
negative subimages while maintaining a near-zero false negative rate of the unconditional density estimate to be important
factors in the overall performance of the image
retrieval system.
3 Precision increase with use of more datatset Various method of image retrieval was discussed

4 Precision measures the ability of the system to retrieve only models that Recall measures the ability of the system to retrieve
are relevant, all models that are relevant for image.
5 The distance between each digital image will be represented by each It helps to recall the various neural network used to
digital value but previously we cannot able to retrieve such an accurate secured the long distance pointers to make its close
value from each of the image that we searched particularly. Some of the
papers will define the mode of communication take place between the
point nodes of the image. The average precision is 59.61.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 14

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Sr.No. Accuracy View about paper

1 High accuracy The main theme of the pare to make aware about the image
retrieval various techniques and most of it the benefits of
retrieving of image using some feature calculation
2 Further performance gains may be possible if more complex This paper uses many objects for image feature extraction but
models (e.g., higher feature dimensionality or more fails to compared the exact feature due to which the accuracy of
partitions) are employed in later feedback rounds. retrieval of object was reduced
3 As we increases the way of retrieving the image the This paper helps to identify various techniques used for image
accuracy in terms of feature increases retrieval

4 It helps to improve the accuracy There is considerable increase in retrieval efficiency when both
color and texture features are combined. Thus it is rightly said
that only color or only texture cannot differentiate for two
similar objects

5 Small dataset with 1000 images the accuracy rate would be This paper helps to determine the various machine learning
98.6% but with a large data set (> 10000 images) the methods are also defined to predict/retrieve the image easily
accuracy would be 96% without losing the time complexity
requirement.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 15

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

The earliest years of MIR were frequently based on computer vision (three excellent
books, Ballard and Brown [1982]; Levine [1985]; and Haralick and Shapiro 1993])
algorithms which focused on feature based similarity search over images, video, and
audio. Influential and popular examples of these systems are QBIC [Flickner et al.
1995] and Virage [Bach et al. 1996], circa mid 90s. Within a few years, the basic
concept of the similarity searchwas transferred to several Internet image search
engines including Webseek [Smith and Chang 1997] and Webseer [Frankel et al.
1996]. Significant effort was also placed on the direct integration of the feature-based
similarity search into enterprises-level databases such as Informix datablades, IBM
DB2 Extenders, or Oracle Cartridges [Bliujute et al. 1999; Egas et al. 1999] to make
MIR more accessible to private industry.
In the area of video retrieval, the main focus in the mid 90swas on robust shot
boundary detection; the most common approaches involved thresholding the distance
between color histograms corresponding to two consecutive frames in a video
[Flickner et al. 1995]. Hanjalic et al. [1997] proposed a method which overcame the
problem of subjective user thresholds. Their approach was not dependent on any
manual parameters. It gave a set of keyframes based on an objective model for the
video information flow. Haas et al. [1997] described a method of using the motion
within the video to determine the shot boundary locations. Their method
outperformed the histogram approaches of the period and also performed semantic
classification of the video shots into categories such as zoom-in, zoom-out, pan, and
so on. A more recent practitioner’s guide to video transition detection is given by
Lienhart [2001]. Near the turn of the 21st century, researchers noticed that the feature-
based similarity search algorithms were not as intuitive or user-friendly as they had
expected. One could say that systems built by research scientists were essentially
systems which could only be used effectively by scientists. The new direction was
geared toward designing systems which would be user-friendly and could bring the
vast multimedia knowledge from libraries, databases, and collections to the world. To
do this, it was noted that the next evolution of systems would need to understand the
semantics of a query, not simply the low-level underlying computational features.
This general problem was called “bridging the semantic gap”. From a pattern
recognition perspective, this roughly meant translating the easily computable low-
level content-based media features to high-level concepts or terms which would be
intuitive to the user. Examples of bridging the semantic gap for the single concept of
human faces were demonstrated by Rowley et al. [1996] and Lew and Huijsmans

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 16

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

[1996]. Perhaps the earliest pictorial content-based retrieval system which addressed
the semantic gap problem in the query interface, indexing, and results was the
ImageScape search engine [Lew 2000]. In this system, the user could make direct
queries for multiple visual objects such as sky, trees, water, and so on, using spatially
positioned icons in aWWWindex containing 10+ million images and videos using
keyframes. The system used information theory to determine the best features for
minimizing uncertainty in the classification.
At this point, it is important to note that the feature-based similarity search
engines were useful in a variety of contexts [Smeulders et al. 2000] such as searching
trademark databases [Eakins et al. 2003], finding video shots with similar visual
content and motion, or for DJs searching for music with similar rhythms [Foote
1999], and automatic detection of pornographic content [Forsyth and Fleck 1999;
Bosson et al. 2002]. Intuitively, the most pertinent applications are those where the
basic features such as color and texture in images and video, or dominant rhythm,
melody, or frequency spectrum in audio [Foote 1999] are highly correlated to the
search goals of the particular application.
The prevalent research topics which have potential for improving multimedia
retrieval by bridging the semantic gap are as follows: human-centered computing,
new features, new media, browsing and summarization, and
evaluation/benchmarking. In human-centered computing, the main idea is to satisfy
the user and allow the user to make queries in their own terminology. User studies
give us insight directly into the interactions between human and computer.
Experiential computing also focuses on methods for allowing the user to explore and
gain insights into media collections. On a fundamental level, the notion of user
satisfaction is inherently emotional. Affective computing is fascinating because it
focuses on understanding the user’s emotional state and intelligently reacting to it. It
can also be beneficial in measuring user satisfaction in the retrieval process.
Learning algorithms are interesting because they potentially allow the
computer to understand the media collection on a semantic level. Furthermore,
learning algorithms may be able to adapt and compensate for the noise and clutter in
real-world contexts. New features are pertinent in that they can potentially improve
the detection and recognition process or be correlated with human perception. New
media types address the changing nature of the media in the collections or databases.
Some of the recent new media include 3D models (i.e., for virtual reality or games)
and biological imaging data (i.e., for understanding the machinery of life). As

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 17

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

scientists, we need to objectively evaluate and benchmark the performance of the

systems and take into account factors such as user satisfaction with results. Currently,
there are no large international test sets for the many problems such as searching
personal media collections so significant effort has been focused on developing
paradigms which are effective for evaluation. Furthermore, as collections grow from
gigabyte to terabyte to petabyte sizes, high performance algorithms will be necessary
in order to respond to a query in an acceptable time period.
Currently, the most commonly used test sets include collections involving
personal photos, web images and videos, cultural heritage images, news video, and
the Corel stock photography collection which is also the most frequently mentioned
collection. We are not asserting that the Corel collection is a good test set. We suspect
it is popular simply because it is widely available and related loosely to real world
usage. Furthermore, we think that it is only representative and suitable if the main
goal of the particular retrieval system is to find professional stock photography. [1]

Content-based image retrieval has been an active area of research for several years.
The goal is to create systems capable of interactively retrieving images that are
semantically related to the user’s query from a database. Recently, much research has
focused on region-based techniques that allow the user to specify a particular region
of an image and request that the system retrieve images that contain similar regions.
Our research focuses on object-based image retrieval, in which searches are based on
structured, physical objects, such as stop signs or cars, rather than unstructured texture
or color patches. The user specifies an object by providing a small set of example
images of a particular object to the system, and the system retrieves all images that
contain the specified object. The key challenge in object-based image retrieval is to
create a system that can learn the target concept online from a small set of examples
provided by the user.
Most existing region or object-based systems rely on segmentation or require that the
region of interest occupy a large portion of the entire image. This facilitates fast
retrieval but causes these systems to fail when accurate segmentation is not possible
or when the object occupies a small portion of the database image. Additionally, most
existing techniques discriminate based on a histogram or clustering of color or texture
features computed over the entire region. This assumes within-region location-
independence of the features (i.e., that regions are homogeneous blobs of color and
texture). We present a system that performs a windowed search over location and

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 18

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

scale for each image in the database. Images are presented to the user based on their
highest ranking subimages. This approach allows the retrieval of an image based on
the presence of objects that may occupy a small portion (e.g., less than 1% in area) of
the entire image. Also, we do not assume that a feature’s value is independent of
location within the window.
One important resource available to any image retrieval system is the user.
Many image retrieval systems benefit from user feedback on results of previous
searches. In this way, the user provides additional positive and negative examples that
can help direct the search. While negative examples have been shown to be essential
in improving retrieval performance, the problem of how to best acquire negative
examples remains unsolved. Systems that make use of negative examples typically
require the user to present or label examples explicitly or randomly select a small
number of images from the database to use as negative examples. Furthermore, once
the system acquires negative examples, the question of how to use the negative
examples to improve performance remains. The common strategy of penalizing
images that are similar to the negative examples suffers from poor generalization and
high sensitivity to labeling errors. The key contribution of this paper is the
introduction of a new Bayesian method for object-based image retrieval that exploits
the statistics of the image domain of the database. We formulate our Bayesian
classifier as a threshold on the posterior probability of the object class and express the
posterior in terms of the unconditional density and the density of the feature vector
conditioned on the object class. The unconditional density, which represents the
general appearance of subimages within the database, is estimated offline using
hundreds of thousands of samples drawn from the entire database image domain.
Thus, estimating the unconditional density provides a superior alternative to
attempting to model the negative or non-object class using a small set of subimages
labeled by the user. We use the domain samples to learn the spatial dependencies that
exist within the subimages in that domain, providing the probabilistic structure for the
unconditional density. Estimation of the object class conditional density remains
problematic due to the small number of positive examples provided by the user. Our
system, however, acquires useful estimates by employing its knowledge of the
statistical structure of images and by using the unconditional density as a strong prior
to avoid excessive overfitting. The Bayesian classifier labels subimages as positive
(object) or negative (non-object) and ranks positive subimages according to the
posterior probability. This classifier is able to correctly classify an overwhelming

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 19

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

majority of the subimages in the database and provides the user with a compact set of
subimages that are similar in appearance to the object of interest and can be used for
relevance feedback. The user’s positive and negative feedback on the search results is
used to train a second Bayesian classifier that focuses on eliminating difficult false
positives. This second classifier needs to consider only subimages labeled as positive
by the first classifier.[2]
Image retrieval (IR) has become an important research area in computer vision where
digital image collections are rapidly being created and made available to multitudes of
users through the World Wide Web. Tremendous increment in the collections of
images from art museums, medical institutes, and environmental agencies, to name a
few. In the commercial sector, companies have been formed that are making large
collections of photographic images of realworld scenes available to users who want
them for illustrations in books, articles, advertisements, and other media meant for the
public at large. Incredibly, the indexing of these images is all being done manually–a
human indexer selects and inputs a set of keywords for each image. Each keyword
can be augmented by terms from a thesaurus that supplies synonyms and other terms
that previous users have tried in searches that led to related images. Keywords can
also be obtained from captions, but these are less reliable. Content-based image
retrieval research has produced a number of search engines. The commercial image
providers, for the most part, are not using these techniques. The main reason is that
most CBIR systems require an example image and then retrieve similar images from
their databases. Real users do not have example images; they start with an idea, not an
image. Some CBIR systems allow users to draw the sketch of the images which they
wanted. Such systems require the users to have their objectives in mind first and
therefore can only be applied in some specific domains, like trademark matching, and
purchase of painting. Thus the recognition of generic classes of objects and concepts
is essential to provide automated indexing of images for CBIR. However, the task is
not easy. Computer programs can extract features from an image, but there is no
simple one-to-one mapping between features and objects. Earlier CBIR systems rely
on global image features, such as color histogram and texture statistics. Global
features cannot capture object properties, so local features are favored for object class
recognition. For the same reason, higher-level image features are preferred to lower-
level ones. Similar image elements, like pixels, patches, and lines can be grouped
together to form higher-level units, which are more likely to correspond to objects or
object parts. Different types of features can be combined to improve the feature

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 20

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

discriminability. For example, using color and texture to identify trees is more reliable
than using color or texture alone. The context information is also helpful for detecting
objects. A boat candidate region more likely corresponds to a boat if it is inside a blue
region. While improving the ability of our system by designing higher-level image
features and combining individual ones, to be prepared to apply more and more
features since a limited number of features cannot satisfying the requirement of
recognizing many different objects in ordinary photographic images. Each image is
annotated by having a list of keywords associated with it.
There are two possibilities for choosing the keywords:
(1) The annotator can use arbitrary keywords as required.
(2) The annotator is restricted to using a predefined list of keywords (a controlled
vocabulary).
This information can be provided at two levels of specificity:
(1) A list of keywords associated with the complete image, listing what is in the image
(2) A segmentation of the image along with keywords associated with each region of
the segmentation. In addition, keywords describing the whole
Often the segmentation is much simpler than that shown, consisting simply of a
rectangular region drawn around the region of interest or a division of the image into
foreground and background pixels. [3]
Content-based image retrieval (CBIR) has become a prominent research topic because
of the proliferation of video and image data in digital form. The increased bandwidth
availability to access the internet in the near future will allow the users to search for
and browse through video and image databases located at remote sites. Therefore, fast
retrieval of images from large databases is an important problem that needs to be
addressed. High retrieval efficiency and less computational complexity are the desired
characteristics of CBIR systems. In conventional image databases, images are text-
annotated and image retrieval is based on keyword searching. Some of the
disadvantages of this approach are:
1. Keyword based image retrieval is not appropriate because there is no fixed set of
words that describes the image content;
2. Keyword annotation is very subjective. To avoid manual annotation, an alternative
approach is content-based image retrieval (CBIR), by which images would be indexed
by their visual content such as color, texture, shape etc. and the desired images are
retrieved from a large collection, on the basis of features that can be automatically
extracted from the images themselves. Considerable research work has been done to

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 21

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

extract these low level image features, evaluate distance metrics, and look for efficient
searching schemes . Basically, most CBIR systems work in the same way: A feature
vector is extracted from each image in the database and the set of all feature vectors is
organized as a database index. At query time, a feature vector is extracted from the
query image and it is matched against the feature vectors in the index. The crucial
difference between the various systems lies in the features that they extract and in the
algorithms that are used to compare feature vectors. Color, texture, local shape and
spatial information in a variety of forms are the most widely used features in such
systems. Because of high demand for searching image databases of ever-growing size,
CBIR is becoming very popular. Since speed and precision are important, we need to
develop a system for retrieving images that is efficient. The color features are the
most widely used visual features in image retrieval because they are easier to extract
compared with texture and shape information. Color feature is relatively robust to
background complication and independent of image size and orientation. Statistically,
it denotes the joint probability of the intensities of the three color channels.
The system presented in integrates mostly the algorithms introduced into a database
environment. Reasonable results can be achieved using the above mentioned
algorithms, but it is clear that the false positives which are retrieved result from the
lack of spatial information in the index. The simplest way to store spatial information
in the index is to divide the image into sub-images and then extract the color features
for each sub-image. Color correlogram and color coherence vector can combine the
spatial correlation of color regions as well as the global distribution of local spatial
correlation of colors. These techniques perform better than traditional color
histograms when used for content-based image retrieval. However, they require very
expensive computation. Color moments have been successfully used in content based
image retrieval systems. It has been shown that characterizing one dimensional color
distributions with the first three moments is more robust and runs faster than the
histogram based methods. Hence, in our proposed method, Color moments are used
for extraction of color features. Texture is an important feature of natural images. A
variety of techniques have been developed for measuring texture similarity. Most
techniques rely on comparing values of what are known as second-order statistics
calculated from query and stored images. These methods calculate measures of image
texture such as the degree of contrast, coarseness, directionality and regularity; or
periodicity, directionality and randomness. Alternative methods of texture analysis for
image retrieval include the use of Gabor filters and fractals. Gabor filter (or Gabor

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 22

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

wavelet) is widely adopted to extract texture features from the images for image
retrieval, and has been shown to be very efficient. A color retrieval method based on
the primitives of color moments is proposed. After dividing an image into several
blocks, the color moments of all blocks are extracted and clustered into several classes
based on a fast non-iterative clustering algorithm. The mean vector of each class is
considered as a primitive of the image and all primitives are used as feature vectors.
Two test databases from Corel were used and compared the performances of the
proposed method with other existing ones. The experimental results showed that the
proposed method is usually better than others. [4]
Information retrieval from encrypted databases is an important technological
capability for privacy protection in multiparty information management.
Representative application scenarios include online services of webmail such as
Gmail, photo hosting such as Flickr, and financial management such as Mint.com,
where users store their private information on some remote server and the server
provides functionalities to the user, such as categorization, search, and data analysis.
Currently, servers operate on plaintext data, making users’ private information
vulnerable to attacks by untrustworthy administrators and malicious intruders. To
provide secure online services, technologies that protect users’ privacy without
sacrificing functionalities are desirable. The growth of online photo services and the
concerns of privacy protection make searching over encrypted images both attractive
and necessary. A desirable feature for online photo services such as Google Picasa or
Flickr would be the capability to encrypt and store private images, and later retrieve
relevant images without revealing any information about the encrypted images to the
server. Prior work on secure information retrieval was focused on text documents.
Techniques for identifying the presence or absence of a keyword in an encrypted text
document were proposed. Recent work in investigated secure rank-ordered search,
where encrypted text documents are returned in the order of relevance to the query
keyword. Although secure text search techniques can be extended to image retrieval
based on user assigned tags, extension to content based image retrieval (CBIR) is not
straightforward. CBIR systems often rely on comparing the similarity among image
features, such as color histograms, shape descriptors, or salient points, which are
usually high dimensional vectors. Comparing similarity among high dimensional
vectors using cryptographic primitives is challenging. To the best of our knowledge,
no existing techniques address secure feature comparison efficiently and effectively.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 23

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

To build a secure CBIR system, both images and features should be protected. For a
feature based retrieval system, images can be encrypted separately using
cryptographic ciphers or image encryption algorithms. This paper focuses on the
problem of image feature protection which allows the computation of similarity
measures among encrypted features, so that secure CBIR can be achieved To our best
knowledge, this work along with the first endeavors on content based image retrieval
in an encrypted domain. We address the problem by jointly using signal processing
and cryptographic techniques. Three feature protection schemes are explored and
compared in terms of security, retrieval performance, and computational complexity.
We show that retrieval performance comparable to conventional CBIR techniques can
be achieved by the proposed feature protection schemes. These schemes can be used
as building blocks to build efficient indexes, for search over large image databases.
They can also be extended to secure video search by protecting features from the key
frames. Parison among encrypted image features, based on which secure content
based image retrieval can be achieved. We show that the combination of signal
processing and cryptographic techniques, such as random projection, unary encoding,
and random permutation, helps us address the problem of secure image retrieval,
which is otherwise difficult using traditional cryptography alone. The feature
protection schemes explored in this paper exhibit retrieval performance comparable to
the state-of-the-art techniques, and good trade-off can be achieved between security
and computational complexity. These schemes can also be combined with efficient
indexing techniques such scaled to large databases. [5]

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 24

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

CHAPTER 3
SYSTEM ARCHITECTURE

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 25

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

3. SYSTEM ARCHITECTURE

Fig.3.1 CBIR result using deep learning

3.1 System Algorithm
Input: Set of training images (Ti) and test query images (Qi) for CBIR
Output: Retrieved images
Procedure: 1. Explorepre-trained DNN based architecture layers at input, output and feature level 2.
Extract the layer size of image data in architecture 3. Pre-process the set of training images and test
query images 4. Deep representation extraction (Ft and Fq)based on ResNet architecture with feature
layer using activation function of Ti and Qi 5. Compute query features relevance with training image
features 6. Sort the relevant retried images as per rank. 7. Find the performance of retrieval images
using mean average precision for all test query images.

3.2 System description

3.2.1 Dataset
For the small dataset with 1000 images the accuracy rate would be 98.6% but
with a large data set (> 10000 images) the accuracy would be 96% without losing the
time complexity requirement. The content features extraction seems to be reliable
compared to the existing algorithms, the DBN generates a huge data set for learning
features and provides a good classification to handle the finding of the efficient
content extraction.
The framework has been implemented and extensively evaluated in different
scenarios. As a future enhancement this same mention can be forward to real time
extraction.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 26

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

The retrieval performance of a content-based image retrieval system crucially

depends on the feature representation and similarity measurements, which have been
extensively studied by multimedia researchers for decades. A wide range of methods
have been proposed due to the well-known“semantic gap” issue that exists between
the low level image pixels captured by machines and the high-level semantic concepts
perceived by humans. From a high-level perspective, such challenge can be deep-
rooted to the fundamental challenge of artificial intelligence (AI) that is, how to build
and train intelligent machines like human to tackle real-world tasks. Machine learning
is a system used to address this grand challenge in the long term. Recently, some
important advances of new techniques in machine learning have developed. One
important technique is known as “deep learning”, which includes a family of machine
learning algorithms that attempt to model high-level abstractions in data by
employing deep architectures composed of multiple non-linear transformations. But
conventional machine learning methods often uses “shallow” architectures, while
deep learning mimics the human brain and is organized in a deep architecture and
processes information through multiple stages of transformation and representation.
By exploring deep architectures to learn features at multiple levels of data abstracts,
deep learning techniques allow a system to learn complex functions that directly map
raw sensory input data to the output, without relying on human crafted features using
domain information. Recent studies have reported encouraging results for applying
deep learning techniques to a variety of applications, including speech recognition,
object recognition, natural language processing and others. Because of this success
history of deep learning, in this paper, we attempt to explore deep learning techniques
with application to CBIR tasks. In computer vision, much research attention is given
to apply deep learning algorithms for image classification and recognition, but still a
limited amount of attention is given to focus on the CBIR applications. In this paper,
we investigate deep learning techniques for learning feature representations from
images and their similarity measures towards CBIR tasks.
The investigation on a framework of deep learning for content-based image
retrieval (CBIR)was done by applying a state-of-the-art deep learning method, that is,
deep belief networks (DBNs) for understanding learning and mapping feature
extracted from an image from image data, and conduct an extensive set of empirical
studies for a variety of CBIR tasks.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 27

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

 We introduce a deep learning framework for CBIR by training huge dataset of

data deep belief networks (DBN) for understanding effective feature
significant features
 We conduct an extensive set of experiments for comprehensive evaluations of
deep belief networks with application to map and understand feature
representations for a variety of CBIR tasks under varied settings.

3.3 Software And Hardware Requirement

3.3.1 Software Requirements

 Python 2.7 or above versions
 Anaconda software 
 OpenCv Package
 Tensorflow

3.3.2 Hardware Requirements

 CORE I5 PROCESSOR
 8 GB Ram / 500GB Hardisk

Table 3.1 Comparison with previous system for image retrieval accuracy
Search Bayesian Blobworld Proposed System
Arabian Horses 77 81 89
AutoRacing 72 41 85
Elephants 53 39 60
Helicopters 19 14 35
Lions 26 26 32
Owls 96 72 92
Plar Bears 41 23 52
Wind Surfing 51 30 63
Whitetail Deer 45 29 52
Watches 21 35 46
Tigers 35 40 48

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 28

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Fig. 3.2 Comparison of accuracy with previous system and proposed system

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 29

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

CHAPTER 4
RESULT AND DISCUSSION

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 30

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

4. RESULT AND DISCUSSION

In our method, a multi-feature image retrieval method is introduced by combining the
features of color histogram, edge, edge directions, edge histogram and texture
features, etc. In this model, the content based image will be extracted from a
collection of intended image groups. After performing some pre-processing steps like
selection removal, its above features are extracted and are stored as small signature
files. Similar images should have similar signatures. These signatures are compared
with the content based signature. During the similarity measure, the distances between
the different features are measured. Appropriate weights are applied to normalize the
distance coefficients [1]. These normalized coefficients are sorted and indexed based
on the distance values and their optimized state of functioning.

Fig.4.1 Different concept of retrieving

JPEG is not a file format but a compression standard. It is based on the

Discrete Cosine Transformation (DCT).Every image is split in to 8*8 blocks and each
of these pixel groups are separately encoded with its own discrete cosine transform. It
can be further exactly replicated by 64 cosine waves. The first one is called the DC
component representing the general intensity of that particular block while the rest 63
AC coefficients follow the same. One of the interesting factor is that the low
frequency components have a much bigger impact than the high frequency ones in the
DCT transformed image. For this reason, these high frequency compo-nents is
removed through the process called as quantization and only the needed ones are
preserved back. It is to be noted that the added sum of the product of these
coefficients with the cosine waves will produce back the original image or the
block.Huffman coding is the final step in JPEG compression, which gets finally stored
along with the header information. The JPEG decompression follows the exact

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 31

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

reverse process as shown in Fig 11 and in this work; we concentrate majorly on DCT
high frequency coefficients along with the JPEG header for building a robust CBIR
system. The DCT equation in this whole compression process is given by

Equation…….1
Here ‘u’ represents the horizontal spatial frequency; v symbolizes the vertical spatial
frequency. g(x,y) matches to pixel value at co-ordinates (x,y) and G(u,v)
characterizes the DCT coefficients at co-ordinates (u,v).

4.1 Procedure to achieve it

 Feature extraction
To extract the features of the high frequency information, a set of feature maps
are extracted from the input (LQ) image by way of a convolution. A high-
dimensional vector is used to represent image patches extracted from the input
image. These vectors are composed of a set of feature maps, and through the
network, the feature map of the image patch is learned from the training data.
The feature extraction block consists of nf convolutional layers with a kernel
size sf which outputs df features as follows:
nf × Conv(sf, df)

Each variable should be determined in consideration of the following:

 nf represents the low-level features, such as edges or corners, that can be
extracted at the lower level of the layer, and more complex features, such as
textures, can be extracted at the higher level [32];
 sf indicates that a large size convolutional kernel can be replaced with
multiple stages of a small size kernel to reduce both the number of parameters
and the computational cost while maintaining the same receptive field [33];
 df represents the number of LQ feature dimensions,which is a factor that
influences the performance. Therefore, it is important to determine optimal
values of all variables.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 32

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

 Mapping
The features extracted from the previous block are non-linearly mapped by
this block, which consists of three modules: shrinking, non-linear mapping,
and expansion. It shows that these modules reduce the number of parameters
and achieve better performance than a single convolution layer.

II. To conduct an extensive set of empirical studies for comprehensive evaluations of

Deep Belief Networks with application to learn feature representations for a variety of
CBIR tasks under varied settings.

The digital image can able processed with two modes of operation i.e. spatial domain
and the frequency domain in which transformation is required. In certain system the
content extraction may take place with the help of Image segmentation.
The extraction takes place in the spatial domain will provide a data on visual features
such as color, radiance, luminance, structures, etc., whereas the frequency domain
features will provide the information related to the frequency, etc., The deep learning
method used will have these both features which indeed a challenging due to the
enormous amount of data. Out of this enormous data, significant features will be
taken which makes the processing better (since it avoids maximum time complexity).
This data will be computed with care to make classification and computer vision
computation a better. The extraction of visual features such as color, texture, shape,
spatial relationship, etc. is one of the major operations in designing a reliable and
efficient image retrieval resource. By considering the above sequence of data or
image extraction phenomenon each datum will be formed with some suitable content
based text. For every formation of data which is holding each and every image will be
analysed by using the neural networking structure. The advanced level of such
formation of data extraction will be majorly done by using the deep learning
phenomenon.
Each single character identification will be done in each and every content
based text structure. Only through the help of such a feature extraction phenomenon
major number of vital errors can be overcome and the précised output will be able to
process. In each and every module of segmentation, each bit of information is utilized
for comparing the inbuilt image of the respective data. Not only for the processing of
feature, but also through this process property based extraction can also be done using
the deep learning phenomenon.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 33

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Fig. 4.2 Process flow of extraction

Color Features: The colour feature of an image is of most significance as it plays a vital role
in the human visual perception mechanism. The ways of representing colour of an image
includes the colour histograms and the statistical methods of colour representation. The HSV
(hue, saturation and value) space is very frequently used because the three dimensions of it
describe the way how humans naturally respond to colour and the ease of computation. In
these methods, the colour histogram of the query image and all database images is obtained.
The proportion of colour can be specified by the user depending on what type of results is
required. All those images whose histogram matches with that of the query image will be
retrieved [25].There are multiple methods for extracting color feature.
Texture Features: If there are some areas in any image with similar colour, then their
textures are matched. The second order statistics are calculated from the query image and all
database images like relative brightness amongst selected pairs of pixel of image, degree of
contrast, coarseness, directionality and regularity, or periodicity, or directionality and
randomness. The structural arrangement of surfaces and objects gives the texture of image.
The Gabor filters and fractals are used for texture analysis. The statistical parameters
calculated from the intensity values of pixels, co-occurrence matrices and texture histograms
built upon trauma features and can be used for the analysis. The recent extension to it is
‘texture thesaurus’ in which code-words presenting classes of texture corresponding to
regions are generated and matched. The texture of images can also be described using
wavelets. The signals can be represented and analyzed at different scales i.e. under different
resolutions by using wavelets. A hierarchical structure based on wavelets wherein each level
represents the original signal with a certain degree of detail, is also possible to be used for the
analysis of texture [25]. There are multiple methods for extracting texture feature

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 34

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Shape Features: A number of characteristic features of object’s shape (independent of size of

object and its orientation) are computed for the query image and all database images. These
shape features are matched for the retrieval. The shape features are extracted by using
commonly used .
global features like aspect ratio, circularly and moment invariants; the local features like sets
of consecutive boundary segments can also be used [25]. There are multiple methods for
extracting shape feature

Fig. 4.3 Image extraction using deep learning

Similarity Measure: Similarity matrix is used so as to find out the similarity among the two
images. In CBIR once the features gets extracted from each individual image, they are stored
in a feature dataset and so as to retrieve the best results from the database, similarity matrix is
used which measures the similarity among the query image and the stored image in the
dataset. There are various similarity matrices

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 35

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

4.2 Result
4.2.1Front Page

Fig.4.4 Selection of input

Fig. 4.5 Classification of feature of compressed image

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 36

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

Fig. 4.6 Result of feature extraction

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 37

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

CHAPTER 5
APPLICATIONS AND ADVANTAGES

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 38

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

5. APPLICATIONS AND ADVANTAGES

The CBIR technology has been used in several applications such as fingerprint
identification, biodiversity information systems, digital libraries, crime prevention,
medicine, historical research, among others. Some of these applications are presented
in this section.
5. 1 Applications
5.1.1 Medical Applications
The use of CBIR can result in powerful services that can benefit biomedical
information systems. Three large domains can instantly take advantage of CBIR
techniques: teaching, research, and diagnostics. From the teaching perspective,
searching tools can be used to find important cases to present to students. Research
also can be enhanced by using services combining image content information with
different kinds of data. For example, scientists can use mining tools to discover
unusual patterns among textual (e.g., treatments reports, and patient records) and
image content information. Similarity queries based on image content descriptors can
also help the diagnostic process. Clinicians usually use similar cases for case-based
reasoning in their clinical decision-making process. In this sense, while textual data
can be used to find images of interest, visual features can be used to retrieve relevant
information for a clinical case (e.g., comments, related literature, HTML pages, etc.).

5.1.2 Biodiversity Information Systems

Biologists gather many kinds of data for biodiversity studies, including spatial
data, and images of living beings. Ideally, Biodiversity Information Systems (BIS)
should help researchers to enhance or complete their knowledge and understanding
about species and their habitats by combining textual, image content-based, and
geographical queries. An example of such a query might start by providing an image
as input (e.g., a photo of a fish) and then asking the system to “Retrieve all database
images containing fish whose fins are shaped like those of the fish in this photo”. A
combination of this query with textual and spatial predicates would consist of “Show
the drainages where the fish species with ‘large eyes’ coexists with fish whose fins
are shaped like those of the fish in the photo”. Examples of initiatives in this area
include.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 39

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

5.1.3 Digital Libraries

There are several digital libraries that support services based on image
content. One example is the digital museum of butterflies, aimed at building a digital
collection of Taiwanese butterflies. This digital library includes a module responsible
for content-based image retrieval based on color, texture, and patterns. In a different
image context, It present a content-based image retrieval digital library that supports
geographical image retrieval. The system manages air photos which can be retrieved
through texture descriptors. Place names associated with retrieved images can be
displayed by cross referencing with a Geographical Name Information System
(GNIS) gazetter. In this same domain, Bergman et al. describe an architecture for
storage and retrieval of satellite images and video data from a collection of
heterogeneous archives. Other initiatives cover different concepts of the CBIR area.
For example, while research presented concentrates on new searching strategies for
improving the effectiveness of CBIR systems, another popular focus is on proposing
image descriptors.

5.2 Advantages
 CBIR is a fast developing technology having considerable potential.
 It provides an overview of the functionality of content based image retrieval
systems and texture analysis with various applications.
 Due to hybrid in nature it can access with both computer vision and of
database systems.
 There are various applications of CBIR in every fields of life like blood cell
detection, archeology, criminal investigation, image search, social networking
sites, forensic Labs, and satellite etc.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 40

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

CHAPTER 6
CONCLUSION AND FUTURE RESEARCH

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 41

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

6. CONCLUSION AND FUTURE RESEARCH

6.1 Conclusion
For the small dataset with 1000 images the accuracy rate would be 98.6% but with a
large data set (> 10000 images) the accuracy would be 96% without losing the time
complexity requirement. The content features extraction seems to be reliable
compared to the existing algorithms, the DBN generates a huge data set for learning
features and provides a good classification to handle the finding of the efficient
content extraction. The framework has been implemented and extensively evaluated
in different scenarios. As a future enhancement this same mention can be forward to
real time extraction. And by using deep learning since deep learning is the kind of
advanced Artificial Neural Network developed by many researchers to make the
machine learning process to different level of frontier. The main role of this deep
learning process is to extract the information in high level abstraction methodology.
This algorithm usually consists of multistage nonlinear transformers which are like
cascading many neural networks together. High level data abstractions are done with
distributed representation i.e., data will be analyzed with different dimensions and
parameters. Each abstractions re done through the hierarchical explanatory factors, in
which the many sublevel information are generated with single previous level of
generated information. Many traditional ANN methods failed to process the unlabeled
data but the almost all deep learning algorithms have a capability of processing the
unlabeled. The deep belief network (DBN) is one of the deep learning algorithms
which is capable of processing the data unsupervised manner. This algorithm also
have a capability classifying the invariant data which has the divergence of ranges i.e.
the noise, displacement, smoothness etc.

6.2 Future Research

Despite the considerable progress of academic research in multimedia information
retrieval, there has been relatively little impact of MIR research on commercial
applications with some niche exceptions such as video segmentation. One .We believe
that the MIR community has a golden opportunity in the growth of the multimedia
search field that is commonly considered the next major frontier of search [Battelle
2005]. An issue in the collaboration between academic researchers and industry is the
opaqueness of private industry. Frequently it is difficult to assess if commercial
projects are using methods from the field of content-based MIR. In the current

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 42

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

atmosphere of intellectual property lawsuits, many companies are reluctant to publish

the details of their systems in open academic circles for fear of being served with a
lawsuit. Nondisclosure can be a protective shield, but it does impede open scientific
progress. This is a small hurdle if the techniques developed by researchers have
significant direct application to practical systems. To assess research effectively in
multimedia retrieval, task-related standardized databases on which different groups
can apply their algorithms are needed. In text retrieval, it has been relatively
straightforward to obtain large collections of old newspaper texts because the
copyright owners do not see the raw text as having much value. However image,
video, and speech libraries do see great value in their collections and consequently are
much more cautious in releasing their content. While it is not a research challenge,
obtaining large multimedia collections for widespread evaluation benchmarking is a
practical and important step that needs to be addressed. One possible solution is to see
that task-related image and video databases with appropriate relevance judgments are
included and made available to groups for research purposes as was done with
TRECVID.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 43

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

References
[1] Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia
information retrieval: State of the art and challenges. ACM Trans. Multimed.
Comput. Commun. Appl. 2(1), 1–19 (2006)
[2] Hoiem, D., Sukthankar, R., Schneiderman, H., Huston, L.: Object based image
retrieval using the statistical structure of images? In: Proceedings of the IEEE
Conference on Computer Vision and Pattern Recognition, (2004)
[3] Xue, B.,Wanjun, L.: Research of Image Retrieval Based on Color. In:
Proceedings of the IEEE International Forum on Computer Science-Technology
and Applications, pp. 283–286 (2009)
[4] Huang, Z.C., Chan, P.P.K., Ng, W.W.Y., Yeung, D.S.: Content based image
retrieval using color moment and Gabor texture feature, in Proceedings the IEEE
Ninth International Conference on Machine Learning and Cybernetics, Qingdao.
pp. 719-724 (2010)
[5] Lu, W., Varna, A.L., Swaminathan, A., Wu, M.: Secure image retrieval through
feature protection. In: Proceedings of the IEEE International Conference on
Acoustics, Speech and Signal Processing, (ICASSP). pp. 1533–1536 (2009)
[6] Gasarch,W.:ASurvey on Private Information Retrieval, in Bulletin of the
EATCS, vol. 82, pp. 72–107. EATCS, Rio (2004)
[7] Danezis, G., Gürses, S.: A critical review of 10 years of privacy technology. In:
Proceedings of the 4th Surveillance and Society Conference, (2010)
[8] Weinzaepfel, P., Jégou, H., Perez, P.: Reconstructing an image from its local
descriptors. In: Proceedings of the IEEE Computer Society Conference on
Computer Vision and Pattern Recognition (CVPR), pp. 337–344 (2011)
[9] Lv, Q., Josephson,W.,Wang, Z., Charikar, M., Li, K.: Multi-probe LSH: Efficient
indexing for high-dimensional similarity search. In: Proceedings of the 33rd
International Conference on Very Large Data Bases (VLDB), pp. 950–961
(2007)
[10] Hinton, G.E., Osindero, S., Teh, Y.-W.: A fast learning algorithm deep belief
nets. Neural Comput. 18(7), 1527–1554 (2006)
[11] Journal of Adv Research in Dynamical & Control Systems, Vol. 10, 13-Special
Issue, 2018 *Corresponding Author: Intedhar Shakir Nasir1
[email protected] Article History: Received: April 04, 2018,
Revised: September 25, 2018, Accepted: September 28, 2018 2349 A New
Approach for Content Based Image Retrieval Using Statistical Metrics

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 44

Feature Extraction Using Deep Learning Methods For Content Based Retrieval Of Image

[12] Poulami Haldar, Joydeep Mukherjee, “Content based Image Retrieval using
Histogram, Color and Edge”, International Journal of Computer Applications,
Volume 48, Issue 11, pp. 25-31, 2012.
[13] Sonika Jindal, Sunpreet Kaur, “Image Retrieval using Variants of Color
Histogram Techniques”, International Journal of Advanced Research in
Computer Science and Software Engineering, Volume 6, Issue 10, pp. 207-212,
2016.
[14] Tawfiq A. Al-asadi, Ahmed J. Obaid, “Object Based Image Retrieval Using
Enhanced SURF”, Asian Journal of Information Technology, Volume 15, Issue
16, pp. 2756-2762, 2016.
[15] J. Vanitha, M. SenthilMurugan, “An Efficient Content Based Image Retrieval
Using Block Color Histogram and Color Co-occurrence Matrix”, International
Journal of Applied Engineering Research, Volume 12, Issue 24, pp. 15966-
15971, 2017.
[16] Umar Manzoor, Mohammed A. Balubaid, Bassam Zafar, Hafsa Umar, M.
Shoaib Khan, “Semantic Image Retrieval: An Ontology Based Approach”,
International Journal of Advanced Research in Artificial Intelligence, Volume 4,
Issue 4, pp. 1-8, 2015.
[17] K. Shubhankar Reddy, K. Sreedhar, “Image Retrieval Techniques: A Survey”,
International Journal of Electronics and Communication Engineering, Volume 9,
Issue 1, pp. 19-27, 2016.
[18] Mussarat Yasmin, Sajjad Mohsin, Muhammad Sharif, “Intelligent Image
Retrieval Techniques: A Survey”, International Journal of Electronics and
Communication Engineering, Volume 12, Issue 1, 2014.
[19] MussaratYasmin, SajjadMohsin, MuhammadSharif, “Intelligent Image Retrieval
Techniques: A Survey”, Journal of Applied Research and Technology, Volume
12, Issue 1, pp. 87-103, 2014.
[20] Rafał Grycuk, Marcin Gabryel, Robert Nowicki, Rafał Scherer, “Content-based
image retrieval optimization by differential evolution”, "", Evolutionary
Computation (CEC) 2016 IEEE Congress on, pp. 86-93, 2016.

Padm. Dr.V.B.K.C.O.E. Malkapur, Dept of E&TC Engineering, ME (E&TC) Page 45