International Journal on Recent and Innovation Trends in Computing and Communication

Volume: 2 Issue: 7

ISSN: 2321-8169
2066 2071

_______________________________________________________________________________________________

Personalized Image Search Using Complex Multiple Words-Based Queries

Nishant Singh Chauhan

Mr. S .Madhu

Galgotias University
Greater Noida, India.
[email protected]

Galgotias University
Greater Noida, India.
[email protected]

AbstractIn todays world internet produces a huge amount of data and to search content among this data keyword based search is commonly
used among web users. To improve the search experience of the searchers the metadata can be used which is being generated by internet. Social
networking and sharing websites like Facebook, Flicker and YouTube give features to users that allow them to share, create, tag, comment and
annotate. This creates the user-generated metadata in a bulk amount which can be utilized for management and media retrieval. We consider the
user preferences and returned the result list accordingly in Personalize Search to improve the web search experience. In this paper, we propose a
model in which user and query relevance considered simultaneously to learn to personalized image search. This model is tested for complex
multiple based query and its showing satisfactory results. In this crucial work is to insert the user preference and query-related search intent into
user-specific topic spaces.
KeywordsPersonalized image search; annotation; Ranking base multicorrelation tensor factorization; Metadata; User specific topic.

__________________________________________________*****_______________________________________
I.
INTRODUCTION
Over a past few decades web search engines have played
the main role in accessing the information available online.
But still even todays best search engine is not able to
provide quality search results. Approximately 50% of the
web search sessions fails to find any relevant results for
searcher[3][4]. This all happens because of generally short
and nonspecific queries. For example HP could be a
petroleum company Hindustan Petroleum or a computer
manufacturing company Hewlett Packard. Another reason
may be that user may have different meaning for the same
word. For example query for word cricket could be an
insect or it could be a sports game played. Therefore to
overcome this problem the solution is Personalize search. In
Personalize search the information related to user is
considered to predict exact intention of the user and then the
result is ranked accordingly. Whereas in Non-personalized
results are given directly without focusing on user
assumptions.

existing methods for tag refinement mainly focus on either

on images and tag or images and user but not on all the three
entities together. As discussed above, user creates the
tagging activity and this user interaction with tagging gives
remarkable results.
We sincerely consider that the integration of user
information adds to a superior understanding and
explanation of the tagging data. Lets consider the following
examples to understand this observation. In Figure. 1 User A
has tagged the image of Taj Mahal monument as Taj and
another User B has tagged hotel Taj as Taj. Second picture
in the same figure shows a aero plane, in which the tagging
done by an engineer is aircraft and a businessman tags as
aero plane. Our main motive is to improve the original
relations between the images and tags which is supported
with unprocessed tagging data on web.

The components of proposed framework:

1.

A improved ranking-based multicorrelation tensor

factorization model is proposed to perform annotation
prediction, this is considered as users potential
annotations for images;

2.

User-specific topic modelling is introduced, which map

query relevance and user preference into same userspecific topic space. Two resources involved with
users social activities are employed for evaluating
better performance.

Problem of missing and noisy tags may occur in a large

scale web dataset, which in turn may restrict the working of
social systems which are based on tag retrieval system.
Therefore to solve this problem refinement of tag to make it
free from noise and enrich it for images is necessary. Earlier

Figure1.

2066
IJRITCC | July 2014, Available @ http://www.ijritcc.org

_______________________________________________________________________________________

International Journal on Recent and Innovation Trends in Computing and Communication

Volume: 2 Issue: 7

ISSN: 2321-8169
2066 2071

_______________________________________________________________________________________________
This module is expanded to find out complex multiple wordbased queries result with the modified ranking tensor
factorization model.
II.

LITERATURE REVIEW

We have first surveyed some previous work on personalize

search in this section. After that we have examined and
discussed the limitation of these works in terms of the user
interest and user profiling that is relevance measurement and
improving results.
1.

Personalized Image Search

To personalize the image search is a challenging task

because images contains very less text that can be used to
explain them. In an example, where user is seeking for the
photos of jaguar. Now what should system return picture
of wild animal or images of luxurious cars? To sort out such
ceases, personalization helps to remove uncertainty from the
keywords used in image search or to remove irrelevant
images from search results. Hence, if the given user is
concerned in nature, the system will show the images of the
predatory cat of South America and not of an automobile
[2].
With the help of user generated metadata and query
expansion personalize system helps to weed out irrelevant
result.
Personalization techniques traditionally falls in one of two
categories: collaborative-filtering or profile based. The first
thing that in collaborative filtering [15] is that it
recommends new items to user of similar class by
aggregating opinions of many users. This all can be done by
asking users to rate items on a universal scale, and designing
such rating system is itself challenging task

Figure 2. Examples for (Top) Non-personalize search and (Bottom)

Personalized search result for the query jaguar.

and process to bring out high quality ratings from users are
equally important. In spite of this there is no assurance that
users getting higher returns for making suggestions is less
and therefore will be hesitant to make the extra effort[14].

Personalization systems for second class uses individual

users interest profile. There are some problems associated
with this approach, one problem is that it would be very
time consuming for every user to maintain their explicit
profiles current and second problem is that this approach use
the personal information of the users and no one wants to
share that information which makes it difficult to access for
researchers[15]. Where as in most cases these data mining
methods have proven commercially successful and helpful.
One of the important resource of metadata are tags, user can
easily understand and identify the data with the help of these
tags as they are user defined keywords. Many challenges got
arise in these tagging systems when user try to attach
semantics to objects though keywords [8][12]. These
challenges can be elaborated as a single tag has multiple
related meanings, multiple tags have same meaning and the
same tag may have different meaning. There are many
methods used by many social websites one of them is that
they display images in the result based on their
interestingness, having the most interesting images on
the top [14]. The information contained in user-generated
metadata, especially the tags are fully utilized by machine
learning-based method, which in turn shows the result for
personalize search for given user. But this method also fail if
user has not shown any interest in that domain in past [13].
Personalize search is divided into two steps as almost all the
existing work uses this scheme: the non-personalize
relevance score is computed between the document and the
query and personalized is calculated by estimating the users
preference over the document. After that merging these two
scores a final ranked list of images is produced [5] [12].
While using this two-step scheme some problems arises. i)
The explaining way is not realistic and straight. The main
purpose of personalize search is by estimating the users
preference over documents, rank the returned documents for
certain queries. Individually by computing a user-document
relevance score and a query-document relevance score all
present schemes estimates user-query-document correlation,
however it could be done at once by just finding user-querydocument correlation.
ii) How merging operation is to be done this question is not
of great importance [11]. Since the searcher themselves
judge the appropriate matter in hand therefore in personalize
search, verification is not an easy task. User study is the
most popular and usual method in which various
participants judge the result coming from various searches.
But this way of finding result is very expensive as it requires
a lot of research and even the results are unfair as the
participant knows that they are being tested. There is an
additional way by click through history or user query logs,
2067

IJRITCC | July 2014, Available @ http://www.ijritcc.org

_______________________________________________________________________________________

International Journal on Recent and Innovation Trends in Computing and Communication

Volume: 2 Issue: 7

ISSN: 2321-8169
2066 2071

_______________________________________________________________________________________________
but it requires really massive and scalable real search logs,
which is not easily available for the researchers [10].
Every one wants to keep their personnel information
confidential because of the privacy issues therefore they
dont share their profiles, which makes updating these
profiles difficult. As personalize system require user data
this becomes a problem. To overcome this problem social
media plays an important role, here user uploads a picture,
mark some object as favorite and write blogs. From this we
get the required user generated data without interfering the
users privacy [9].
2.

2.

To map the user preference and query relevance

into the same user-specific topic space a userspecific topic modeling is done.

According to the calculated users choices, considering the

query and user information at the same time, the images are
ranked finally. With the three tier architecture the projected
system is implemented. The first tier is client site where user
submits query, then in second tier at server site searching is
done and then in third tier contains remote database site
where results are stored. This framework is also tested for
complex multiple word-based queries.

Problem Identification

World Wide Web contains a large number of photo sharing

websites having large image collections available online,
such as Zooomr, Picasa, Flickr and Pintrest4.To form a
communication channel in a social media these websites
assign their users as the owner, tagger, and commenter for
their all contributed images which in turn work together and
they are able to relate with each other [9] [8].
Since this web contain a huge amount of dataset, missing of
tags and noisy tags are unavoidable, due to which the
performance of social tag-based retrieval system is limited
[7] [6] [5].To solve this problem refinement of tags noise
removal from tags and enriching them for images is
necessary. The more efforts are being done on tag
refinement to tackle with missing and noisy tags issues,
while the most important source of user study that is the data
of users communication in social tagging is neglected [8].
In this paper the solution is provided by doing personalize
search by considering the users query online and by
analyzing the users information offline simultaneously.
User annotation to the image is predicted by using ranking
based tensor factorization model system.
III.

PROPOSED FRAMEWORK

There are two stages in the framework of this paper: Online

stage and offline personalize search stage as shown in Fig.3.
The basic idea is to embed the user preference and queryrelated search intent into user-specific topic spaces. Since
for topic modeling the users original explanation is
scattered, therefore we need to improve quality of
explanation pool giving users notes before user-specific
topic spaces construction.
Components of framework:
1.

Figure.3. Three tier architecture system[1].

To perform the basic search as per by predicting

users interest related with the query, considering it
as the main annotations for the images, a rankingbased multi correlation model is proposed.

A. Ranking
Based
Multi-correlation
Factorization (RMTF)

Tensor

There are three types of entities of entities considered in

all photo sharing social websites for tagging data. We
can view this classified data as a set of triplets, in which
let U be the set of users, I be the set of images and T the
set of tags and the set of observed tagging is denoted by
O, i.e., each triplet (U,I,T) O means that user has
annotated image with tag. The ternary interrelations can
then constitute a three dimensional tensor, which is
defined as
Yu,i,t = 1, if (u,i,t) O
0, otherwise

(1)

2068
IJRITCC | July 2014, Available @ http://www.ijritcc.org

_______________________________________________________________________________________

International Journal on Recent and Innovation Trends in Computing and Communication

Volume: 2 Issue: 7

ISSN: 2321-8169
2066 2071

_______________________________________________________________________________________________
factor subspaces we assume that two items with high
affinities should be mapped closed to each other.
C. User Specific Topic Modeling
Personalized search can be directly performed once the
remodeling of user-tag-image ternary interrelations are
done: for a users query, the rank of image is inversely
proportional to the probability of annotating with tag.
D. Online Personalize Search

Figure.4. Tagging data interpretation (a) 0/1 scheme (b) Ranking Scheme

A three dimensional matrixes constructed for individual user

is called tensor and initially it is created for individual user
per image. If a user gives a tag to the image 1is entered into
the matrix otherwise 0. We refer this as the 0/1 scheme since
this optimization scheme uses 1 and 0 numerical values. All
observed data is given value 1 and unobserved data as value
0. There is a problem associated with it, that is if a user
doesnt give a tag to an image it assigns a value 0 to it but it
doesnt mean that if a user is not giving tag to an image then
the user doesnt like the image. Maybe that user does not
want to tag that image or has no chance to see that image As
some concept may be missing in 0/1 scheme, therefore to
address this problem ranking optimization scheme is used as
it considers user tagging behavior.
Every user image combination is defined as post, on each of
this post ranking scheme is performed and a positive tag set
and negative tag set is constructed in post (u,i). These sets
form training pairs and we have considered that all positive
tag sets give better description of images than negative tag
set. There is a possibility that user generated tags may have
some concept missing. All context relevant tags (tags that
occur frequently) are supposed or likely to occur in the same
image but user will not bother for all the relevant tags to
express the image. The good description for image is
possible by the tags which are semantic-relevant with
noticed tags.

In an online personalize search first user-specific query

mapping-estimate is performed after user submits a query
which is the conditional probability that belongs to userspecific topics. A list of topics is generated from the user
which is compared with user query and a prediction is made
that user has interest in certain area to rank images
accordingly.
IV.

There are two sets of records in database, one set contains

records of image and tags associated with the images given
by the users and other set contains the description of images.
1.

In the beginning a three dimensional matrix tensor

is created containing user, image and tag.
Yu,i,t = 1, if (u,i,t) O
0, otherwise

2.

Then all the records from the database are retrieved

and query is matched with all the records to find
the relevance between them. All the tags present in
dataset are compared with query word one by one.
We have set threshold value in our system to 0.5. if
value of comparison is 0.5 or more than that we
take the value as 1in a tensor otherwise 0 is taken.

If matchtopic >= 0.5

then tensorvalue =1 else 0
3.

Now for multiple words query if query is apple

puma we compare the first word of query with the
first tag present in the database after that we
compare it with second word of the query.

4.

A graph is formed based on tags semantic and

context intra-relations using all above information,
which generates lists of topic for user.

5.

After that a calculated matrix value is taken and

placed in an array which contains images and
values.

B. Constraint for Muticorrelation Smoothness

The average number of tagger for each image in Flicker
is about 1.9, because 90% images in Flicker have not more
than four taggers. In Del.icio.us have 6.1 average tagger for
each web page. To enable information propagation
information system considers the external resources because
of limitations and multiple relations between users, images
and tags are collected by the system. System also collect
ternary interrelations among user-query-document. In learnt

ALGORITHM

2069
IJRITCC | July 2014, Available @ http://www.ijritcc.org

_______________________________________________________________________________________

International Journal on Recent and Innovation Trends in Computing and Communication

Volume: 2 Issue: 7

ISSN: 2321-8169
2066 2071

_______________________________________________________________________________________________
6.

There may be multiple tags for the same images

that could be relevant with the query and it may
create duplicates. Therefore those duplicates need
to be removed from the final list as many images
may occur several times in the result.

7.

With the help of sorted array images are arranged

in an order from highest to lowest value of
relevance.

8.

Final sorted and personalized search result for

complex multiple word-based query is generated.
V.

Figure.5. Showing Non-personalize and personalize search for User A and

User B

In our second experiment we tested our system for complex

multiple word-based query and we got the desired results.
In this experiment User A tagged another image of a car
named Alto as Alto and User B tagged a music
instrument named Alto as Alto. After that multiple word
query Apple Alto was given to system.

EXPERIMENTAL RESULTS

The result based on users intent are shown by personalize

search and are more accurate as compared to nonpersonalize search which contains many irrelevant images.
For showing Personalize search in out experiment we have
considered two users User A and User B who searches for
word Apple. Now an Apple could be a fruit to a user or
it could be a product from Apple Company. Lets assume
according to User A its a fruit therefore he tags Apple as a
fruit and User B tags Apple as a product from Apple
company. In the figure (i)It shows non-personalize search
result for both the users it contains the picture of apple and
apple company products like ipad and iphone. (ii) It shows
the personalize search result and it have the picture of apple
fruit for User A (iii) It shows the personalize search result
for User B and shows the images oh iPhone.
Here we created a huge database containing the several
images of apple, iphone, jaguar cat, jaguar car and with this
we created two users profile. Now this whole system was
tested for Personalize and Non-personalize search.
Now we have done some advancement in previous system
and we took it to another level of searching.

Figure.6. Showing Non-personalize and personalize search for User A and

User B for complex multiple word-base query.

In above figure (i) First row shows the result for the nonpersonalize search for both the users and it shows the
images of an Alto car, Alto music instrument, Apple fruit
and Apple iphone. (ii) Second row shows the result for User
A, as he has tagged Alto as a car and Apple as a fruit,
therefore it shows the images of car and fruit. (iii) Third row
shows the result for User B, as he has tagged Alto as a music
instrument and Apple as an Iphone, therefore it shows the images
of music instruments and Iphones.
It was observed that proposed framework gave the desired
output and greatly performs the outline.
VI.

CONCLUSION

In todays world every searcher wants that the result

returned for his query should be according to his intent
(semantic) which a very challenging task is as web contains
a lots of data and which accordingly generates a huge
amount of metadata. This metadata is created by user itself
in form of tag, like and post which they perform on social
media sites. Now to utilize this user generated information
in social sharing websites for personalized search is a
challenging task is an important part of the framework. In
our proposed framework we found that by utilizing users
2070
IJRITCC | July 2014, Available @ http://www.ijritcc.org

_______________________________________________________________________________________

International Journal on Recent and Innovation Trends in Computing and Communication

Volume: 2 Issue: 7

ISSN: 2321-8169
2066 2071

_______________________________________________________________________________________________
social activities and user created metadata for personalized
search has performed extremely well and shows satisfactory
result. We have even extended our work to complex
multiple word-based queries and got the same satisfactory
and outperforming results. One of the important advantage
of this multiple word search is user can save its time by
simultaneously searching for two queries in one search
operation.
Hence for future work (i) current work can be
extended and embedded to any application based on
searching according to users intent. (ii) It may increase the
computation cost while using large tensors therefore
parallelization can be done by using different platform for
this framework (e.g. parallel MATLAB).
VII.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

P. Heymann, G. Koutrika, and H. Garcia-Molina, Can

social bookmarking improve web search? in WSDM,
2008, pp. 195206.
Lerman, K., Plangprasopchok, A. & Wong, C. (2007).
Personalizing Image Search Results on Flickr. In
Proceedings of AAAI workshop on Intelligent
Techniques for Information Personalization. Vancouver,
Canada, AAAI Press.
Krishanan lerman, Anon Plangprasopchok (2010)
Leveraging user specified metadata to personalize
image search www.igi-global.com/.../leveraging-userspecified-metadata-personalization. .
Google. (2014). Information Retrieval Wikipedia.
[Online].
Available:
http://en.wikipedia.org/wiki/Information_retrieval

REFERENCES

Jianping Fan, Daniel A. Keim, Yuli Gao, Hangzai Luo,

Zongmin
LI,JustClick:
Personalized
Image
Recommendation via Exploratory Search from LargeScale Flicker Images IEEE Transaction, Vol.19, No.2,
Februrary 2009.
Jitao Sang, Changsheng Xu, Dongyuan Lu, Learn to
Personalized Image Search from the Photo Sharing
Websites IEEE Transaction, Vol.14, No.4, August
2012.
Mark Sanderson, W.Bruce Croft, The History of
Information Retrieval Research, IEEE, Vol. 100, May
13th 2012.
Duygu Tumer, Mohammad Ahmed Shah, Yiltan Bitirim,
An Empirical Evaluation on Semantic Search
Performance of Keyword-Based and Semantic Search
Engines: Google, Yahoo, Msn and Hakia, Fourth
International Conference on Internet Monitoring and
Protection IEEE, 2009.
B. Smyth, A community-based approach to
personalizing web search, Computer, vol. 40, no. 8, pp.
4250, 2007.
Breese, J., Heckerman, D.& Kadie, C. (1998). Empirical
analysis of predictive algorithms for collaborative
filtering , 2008. In Proceedings of the 14th Annual
Conference on Uncertainty in Artificial Intelligence (pp.
4352). San Francisco.
Agrawal, R., & Srikant, R. (1994). Fast algorithms for
mining association rules. In Bocca, J. B., Jarke, M.&
Zaniolo, C. (Eds.), Proceedings of the 20th Int. Conf.
Very Large Data Bases, VLDB (pp. 487 499).
Dongyuan Lu, Qiudan Li Personalize search o Flickr
based on searchers preference prediction WWW 2011
Hyderabad India.
D. Carmel, N. Zwerdling, I. Guy, S. Ofek-Koifman, N.
Herell, I. Ronen,E. Uziel, S. Yogev, and S. Chernov,
Personalized social search based on the users social
network, in CIKM, 2009, pp. 12271236.
J. Teevan, S. T. Dumais, and D. J. Liebling, To
personalize or not to personalize: Modeling queries with
variation in user intent, in Proc. SIGIR, 2008, pp. 163
170.
M. J. Carman, M. Baillie, and F. Crestani,Tag data
and personalized information retrieval, in SSM, 2008,
pp. 2734.
Golder, S.A. & Huberman, B.A.(2006). The structure of
collaborative tagging systems. Journal of Information
Science 32(2), 198-208.
2071

IJRITCC | July 2014, Available @ http://www.ijritcc.org

_______________________________________________________________________________________