Vector Space Model

Jaime Arguello
INLS 509: Information Retrieval
[email protected]
 The Search Task

• Given a query and a corpus, find relevant items

query: a textual description of the user’s information need

corpus: a repository of textual documents
relevance: satisfaction of the user’s information need

2
 What is a Retrieval Model?

• A formal method that predicts the degree of relevance

of a document to a query

3
Basic Information Retrieval Process
doc
the retrieval model doc
doc
doc
doc
is responsible for
information need performing this document
comparison and
retrieving objects
representation that are likely to representation
satisfy the user

query comparison indexed objects

retrieved objects

evaluation
4
 Boolean Retrieval Models
• The user describes their information need using boolean
constraints (e.g., AND, OR, and AND NOT)
• Unranked Boolean: retrieves documents that satisfy the
constraints in no particular order
• Ranked Boolean: retrieves documents that satisfy the
constraints and orders them based on the number of ways
they satisfy the constraints
• Also known as ‘exact-match’ retrieval models

• Advantages and disadvantages?

5
 Boolean Retrieval Models
• Advantages:

‣ Easy for the system

‣ Users get transparency: it is easy to understand why a
document was or was not retrieved
‣ Users get control: it is easy to determine whether the
query is too specific (few results) or too broad (many
results)
• Disadvantages:

‣ The burden is on the user to formulate an effective

query
6
 Relevance

• Many factors affect whether a document satisfies a

particular user’s information need
• Topicality, freshness, authority, formatting, reading level,
assumed level of prior knowledge/expertise, novelty…
• Topical relevance: the document is on the same topic as
the query
• User relevance: everything else!

• For now, we will only try to predict topical relevance

7
Introduction to Best-Match Retrieval Models

• Best-match models predict the degree to which a

document is relevant to a query
• Ideally, this would be expressed as RELEVANT(q,d)

• In practice, it is expressed as SIMILAR(q,d)

8
Vector Space Model
 What is a Vector Space?
• Formally, a vector space is defined by a set of linearly
independent basis vectors
• The basis vectors correspond to the dimensions or
directions of the vector space

Y Y

basis vectors for 2- basis vectors for 3-

dimensional space dimensional space

X X

Z 10
 What is a Vector?

• A vector is a point in a vector space and has length (from

the origin to the point) and direction

Y Y

X X

Z
11
 What is a Vector?

• A 2-dimensional vector can be written as [x,y]

• A 3-dimensional vector can be written as [x,y,z]

Y Y

y y
X X
z
x
Z
12
 What is a Vector Space?

• The basis vectors are linearly independent because

knowing a vector’s value along one dimension doesn’t say
anything about its value along another dimension

Y Y

X X

basis vectors for 2- basis vectors for 3-

Z
dimensional space dimensional space
13
 Binary Text Representation

a aardvark abacus abba able ... zoom

doc_1 1 0 0 0 0 ... 1
doc_2 0 0 0 0 1 ... 1
:: :: :: :: :: :: ... 0
doc_m 0 0 1 1 0 ... 0

• 1 = the word appears in the document

• 0 = the word does not appear in the document

• Does not represent word frequency, word location, or

word order information

14
 Vector Space Representation

• Let V denote the size of the indexed vocabulary

‣ V = the number of unique terms,

‣ V = the number of unique terms excluding stopwords,
‣ V = the number of unique stems, etc...
• Any arbitrary span of text (i.e., a document, or a query)
can be represented as a vector in V-dimensional space
• For simplicity, let’s assume three index terms: dog, bite,
man (i.e., V=3)
• Why? Because it’s easy to visualize 3-D space

15
 Vector Space Representation
with binary weights

• 1 = the term appears at least once

• 0 = the term does not appear

man

“dog bite man”

dog man bite [1, 1, 1]
doc_1 1 1 1
doc_2 1 0 1 1
dog
doc_3 0 1 1 1

bite 1

16
 Vector Space Representation
with binary weights

• 1 = the term appears at least once

• 0 = the term does not appear

man

“dog bite man”

dog man bite [1, 1, 1]
doc_1 1 1 1
doc_2 1 0 1 1
dog
doc_3 0 1 1 1

bite 1 “dog bite”

[1, 0, 1]

17
 Vector Space Representation
with binary weights

• 1 = the term appears at least once

• 0 = the term does not appear

man

“dog bite man”

1
dog man bite [1, 1, 1]
doc_1 1 1 1
“bite man”
[0, 1, 1] 1 1
doc_2 1 0 1
dog
doc_3 0 1 1 1

bite 1 “dog bite”

[1, 0, 1]

18
 Vector Space Representation
with binary weights

• What span(s) of text does this vector represent?

man

1
dog

bite
19
 Vector Space Representation
with binary weights

• What span(s) of text does this vector represent?

man

dog
1

bite
20
 Vector Space Representation
with binary weights

• What span(s) of text does this vector represent?

man

1
dog
1
1
bite
21
 Vector Space Representation

• Any span of text is a vector in V-dimensional space,

where V is the size of the vocabulary

man

1 doc1: “dog bite man”

doc2: “bite man” [1,1,1]
[0,1,1]
1 1
dog
1
1 doc3: “dog bite”
bite
[1,0,1]

22
 Vector Space Representation

• A query is a vector in V-dimensional space, where V is

the number of terms in the vocabulary

man

query: “man dog”

[1,1,0]

dog

doc3: “dog bite”

bite
[1,0,1]

23
 Vector Space Similarity

• The vector space model ranks documents based on the

vector-space similarity between the query vector and
the document vector
• There are many ways to compute the similarity
between two vectors
• One way is to compute the inner product

V
∑ xi × yi
i =1

24
 The Inner Product
xi yi xi × yi
• Multiply a 1 1 1
corresponding
components and aardvark 0 1 0
then sum those abacus 1 1 1
products
abba 1 0 0

V able 0 1 0
∑ xi × yi :: 0 0 0
i =1
zoom 0 0 0
inner product => 2

25
 The Inner Product
xi yi xi × yi
• What does the a 1 1 1
inner product
(with a binary aardvark 0 1 0
representation) abacus 1 1 1
correspond to?
abba 1 0 0

V able 0 1 0
∑ xi × yi :: 0 0 0
i =1
zoom 0 0 0
inner product => 2

26
 The Inner Product
xi yi xi × yi
• When using 0’s a 1 1 1
and 1’s, this is
just the number aardvark 0 1 0
of unique terms abacus 1 1 1
in common
abba 1 0 0
between the
query and the able 0 1 0
document
:: 0 0 0

V zoom 0 0 0
∑ xi × yi inner product => 2
i =1

27
 The Inner Product

• 1 = the term appears at least once

• 0 = the term does not appear

man
“man”
[0, 1, 0] “dog bite man”
dog man bite 1
[1, 1, 1]
doc_1 1 1 1 “bite man”
doc_2 1 0 1 [0, 1, 1] 1 1
doc_3 0 1 1 dog
doc_4 0 1 0 1

bite 1 “dog bite”

[1, 0, 1]

28
 The Inner Product

• Multiply corresponding components and then sum

those products
• Using a binary representation, the inner product
corresponds to the number of terms appearing (at least
once) in both spans of text
• Scoring documents based on their inner-product with
the query has one major issue. Any ideas?

29
 The Inner Product
• What is more relevant to a query?

‣ A 50-word document which contains 3 of the query-

terms?
‣ A 100-word document which contains 3 of the
query-terms?
• All things being equal, longer documents are more
likely to have the query-terms
• The inner-product doesn’t account for the fact that
documents have widely varying lengths
• So, the inner-product favors long documents

30
 The Cosine Similarity
• The numerator is the inner product

• The denominator is the product of the two vector-lengths

• Ranges from 0 to 1 (equals 1 if the vectors are identical)

• The cosine of the angle between the two vectors

• 0 if the angle is 90 degrees

V
∑ i =1xi × yi
! !
V 2× V 2
x
∑ i =1 i y
∑ i =1 i
length of length of
vector x vector y
31
 Vector Space Model
cosine similarity example (binary weights)

V
xi × yi
∑ i =1
! !
V V
∑i=1 xi × ∑i=1 y2i
2

cosine( [1,0,1] , [1,1,0] ) =

(1 × 1) + (0 × 1) + (1 × 0)
√ √ = 0.5
12 + 02 + 12 × 12 + 12 + 02

32
 ∑V xi × yi
In Class Exercise
! i = 1 !
∑V x
i =1 i
2×
∑ V 2
i =1 y i

• For each document, compute the inner-product and

cosine similarity score for the query: Jill

doc_1 Jack and Jill went up the hill

doc_2 To fetch a pail of water.
doc_3 Jack fell down and broke his crown,
doc_4 And Jill came tumbling after.
doc_5 Up Jack got, and home did trot,
doc_6 As fast as he could caper,
doc_7 To old Dame Dob, who patched his nob
doc_8 With vinegar and brown paper.

33
 ∑V xi × yi
In Class Exercise
! i = 1 !
∑V x
i =1 i
2×
∑ V 2
i =1 y i

• For each document, compute the inner-product and

cosine similarity score for the query: Jack

doc_1 Jack and Jill went up the hill

doc_2 To fetch a pail of water.
doc_3 Jack fell down and broke his crown,
doc_4 And Jill came tumbling after.
doc_5 Up Jack got, and home did trot,
doc_6 As fast as he could caper,
doc_7 To old Dame Dob, who patched his nob
doc_8 With vinegar and brown paper.

34
 Vector Space Representation
a aardvark abacus abba able ... zoom
doc_1 1 0 0 0 0 ... 1
doc_2 0 0 0 0 1 ... 1
:: :: :: :: :: :: ... 0
doc_m 0 0 1 1 0 ... 0

a aardvark abacus abba able ... zoom

query 0 1 0 0 1 ... 1

• So far, we’ve assumed binary vectors

• 0’s and 1’s indicate whether the term occurs (at least
once) in the document/query
• Let’s explore a more sophisticated representation 35
 Term-Weighting
what are the most important terms?

• Movie: Rocky (1976)

• Plot:
Rocky Balboa is a struggling boxer trying to make the big time. Working in a meat factory in Philadelphia for
a pittance, he also earns extra cash as a debt collector. When heavyweight champion Apollo Creed visits
Philadelphia, his managers want to set up an exhibition match between Creed and a struggling boxer,
touting the fight as a chance for a "nobody" to become a "somebody". The match is supposed to be easily
won by Creed, but someone forgot to tell Rocky, who sees this as his only shot at the big time. Rocky Balboa
is a small-time boxer who lives in an apartment in Philadelphia, Pennsylvania, and his career has so far not
gotten off the canvas. Rocky earns a living by collecting debts for a loan shark named Gazzo, but Gazzo
doesn't think Rocky has the viciousness it takes to beat up deadbeats. Rocky still boxes every once in a
while to keep his boxing skills sharp, and his ex-trainer, Mickey, believes he could've made it to the top if he
was willing to work for it. Rocky, goes to a pet store that sells pet supplies, and this is where he meets a
young woman named Adrian, who is extremely shy, with no ability to talk to men. Rocky befriends her.
Adrain later surprised Rocky with a dog from the pet shop that Rocky had befriended. Adrian's brother
Paulie, who works for a meat packing company, is thrilled that someone has become interested in Adrian,
and Adrian spends Thanksgiving with Rocky. Later, they go to Rocky's apartment, where Adrian explains that
she has never been in a man's apartment before. Rocky sets her mind at ease, and they become lovers.
Current world heavyweight boxing champion Apollo Creed comes up with the idea of giving an unknown a
shot at the title. Apollo checks out the Philadelphia boxing scene, and chooses Rocky. Fight promoter
Jergens gets things in gear, and Rocky starts training with Mickey. After a lot of training, Rocky is ready for
the match, and he wants to prove that he can go the distance with Apollo. The 'Italian Stallion', Rocky
Balboa, is an aspiring boxer in downtown Philadelphia. His one chance to make a better life for himself is
through his boxing and Adrian, a girl who works in the local pet store. Through a publicity stunt, Rocky is
set up to fight Apollo Creed, the current heavyweight champion who is already set to win. But Rocky really
needs to triumph, against all the odds... 36
 Term-Frequency
how important is a term?
rank term freq. rank term freq.
1 a 22 16 creed 5
2 rocky 19 17 philadelphia 5
3 to 18 18 has 4
4 the 17 19 pet 4
5 is 11 20 boxing 4
6 and 10 21 up 4
7 in 10 22 an 4
8 for 7 23 boxer 4
9 his 7 24 s 3
10 he 6 25 balboa 3
11 adrian 6 26 it 3
12 with 6 27 heavyweigh 3
13 who 6 28 t
champion 3
14 that 5 29 fight 3
15 apollo 5 30 become 3 37
 Term-Frequency
how important is a term?
rank term freq. rank term freq.
1 a 22 16 creed 5
2 rocky 19 17 philadelphia 5
3 to 18 18 has 4
4 the 17 19 pet 4
5 is 11 20 boxing 4
6 and 10 21 up 4
7 in 10 22 an 4
8 for 7 23 boxer 4
9 his 7 24 s 3
10 he 6 25 balboa 3
11 adrian 6 26 it 3
12 with 6 27 heavyweigh 3
13 who 6 28 t
champion 3
14 that 5 29 fight 3
15 apollo 5 30 become 3 38
 Inverse Document Frequency (IDF)
how important is a term?

N
id f t = log( )
d ft

• N = number of documents in the collection

• dft = number of documents in which term t appears

39
 Inverse Document Frequency (IDF)
how important is a term?
rank term idf rank term idf
1 doesn 11.66 16 creed 6.84
2 adrain 10.96 17 paulie 6.82
3 viciousness 9.95 18 packing 6.81
4 deadbeats 9.86 19 boxes 6.75
5 touting 9.64 20 forgot 6.72
6 jergens 9.35 21 ease 6.53
7 gazzo 9.21 22 thanksgivin 6.52
8 pittance 9.05 23 g
earns 6.51
9 balboa 8.61 24 pennsylvani 6.50
10 heavyweigh 7.18 25 a
promoter 6.43
11 t
stallion 7.17 26 befriended 6.38
12 canvas 7.10 27 exhibition 6.31
13 ve 6.96 28 collecting 6.23
14 managers 6.88 29 philadelphia 6.19
15 apollo 6.84 30 gear 6.18 40
 TF.IDF
how important is a term?

t f t × id f t

greater when greater when

the term is the term is rare
frequent in in in the
the document collection
(does not
appear in many
documents)

41
 TF.IDF
! "
N
t f t × log
d ft

term tf N df idf tf.idf

rocky 19 230721 1420 5.09 96.72
philadelphia 5 230721 473 6.19 30.95
boxer 4 230721 900 5.55 22.19
fight 3 230721 8170 3.34 10.02
mickey 2 230721 2621 4.48 8.96
for 7 230721 117137 0.68 4.75

42
 TF.IDF
how important is a term?
rank term tf.idf rank term tf.idf
1 rocky 96.72 16 meat 11.76
2 apollo 34.20 17 doesn 11.66
3 creed 34.18 18 adrain 10.96
4 philadelphia 30.95 19 fight 10.02
5 adrian 26.44 20 viciousness 9.95
6 balboa 25.83 21 deadbeats 9.86
7 boxing 22.37 22 touting 9.64
8 boxer 22.19 23 current 9.57
9 heavyweigh 21.54 24 jergens 9.35
10 t
pet 21.17 25 s 9.29
11 gazzo 18.43 26 struggling 9.21
12 champion 15.08 27 training 9.17
13 match 13.96 28 pittance 9.05
14 earns 13.01 29 become 8.96
15 apartment 11.82 30 mickey 8.96 43
 TF, IDF, or TF.IDF?

,.",*-! ,."*,- ! ,22! ,2"),.& ! ,2+#! ,- ,-. ,1,"'7)-' ,1#22# ,+

! ,'
! ! ! ! ,+1*"*-5!

3,23#,! 3)$#7)! 3)'')"! 3*5 3#4)" 3#4*-5 36' 3&

! !$(,71*#- ! ! ! $,-! $,"))" !

$(,-$)! $")). $6"")-' ! ! .)3'! .#)+-! ),"-+! )8)"&! )4(*3*'*#-! )4'",! /,"! /*5('! /#" 5,99# ! ! 5)'+ ! 5*"2

5#! (,+! () (),8&0)*5(' ()"

! ! ! (*7+)2/! (*+ *- *+ *'
! */! ! ! ! %))1! 2,')"! 2*/)! 2*8*-5! 2#,-! 2#8)"+
7,%)!7,-!7,'$(!7),'!7)-!7*$%)&!-,7).!-#3#.&!#/!1,62*)! 1)'! 1(*2,.)21(*,
"#$%& +)' ! ! +()! +(#'! +7,22! +#7)3#.&! +#7)#-)! +'*22! +'#")! +'"6552*-5! +6112*)+! +6"1"*+).
'(,' '() '()&
! ! !'(*-% ! '(*+! '("#65(! '*7)! '*'2)! '# ! '",*-)" ! '",*-*-5! 61! 0,-'! 0()-! 0()")
0(# 0*'( !0*22*-5! !0#7,-!0#-! 0#"%+

44
 TF, IDF, or TF.IDF?

'/-)-5& ! '+"'-.! '+"-'. '('"56*.5 '(#))# /')/#' /*$#6*

! ')"*'+& ! ! ! '7(-"-.1! !

/#0*" /#0*7 /#0-.1 $'.3'7 $,'6(-#. $,'.$*

/*9"-*.+*+! /*9"-*.+7 ! /-1! ! ! ! ! ! ! $,*$%7

$,##7*7 !$#))*$5-.1! $#))*$5#" ! $"**+ $8""*.5 +*'+/*'57 ! ! ! +*/5! +*/57! +-75'.$*! +#*7.! +#4.5#4.
*'".7! *'7*! *'7-)& ! *0,-/-5-#.! *05"'! *05"*6*)& ! 9'$5#"& ! 9-1,5! 9#"1#5! 1'22# ! 1*'"! 1#55*.

,*'3&4*-1,5 ! ,-7 ! -7 ! :*"1*.7! )'5*"! )#'.! )#5! )#3*"7! 6'.'1*"7! 6'5$,! 6*'5! 6-$%*&! .'6*+
.#/#+& ! #++7 ! ('$%-.1! ('8)-*! (*..7&)3'.-'! (*5 (,-)'+*)(,-' (-55'.$* ("#6#5*"
! ! !

(8/)-$-5&! "*'+& ! "#$%& ! 7*))7 ! 7*5! 7,'"% ! 7,'"(! 7,#5! 7,& ! 7#6*/#+&! 7#6*#.*! 75'))-#.! 75#"*
75"811)-.1! 758.5! 78(()-*7! 78((#7*+! 78"("-7*+! 5,'.%71-3-.1! 5,-.%! 5,"-))*+! 5-6*! 5-5)*! 5#85-.1! 5"'-.*"! 5"'-.-.1
5"-86(,!8(!3*!3-$-#87.*77!3-7-57 !4,*"*!4,#!4-))-.1!4#.!4#"%7

45
 TF, IDF, or TF.IDF?
'*)3)+5! '"(')& '"()'& '3($'"5 '/#33# '%/)()&, *'3*#'
! ! ! ! !

*$'+! *$7()$&"$"
*$7()$&"%! *$++$(! *#8$(! *#8$%! *#8)&,
!

0'&2'%! 0'%4! 04'9/)#&! 04$0:%! 04##%$%! 0#33$0+)&,

0#33$0+#(! 0($$"! 01(($&+! ! "$*+! "$*+% "$'"*$'+%
")%+'&0$! "#$%& "#6&+#6& $'(&% $'%$ $'%)35 ! ! ! !

$84)*)+)#& $8/3')&% $8+(' $8+($9$35 7'0+#(5

! 7#(,#+
! ! ! ! 7'( !

,'--# ,$'( ,)2)&, ,#++$& 4$'256$),4+ )"$' )&+$($%+$"

! ! ! ! ! !

)+'3)'& .$(,$&%
! 3#'& 3#+ 3#2$(% 9'&',$(% 9'+04 9$'+
! :$$/! 3)2)&,! ! ! ! ! !

9)0:$5 &#*#"5 #""% /'0:)&, /'13)$ /$&&%532'&)' /$+

! ! ! ! ! !

/4)3'"$3/4)' /)++'&0$ /(#9#+$( /(#2$ /1*3)0)+5

! ! ! !

($'"5 (#0:5 %$33% %4'(: %4'(/ %4#/ %45 %:)33% %#9$*#"5 %/$&"%
! ! ! ! ! ! ! ! !

%+'33)#& %+(1,,3)&, %+1&+ %1//3)$% %1//#%$" %1(/()%$"

! ! ! ! !

+4'&:%,)2)&, +4)&: +4()33$" +)+3$ +#1+)&, +(')&$( +(')&)&,

! ! ! ! ! !

+()19/4 1&:&#6& 2$ 2)0)#1%&$%% 2)%)+% 6)33)&, 6)&

! ! ! ! ! 6'&+! !

6#&
46
 Queries as TF.IDF Vectors

• Terms tend to appear only once in the query

• TF usually equals 1

• IDF is computed using the collection statistics

N
id f t = log( )
d ft

• Terms appearing in fewer documents get a higher weight

47
 Queries as TF.IDF Vectors
examples from AOL queries with clicks on IMDB results
term 1 tf.idf term 2 tf.idf term 3 tf.idf
central 4.89 casting 6.05 ny 5.99
wizard 6.04 of 0.18 oz 6.14
sam 2.80 jones 3.15 iii 2.26
film 2.31 technical 6.34 advisors 8.74
edie 7.41 sands 5.88 singer 3.88
high 3.09 fidelity 7.66 quotes 8.11
quotes 8.11 about 1.61 brides 6.71
title 4.71 wave 5.68 pics 10.96
saw 4.87 3 2.43 trailers 7.83
the 0.03 rainmaker 9.09 movie 0.00
nancy 5.50 and 0.09 sluggo 9.46
audrey 6.30 rose 4.52 movie 0.00
mark 2.43 sway 7.53 photo 5.14
piece 4.59 of 0.18 cheese 6.38
date 3.93 movie 0.00 cast 0.00 48
 Vector Space Model
cosine similarity example (tf.idf weights)

V
xi × yi
∑ i =1
! !
V V
∑i=1 xi × ∑i=1 y2i
2

cosine( [2.3, 0.0, 1.5] , [5.4, 2.0, 0.0] ) =

(2.3 ⇥ 5.4) + (0.0 ⇥ 2.0) + (1.5 ⇥ 0.0)

p p
2.32 + 0.02 + 1.52 ⇥ 5.42 + 2.02 + 0.02

49
 Vector Space Model
cosine similarity example (tf.idf weights)

• Rank documents based on cosine similarity to the query

dog
doc_1
query

doc_2
bite

50
 TF.IDF

• Many variants of this formula have been proposed

• However, they all have two components in common:

‣ TF: favors terms that are frequent in the document

‣ IDF: favors terms that do not occur in many
documents
! "
N
t f t × log
d ft

51
 Independence Assumption
• The basis vectors (X, Y, Z) are linearly independent
because knowing a vector’s value on one dimension
doesn’t say anything about its value along another
dimension
Y =man

does this hold true

for natural language
text?
X = bite

Z = dog
basis vectors for 3-dimensional space
52
 Mutual Information
IMDB Corpus
• If this were true, what would these mutual information
values be?
w1 w2 MI w1 w2 MI
francisco san ? dollars million ?
angeles los ? brooke rick ?
prime minister ? teach lesson ?
united states ? canada canadian ?
9 11 ? un ma ?
winning award ? nicole roman ?
brooke taylor ? china chinese ?
con un ? japan japanese ?
un la ? belle roman ?
belle nicole ? border mexican ? 53
 Mutual Information
IMDB Corpus

• These mutual information values should be zero!

w1 w2 MI w1 w2 MI
francisco san 6.619 dollars million 5.437
angeles los 6.282 brooke rick 5.405
prime minister 5.976 teach lesson 5.370
united states 5.765 canada canadian 5.338
9 11 5.639 un ma 5.334
winning award 5.597 nicole roman 5.255
brooke taylor 5.518 china chinese 5.231
con un 5.514 japan japanese 5.204
un la 5.512 belle roman 5.202
belle nicole 5.508 border mexican 5.186 54
 Independence Assumption

• The vector space model assumes that

terms are independent
• The fact that one occurs says nothing
about another one occurring Y
• This is viewed as a limitation

• However, the implications of this

limitation are still debated
• A very popular solution X

Z
55
 Vector Space Model

• Any text can be seen as a vector in V-dimensional space

‣ a document
‣ a query
‣ a sentence
‣ a word
‣ an entire encyclopedia
• Rank documents based on their cosine similarity to query

• If a document is similar to the query, it is likely to be

relevant (remember: topical relevance!)

56
 Vector Space Representation

• A power tool!

• A lot of problems in IR can be cast as:

‣ Find me _____ that is similar to _____ !

• As long as _____ and _____ are associated with text,
one potential solution is:
‣ represent these items as tf.idf term-weight vectors
and compute their cosine similarity
‣ return the items with the highest similarity

57
 Vector Space Representation
• Find documents that are similar to this query

58
 Vector Space Representation
• Find ads that are similar to these results

59
 Vector Space Representation
• Find ads similar to this this document

60
 Vector Space Representation
• Find queries that are similar to this query

61
 Vector Space Representation
• Topic categorization: automatically assigning a
document to a category

62
 Vector Space Representation
• Find documents (with a known category assignment)
that are similar to this document

63
 Vector Space Representation
• Find documents (with a known category assignment)
that are similar to this document

computers
sports
politics

64
 Summary

• Any text can be seen as a vector in V-dimensional space

‣ a document
‣ a query
‣ a sentence
‣ a word
‣ an entire encyclopedia
• Rank documents based on their cosine similarity to query

• If a document is similar to the query, it is likely to be

relevant (remember: topical relevance!)

65