Unifying User-based and Item-based Collaborative Filtering Approaches by Similarity Fusion

Jun Wang1 , Arjen P. de Vries1, 2 , Marcel J.T. Reinders1

Information and Communication Theory Group1 , Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology Mekelweg 4, 2628 CD Delft, The Netherlands CWI2 , Amsterdam, The Netherlands
{jun.wang,m.j.t.reinders}@tudelft.nl,

[email protected]

ABSTRACT
Memory-based methods for collaborative ltering predict new ratings by averaging (weighted) ratings between, respectively, pairs of similar users or items. In practice, a large number of ratings from similar users or similar items are not available, due to the sparsity inherent to rating data. Consequently, prediction quality can be poor. This paper reformulates the memory-based collaborative ltering problem in a generative probabilistic framework, treating individual user-item ratings as predictors of missing ratings. The nal rating is estimated by fusing predictions from three sources: predictions based on ratings of the same item by other users, predictions based on dierent item ratings made by the same user, and, third, ratings predicted based on data from other but similar users rating other but similar items. Existing user-based and item-based approaches correspond to the two simple cases of our framework. The complete model is however more robust to data sparsity, because the dierent types of ratings are used in concert, while additional ratings from similar users towards similar items are employed as a background model to smooth the predictions. Experiments demonstrate that the proposed methods are indeed more robust against data sparsity and give better recommendations.

1.

INTRODUCTION

Categories and Subject Descriptors

H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval - Information Filtering

General Terms
Algorithms, Performance, Experimentation

Keywords
Recommender Systems, Collaborative Filtering, Smoothing, Similarity Fusion

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and the full citation on the rst page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specic permission and/or a fee. SIGIR06, August 611, 2006, Seattle, Washington, USA. Copyright 2006 ACM 1-59593-369-7/06/0008 ...$5.00.

Collaborative ltering aims at predicting the user interest for a given item based on a collection of user proles. Commonly, these proles either result from asking users explicitly to rate items or are inferred from log-archives ([7]). Research started with memory-based approaches to collaborative ltering, that can be divided in user-based approaches like [1, 5, 9, 14] and item-based approaches like [3, 15]. The former approaches form a heuristic implementation of the Word of Mouth phenomenon. Memory-based approaches are widely used in practice, e.g., [5, 11]. Given an unknown test rating (of a test item by a test user) to be estimated, memory-based collaborative ltering rst measures similarities between test user and other users (user-based), or, between test item and other items (itembased). Then, the unknown rating is predicted by averaging the (weighted) known ratings of the test item by similar users (user-based), or the (weighted) known ratings of similar items by the test user (item-based). In both cases, only partial information from the data embedded in the user-item matrix is employed to predict unknown ratings (using either correlation between user data or correlation between item data). Because of the sparsity of user prole data however, many related ratings will not be available for the prediction, Therefore, it seems intuitively desirable to fuse the ratings from both similar users and similar items, to reduce the dependency on often missing data. Also, methods known previously ignore the information that can be obtained from ratings made by other but similar users to the test user on other but similar items. Not using such ratings causes the data sparsity problem of memory-based approaches to collaborative ltering: for many users and items, no reliable recommendation can be made because of a lack of similar ratings. This paper sets up a generative probabilistic framework to exploit more of the data available in the user-item matrix, by fusing all ratings with predictive value for a recommendation to be made. Each individual rating in the user-item matrix is treated as a separate prediction for the unknown test rating (of a test item from a test user). The condence of each individual prediction can be estimated by considering both its similarity towards the test user and that towards the test item. The overall prediction is made by averaging the individual ratings weighted by their condence. The more similar a rating towards the test rating, the higher the weight assigned to that rating to make the prediction. Under this

framework, the item-based and user-based approaches are two special cases, and these can be systematically combined. By doing this, our approach allows us to take advantage of user correlations and item correlations embedded in the user-item matrix. Besides, smoothing from a background model (estimated from known ratings of similar items by similar users) is naturally integrated into our framework to improve probability estimation and counter the problem of data sparsity. The remainder of the paper is organized as follows. We rst summarize related work, introduce notation, and present additional background information for the two main memorybased approaches, i.e., user-based and item-based collaborative ltering. We then introduce our similarity fusion method to unify user-based and item-based approaches. We provide an empirical evaluation of the relationship between data sparsity and the dierent models resulting from our framework, and nally conclude our work.

approaches to collaborative ltering [5, 15]. For M items and K users, the user proles are represented in a K M user-item matrix X (Fig. 1(a)). Each element xk,m = r indicates that user k rated item m by r, where r {1, ..., |r|} if the item has been rated, and xk,m = means that the rating is unknown. The user-item matrix can be decomposed into row vectors: X = [u1 , . . . , uK ]T , uk = [xk,1 , . . . , xk,M ]T , k = 1, . . . , K where T denotes transpose. Each row vector uT k corresponds to a user prole and represents a particular users item ratings. As discussed below, this decomposition leads to userbased collaborative ltering. Alternatively, the matrix can also be represented by its column vectors: X = [i1 , ..., iM ], im = [x1,m , ..., iK,m ]T , m = 1, ..., M where each column vector im corresponds to a specic items ratings by all K users. This representation results in itembased recommendation algorithms.

2.

RELATED WORK

Collaborative ltering approaches are often classied as memory-based or model-based. In the memory-based approach, all rating examples are stored as-is into memory (in contrast to learning an abstraction). In the prediction phase, similar users or items are sorted based on the memorized ratings. Based on the ratings of these similar users or items, a recommendation for the test user can be generated. Examples of memory-based collaborative ltering include user-based methods [1, 5, 9, 14] and item-based methods [3, 15]. The advantage of the memory-based methods over their model-based alternatives is that less parameters have to be tuned; however, the data sparsity problem is not handled in a principled manner. In the model-based approach, training examples are used to generate a model that is able to predict the ratings for items that a test user has not rated before. Examples include decision trees [1], aspect models [7, 17] and latent factor models [2]. The resulting compact models solve the data sparsity problem to a certain extent. However, the need to tune an often signicant number of parameters has prevented these methods from practical usage. Lately, researchers have introduced dimensionality reduction techniques to address data sparsity [4, 13, 16]. However, as pointed out in [8, 19], some useful information may be discarded during the reduction. Recently, [8] has explored a graph-based method to deal with data sparsity, using transitive associations between user and items in the bipartite user item graph. [18] has extended the probabilistic relevance model in text retrieval ([6]) to the problem of collaborative ltering and a linear interpolation smoothing has been adopted. These approaches are however limited to binary rating data. Another recent direction in collaborative ltering research combines memory-based and model-based approaches [12, 19]. For example, [19] clusters the user data and applies intra-cluster smoothing to reduce sparsity. The framework proposed in our paper extends this idea to include item-based recommendations into the nal prediction, and does not require to cluster the data set a priori.

3.1

User-based Collaborative Filtering

User-based collaborative ltering predicts a test users interest in a test item based on rating information from similar user proles [1, 5, 14]. As illustrated in Fig. 1(b), each user prole (row vector) is sorted by its dis-similarity towards the test users prole. Ratings by more similar users contribute more to predicting the test item rating. The set of similar users can be identied by employing a threshold or selecting top-N. In the top-N case, a set of top-N similar users Su (uk ) towards user k can be generated according to: Su (uk ) = {ua |rank su (uk , ua ) N, xa,m = } (1)

where |Su (uk )| = N . su (uk , ua ) is the similarity between users k and a. Cosine similarity and Pearsons correlation are popular similarity measures in collaborative ltering, see e.g. [1, 5]. The similarity could also be learnt from training data [9]. This paper adopts the cosine similarity measure, comparing two user proles by the cosine of the angle between the corresponding row vectors. Consequently, the predicted rating xk,m of test item m by test user k is computed as (see also [1, 5]) su (uk , ua )(xa,m ua ) xk,m = uk +
ua Su (uk )

(2) su (uk , ua )

ua Su (uk )

3.

BACKGROUND
This section introduces briey the user- and item-based

where uk and ua denote the average rating made by users k and a, respectively. Existing methods dier in their treatment of unknown ratings from similar users (xa,m = ). Missing ratings can be replaced by a 0 score, which lowers the prediction, or the average rating of that similar user could be used [1, 5]. Alternatively, [19] replaces missing ratings by an interpolation of the users average rating and the average rating of his or her cluster. Before we discuss its dual method, notice in Eq. 2 and the illustration in Fig. 1(b) how user-based collaborative ltering takes only a small proportion of the user-item matrix into consideration for recommendation. Only the known test item ratings by similar users are used. We refer to these ratings as the set of similar user ratings (the blocks

i1 u1

im

iM

im

im
xk ,M
x1, m

Sorted Item Dis-similarity

x1, m

uk x k ,1

xk,m ?
Rating Prediction

im u k xk,m ?
Sorted User Dis-similarity

Sorted Item Dis-similarity

Ra tin

Sorted User Dis-similarity

gP

red icti

uk

uK

Figure 1: (a) The user-item matrix (b) Rating prediction based on user similarity (c) Rating prediction based on item similarity (d) Rating prediction based on rating similarity. with upward diagonal pattern in Fig. 1(b)): SURk,m = {xa,m |ua Su (uk )}. For simplicity, we drop the subscript k, m of SURk,m in the remainder of the paper. and columns according to their dis-similarities towards the test user and the test item respectively. In the remainder, this part of the matrix is referred to as similar user item ratings (the grid blocks in Fig. 1(d)): SUIRk,m = {xa,b |ua Su (uk ), ib Si (im ), a = k, b = m}. The subscript k, m of SUIRk,m is dropped. Combining these three types of ratings in a single collaborative ltering method is non-trivial. We propose to treat each element of the user-item matrix as a separate predictor. Its reliability or condence is then estimated based upon its similarity towards the test rating. We then predict the test rating by averaging the individual predictions weighted by their condence. The remainder of the section gives a probabilistic formulation for the proposed method.

3.2

Item-based approaches such as [3, 11, 15] apply the same idea, but use similarity between items instead of users. As illustrated in Fig. 1(c), the unknown rating of a test item by a test user can be predicted by averaging the ratings of other similar items rated by this test user [15]. Again, each item (column vector) is sorted and re-indexed according to its dis-similarity towards the test item in the user-item matrix, and, ratings from more similar items are weighted stronger. Formally (see also [15]), si (im , ib )(xk,b ) xk,m =
ib Si (im )

Where item similarity si (im , ib ) can be approximated by the cosine measure or Pearson correlation [11, 15]. To remove the dierence in rating scale between users when computing the similarity, [15] has proposed to adjust the cosine similarity by subtracting the users average rating from each co-rated pair beforehand. We adopt this similarity measure in this paper. Like the top-N similar users, a set of top-N similar items towards item m, denoted as Si (im ), can be generated according to: Si (im ) = {ib |rank si (im , ib ) N, xk,b = } (4)

Fig. 1(c) illustrates how Eq. 3 takes only the known similar item ratings by the test user into account for prediction. We refer to these ratings as the set of similar item ratings (the blocks with downward diagonal pattern in Fig. 1(c)): SIRk,m = {xk,b |ib Si (im )}. Again, for simplicity, we drop the subscript k, m of SIRk,m in the remainder of the paper.

4.

Relying on SUR or SIR data only is undesirable, especially when the ratings from these two sources are quite often not available. Consequently, predictions are often made by averaging ratings from not-so-similar users or items. We propose to improve the accuracy of prediction by fusing the SUR and SIR data, to complement each other under the missing data problem. Additionally, we point out that the user-item matrix contains useful data beyond the previously used SUR and SIR ratings. As illustrated in Fig. 1 (d), the similar item ratings made by similar users may provide an extra source for prediction. They are obtained by sorting and re-indexing rows

on

g tin Ra

x k ,1

xk , m ?

xk ,M

u k xk,m ?

Rating Prediction

ion ict ed Pr

SUIR SUR SIR Unknown Rating

SUR Unknown Rating

x K ,m

SIR

xK ,m

Unknown Rating

(a)

(b)

(c)

(d)

Item-based Collaborative Filtering

4.1
(3)

Individual Predictors

si (im , ib )
ib Si (im )

Users rate items dierently. Some users have a preference for the extreme values of the rating scale, while others rarely deviate from the median. Likewise, items may be rated by dierent types of users. Some items get higher ratings than their true value, simply because they have been rated by a positive audience. Addressing the dierences in rating behavior, we rst normalize the user-item matrix before making predictions. Removing the mean ratings per user and item gives individual predictions as pk,m (xa,b ) = xa,b ( xa x k ) ( xb x m ) (5)

where pk,m (xa,b ) is the prediction function for the test item k rating made by test user m, x a and x k are the average ratings by user a and k, and x b and x m are the average ratings of item b and m. Appendix A derives that normalizing the matrix by independently subtracting the row and column means gives the same result.

4.2

Probabilistic Fusion Framework

SIMILARITY FUSION

Let us rst dene the sample space of ratings as r = {, 1, ..., |r|} (like before, denotes the unknown rating). Let xa,b be a random variable over the sample space r , captured in the user-item matrix, a {1, . . . , K } and b {1, . . . , M }. Collaborative ltering then corresponds to estimating conditional probability P (xk,m |Pk,m ), for an unknown test rating xk,m , given a pool of individual predictors Pk,m = {pk,m (xa,b )|xa,b = }. Consider rst a pool that consists of SUR and SIR ratings only (i.e., xa,b (SUR SIR)). P (xk,m |SUR, SIR) P (xk,m |{pk,m (xa,b )|xa,b SUR SIR}) (6)

We write P (xk,m |SUR, SIR) for the conditional probability depending on the predictors originating from SUR and SIR. Likewise, P (xk,m |SUR) and P (xk,m |SIR) specify a pool consisting of SUR or SIR predictors only. Now introduce a binary variable I1 , that corresponds to the relative importance of SUR and SIR. This hidden variable plays the same role as the prior introduced in [6] to capture the importance of a query term in information retrieval. I1 = 1 states that xk,m depends completely upon ratings from SUR, while I1 = 0 corresponds to full dependency on SIR. Under these assumptions, the conditional probability can be obtained by marginalization of variable I1 : P (xk,m |SUR, SIR)

Finally, the following equation gives the expected value of the unknown test rating:
|r |

x k,m =
r =1

rP (xk,m = r|SUR, SIR, SUIR)

|r |

=
r =1

rP (xk,m = r|SUIR) +
|r |

(13) rP (xk,m = r|SUR)(1 ) +

r =1 |r |

rP (xk,m = r|SIR)(1 )(1 )

r =1

=
I1

P (xk,m |SUR, SIR, I1 )P (I1 |SUR, SIR)

= P (xk,m |SUR, SIR, I1 = 1)P (I1 = 1|SUR, SIR)+ P (xk,m |SUR, SIR, I1 = 0)P (I1 = 0|SUR, SIR)

(7)

The resulting model can be viewed as using importance sampling of the neighborhood ratings as predictors. and control the selection (sampling) of data from the three dierent sources.

By denition, xk,m is independent from SUR when I1 = 1, so P (xk,m |SUR, SIR, I1 = 1) = P (xk,m |SUR). Similarly, P (xk,m |SUR, SIR, I1 = 0) = P (xk,m , |SIR). If we provide a parameter as shorthand for P (I1 = 1|SUR, SIR), we have P (xk,m |SUR, SIR) = P (xk,m |SUR) + P (xk,m |SIR)(1 ) (8)

4.3

Probability Estimation

Next, we extend the model to take into account the SUIR ratings: P (xk,m |SUR, SIR, SUIR) P (xk,m |{pk,m (xk,m )|xa,b SUR SIR SUIR})

(9)

We introduce a second binary random variable I2 , that corresponds to the relative importance of the SUIR predictors. I2 = 1 species that the unknown rating depends on ratings from SUIR only and I2 = 0 that it depends on the ratings from SIR and SUR instead. Marginalization on variable I2 gives: P (xk,m |SUR, SIR, SUIR) =
I2

The next step is to estimate the probabilities in the fusion framework expressed in Eq. 13. and are determined experimentally by using the crossvalidation, for example following the methodology of Section 5.3. The three remaining probabilities can be viewed as estimates of the likelihood of a rating xa,b from SIR, SUR, or SUIR, to be similar to the test rating xk,m . We assume that the probability estimates for SUR and SIR are proportional to the similarity between row vectors su (uk , ua ) (Section 3.1) and column vectors si (im , ib ) (Section 3.2), respectively. For SUIR ratings, we assume the probability estimate to be proportional to the combination of su and si . To combine them, we use a Euclidean dis-similarity space such that the resulting combined similarity is lower than either of them. sui (xk,m , xa,b ) = 1 (1/su (uk , ua ))2 + (1/si (im , ib ))2 (14)

This results in the following conditional probability estimates: P (xk,m = r|SUR) su (uk , ua ) =
xa,b :(xa,b SUR)(pk,m (xa,b )=r )

P (xk,m |SUR, SIR, SUIR, I2 )P (I2 |SUR, SIR, SUIR)

= P (xk,m |SUR, SIR, SUIR, I2 = 1) P (I2 = 1|SUR, SIR, SUIR)+ P (xk,m |SUR, SIR, SUIR, I2 = 0) (1 P (I2 = 1|SUR, SIR, SUIR)) (10)

su (uk , ua )
xa,b :xa,b SUR

P (xk,m = r|SIR) si (im , ib ) =

xa,b :(xa,b SIR)(pk,m (xa,b )=r )

Following the argument from above and providing a parameter as shorthand for P (I2 = 1|SUR, SIR, SUIR), we have P (xk,m |SUR, SIR, SUIR) = P (xk,m |SUR, SIR)(1 ) + P (xk,m |SUIR) Substitution of Eq. 8 then gives: P (xk,m |SUR, SIR, SUIR) = P (xk,m |SUR) + P (xk,m |SIR)(1 ) (1 )+ P (xk,m |SUIR) (12) = (11)

si (im , ib )
xa,b :xa,b SIR

P (xk,m = r|SUIR) sui (xk,m , xa,b )

xa,b :(xa,b SUIR)(pk,m (xa,b )=r )

(15)

sui (xk,m , xa,b )

xa,b :xa,b SUIR

Table 1: Percentage of the ratings that are available (= ).

test user 1st most sim. user 2nd most sim. user 3rd most sim. user 4th most sim. user test item 0.54 0.51 0.51 0.49 1st most sim. item 0.58 0.58 0.56 0.57 0.55 2nd most sim item 0.56 0.58 0.56 0.57 0.55 3rd most sim. item 0.55 0.58 0.56 0.57 0.56 4th most sim. item 0.54 0.57 0.56 0.56 0.55

Table 2: Mean Absolute Err (MAE) of individual predictions.

test user 1st most sim. user 2nd most sim. user 3rd most sim. user 4th most sim. user test item 0.914 0.917 0.927 0.928 1st most sim. item 0.824 0.925 0.921 0.947 0.929 2nd most sim. item 0.840 0.927 0.931 0.952 0.939 3rd most sim. item 0.866 0.942 0.935 0.953 0.946 4th most sim. item 0.871 0.933 0.927 0.945 0.932

After substitution from Eq. 15 (for readability, we put the detailed derivations in Appendix B), Eq. 13 results in: xk,m =
xa,b a,b pk,m (xa,b )Wk,m

5. 5.1

EMPIRICAL EVALUATION Experimental Setup

(16)

where
su (uk ,ua ) su (uk ,ua ) xa,b SUR si (im ,ib ) si (im ,ib ) xa,b SIR

(1 )

xa,b SUR xa,b SIR (17) xa,b SUIR otherwise

a,b Wk,m =

(1 )(1 )

xa,b SUIR

sui (xk,m ,xa,b ) sui (xk,m ,xa,b )

0
xa,b

It is easy to prove that

a,b a,b Wk,m = 1. Wk,m acts as a

unied weight matrix to combine the predictors from the three dierent sources.

4.4

Discussion

Sum as Combination Rule and control the importance of the dierent rating sources. Their introduction results in a sum rule for fusing the individual predictors (Eq. 12 and 16.). Using the independence assumption on the three types of ratings and the Bayes rule, one can easily derive a product combination from the conditional probability ([10]). However, the high sensitivity to estimation errors makes this approach less attractive in practice. We refer to [10] for a more detailed discussion of using a sum rule vs. the product rule for combing classiers. Unied Weights The unied weights in Eq. 17 provide a generative framework for memory-based collaborative ltering. Eq. 17 shows how our scheme can be considered as two subsequent steps of linear interpolation. First, predictions from SUR ratings are interpolated with SIR ratings, controlled by . Next, the intermediate prediction is interpolated with predictions from the SUIR data, controlled by . Viewing the SUIR ratings as a background model, the second interpolation corresponds to smoothing the SIR and SUR predictions from the background model. A bigger emphasizes user correlations, while smaller emphasizes item correlations. When equals one, our algorithm corresponds to a user-based approach, while equal to zero results in an item-based approach. Tuning parameter controls the impact of smoothing from the background model (i.e. SUIR ). When approaches zero, the fusion framework becomes the mere combination of userbased and item-based approaches without smoothing from the background model.

We experimented with the MovieLens1 , EachMovie2 , and book-crossing3 data sets. While we report only the MovieLens results (out of space considerations), the model behaves consistently across the three data sets. The MovieLens data set contains 100,000 ratings (1-5 scales) from 943 users on 1682 movies (items), where each user has rated at least 20 items. To test on dierent number of training users, we selected the users in the data set at random into a training user set (100, 200, 300 training users, respectively) and the remaining users into a test user set. Users in the training set are only used for making predictions, while test users are the basis for measuring prediction accuracy. Each test users ratings have been split into a set of observed items and one of held-out items. The ratings of observed items are input for predicting the ratings of held-out items. We are specically interested in the relationship between the density of the user-item matrix and the collaborative ltering performance. Consequently, we set up the following congurations: Test User Sparsity Vary the number of items rated by test users in the observed set, e.g., 5, 10, or 20 ratings per user. Test Item Sparsity Vary the number of users who have rated test items in the held-out set; less than 5, 10, or 20 (denoted as < 5, < 10, or < 20), or, unconstrained (denoted as No constraint). Overall Training User Sparsity Select a part of the rating data at random, e.g., 20%, 40%, 60% of the data set. For consistency with experiments reported in the literature, e.g., [9, 15, 19]), we report the mean absolute error (MAE) evaluation metric. MAE corresponds to the average absolute deviation of predictions to the ground truth data, for all test item ratings and test users: |xk,m xk,m | M AE =
k,m

(18)

where L denotes the number of tested ratings. A smaller value indicates a better performance. http://www.grouplens.org/ http://research.compaq.com/SRC/eachmovie/ 3 http://www.informatik.uni-freiburg.de/cziegler/ BX/
2 1

1.25 1.2 1.15 Rating Per User: 5, Rating Per Item: 5 Rating Per User: 5 Rating Per User: 20, Rating Per Item: 5 Rating Per User: 20

1.15 Rating Per User: 5, Rating Per Item: 5 Rating Per User: 5 Rating Per User: 20, Rating Per Item: 5 Rating Per User: 20

0.795 SF2

1.1

0.79

1.05 1.1 1.05 MAE 1 0.95 0.9 0.85 0.85 0.8 0.75 0.8
0.77 0.785

MAE

MAE

0.95

0.78

0.9
0.775

0.1

0.2

0.3

0.4

0.5 lambda

0.6

0.7

0.8

0.9

0.75

0.1

0.2

0.3

0.4

0.5 delta

0.6

0.7

0.8

0.9

0.765

50

100

150

200 250 300 Num. of Neighborhood Ratings

350

400

450

500

(a) Lambda

(b) Delta Figure 3: Size of neighborhood. of SUIR ratings, which we called fusion scheme SF2. We observe that reaches its optimal in 0.8 when the rating data is sparse in the neighborhood ratings from the item and user aspects (upper two curves). In other words, smoothing from a pool of SUIR ratings improves the performance for sparse data. However, when the test item sparsity is not constrained, its optimum spreads a wide range of values, and the improvement over MAE without smoothing ( = 0) is not clear. Additional experiments (not reported here) veried that there is little dependency between the choice of and the optimal value of . The optimal parameters can be identied by using the cross validation from the training data. Like pure user-based and item-based approaches, the size of neighborhood N also inuences the performance of our fusion methods. Fig. 3 shows MAE of SF2 when the number of neighborhood ratings is varied. The optimal results are obtained with the neighborhood size between 50 and 100. We select 50 as our optimal choice.

Figure 2: Impact of the two parameters.

5.2

Individual Predictors

We rst report some properties of the three types of individual predictions used in our approach. Table 1 illustrates the availability of the top-4 neighborhood ratings in the MovieLens data set. The rst column contains the top-4 SUR ratings, the rst row the top-4 SIR ratings; the remaining cells correspond to the top-4x4 SUIR ratings. We observe that only about half of these ratings are given. Table 2 summarizes recommendation MAE of individual predictors (applying Eq. 5) using leave-one-out cross-validation. Clearly, more similar ratings provide more accurate predictions. While SUIR s ratings are in general less accurate than SUR s and SIR s, these may indeed complement missing or unreliable SIR and SUR ratings.

5.3

Impact of Parameters

Recall the two parameters in Eq. 17: balances the predictions between SUR and SIR, and smoothes the fused results by interpolation with a pool of SUIR ratings. We rst test the sensitivity of , setting to zero. This scheme, called SF1, combines user-based and item-based approaches, but does not use additional background information. Fig. 2(a) shows recommendation MAE against varying from zero (a pure item-based approach) to one (a pure user-based approach). The graph plots test user sparsity 5 and 20, and test item sparsity settings < 5 and unconstrained. The value of the optimal demonstrates that interpolation between user-based and item-based approaches (SF1 ) improves the recommendation performance. More specically, the best results are obtained with between 0.6 and 0.9. This optimal value emphasizing the SUR ratings may be somewhat surprising, as Table 2 indicated that the SIR ratings should be more reliable for prediction. However, in the data sets considered, the number of users is smaller than the number of items, causing the user weights su (uk , ua ) to be generally smaller than the item weights si (im , ib ). When removing the constraint on test item sparsity, the optimal shifts down from about 0.9 for the two upper curves (< 5) to 0.6 for the two lower curves (unconstrained). A lower conrms the expectation that SIR ratings gain value when more items have been rated. Fig. 2 (b) shows the sensitivity of after xing to 0.7. The graph plots the MAE for the same four congurations when parameter is varied from zero (without smoothing) to one (rely solely on the background model: SUIR ratings). When is non-zero, the SF1 results are smoothed by a pool

5.4

Data Sparsity

The next experiments investigate the eect of data sparsity on the performance of collaborative ltering in more detail. Fig. 4(a) and (b) compare the behavior of scheme SF1 to that obtained by simply averaging user-based and itembased approaches, when varying test user sparsity (Fig. 4(a)) and test item sparsity (4(b)). The results indicate that combining user-based and item-based approaches (SF1 ) consistently improves the recommendation performance regardless neighborhood sparsity of test users or items. Next, Fig. 4(c) plots the gain of SF2 over SF1 when varying overall training user sparsity. The gure shows that SF2 improves SF1 more and more when the rating data becomes more sparse. This can be explained as follows. When the user-item matrix is less dense, it contains insucient test item ratings by similar users (for user-based recommendation), and insucient similar item ratings by the test user (for item-based recommendation) as well. Therefore, smoothing using ratings by similar items made by similar users improves predictions. We conclude from these experiments that the proposed fusion framework is eective at improving the quality of recommendations, even when only sparse data are available.

5.5

Comparison to Other Methods

We continue with a comparison to results obtained with other methods, setting to 0.7 and to 0 for SF1 and

1.1 SF1 Avg. of Userbased and Itembased 1.05

0.94 SF1 Avg. of Userbased and Itembased 0.92

0.88 SF2 SF1 0.86

1 0.9 0.84 0.95 0.88 MAE MAE 0.9 MAE 0.86 0.85 0.8 0.84 0.8 0.78 0.82 0.8 2 4 6 8 10 12 14 Max Num. of Given Rating Per Test Item 16 18 20 0.82

0.75

0.7

10 15 Num. of Given Rating Per Test User

20

25

0.76 0.1

0.2

0.3

0.4

0.5 0.6 Sparsity %

0.7

0.8

0.9

(a) Test User Sparsity

(b) Test Item Sparsity (c) Overall Training User Sparsity Figure 4: Performance under dierent sparsity.
< 5 10 0.966 1.007 1.034 1.055 < 5 10 0.945 0.960 1.028 1.071 < 5 10 0.908 0.968 0.971 1.043 < 10 10 0.917 0.942 0.972 0.992 < 10 10 0.875 0.883 0.962 1.006 < 10 10 0.885 0.908 0.919 0.990 < 20 10 0.879 0.898 0.913 0.954 < 20 10 0.802 0.804 0.842 0.875 < 20 10 0.828 0.834 0.877 0.910 No 5 0.825 0.836 0.891 0.938 No 5 0.806 0.808 0.879 0.921 No 5 0.798 0.802 0.886 0.914 constrain 10 20 0.794 0.805 0.796 0.809 0.809 0.836 0.842 0.842 constrain 10 20 0.786 0.803 0.786 0.804 0.811 0.848 0.840 0.847 constrain 10 20 0.782 0.805 0.783 0.807 0.808 0.852 0.837 0.850

Table 3: Comparison with other memory-based approaches. A smaller value means a better performance.
Ratings Given (Test Item): Ratings Given (Test User): SF2 SF1 UBVS IBVS Ratings Given (Test Item): Ratings Given (Test User): SF2 SF1 UBVS IBVS Ratings Given (Test Item): Ratings Given (Test User): SF2 SF1 UBVS IBVS 5 1.054 1.086 1.129 1.190 20 1.070 1.097 1.117 1.131 5 0.995 1.035 1.052 1.108 20 0.997 1.024 1.054 1.068 5 0.945 0.976 0.996 1.066 20 0.923 0.936 0.969 0.977

(a) Number of Training Users: 100

5 0.960 0.976 1.108 1.187 20 0.948 0.963 1.024 1.034 5 0.915 0.927 1.070 1.122 20 0.885 0.895 0.972 0.976 5 0.826 0.832 0.914 0.974 20 0.828 0.831 0.885 0.886

(b) Number of Training Users: 200

5 0.956 1.013 1.024 1.117 20 0.941 0.977 1.044 1.024 5 0.911 0.928 0.966 1.044 20 0.912 0.938 0.980 1.004 5 0.842 0.847 0.921 0.962 20 0.859 0.867 0.936 0.932

(c) Number of Training Users: 300 using = 0.7 and = 0.7 for SF2. We rst compare our results to the standard user-based vector similarity (UBVS) approach of [1] and the item-based adjusted cosine similarity (IBVS) of [15]. We report results for test user sparsity 5, 10, or 20, and test item sparsity < 5, < 10, < 20 or No constrain. Table 3 summarizes the results, showing how SF1 and SF2 outperform the other methods in all twelve resulting congurations. Next, we adopt the subset of MovieLens (see [9, 19]), which consists of 500 users and 1000 items. We followed the exact evaluation procedure described in [19] to compare the performance of our SF2 scheme with the state-of-art results listed in [19]. Table 4 presents our experimental results, as well as the four best methods according to their experiments, i.e., cluster-based Pearson Correlation Coecient (SCBPCC) [19], the Aspect Model (AM) ([7]), Personality Diagnosis (PD) ([12]) and the user-based Pearson Correlation Coecient (PCC) ([1]). Our method outperforms these methods in all congurations. ground model to smooth the rating predictions. The experiments showed that our new fusion framework is eective in improving the prediction accuracy of collaborative ltering and dealing with the data sparsity problem. In the future, we plan to conduct better formal analyses of the fusion model and more complete comparisons with previous methods.

7.

REFERENCES

6.

CONCLUSIONS

We proposed a novel algorithm to unify the user-based and item-based collaborative ltering approaches to overcome limitations specic to either of them. We showed that user-based and item-based approaches are only two special cases in our probabilistic fusion framework. Furthermore, by using a linear interpolation smoothing, other ratings by similar users towards similar items can be treated as a back-

[1] J. S. Breese, D. Heckerman, and C. Kadie. Empirical analysis of predictive algorithms for collaborative ltering. In Proc. of UAI, 1998. [2] J. Canny. Collaborative ltering with privacy via factor analysis. In Proc. of SIGIR, 1999. [3] M. Deshpande and G. Karypis. Item-based top-n recommendation algorithms. ACM Trans. Inf. Syst., 22(1):143177, 2004. [4] K. Goldberg, T. Roeder, D. Gupta, and C. Perkins. Eigentaste: A constant time collaborative ltering algorithm. Information Retrieval Journal, 4(2):133151, July 2001. [5] J. L. Herlocker, J. A. Konstan, A. Borchers, and J. Riedl. An algorithmic framework for performing collaborative ltering. In Proc. of SIGIR, 1999. [6] D. Hiemstra. Term-specic smoothing for the language modeling approach to information retrieval: the importance of a query term. In Proc. of SIGIR, pages 3541, 2002.

Table 4: Comparison with the result reported in [19]. A smaller value means a better performance.
Num. of Training Users: Ratings Given (Test User): SF2 SCBPCC AM PD PCC 5 0.847 0.848 0.963 0.849 0.874 100 10 0.774 0.819 0.922 0.817 0.836 20 0.792 0.789 0.887 0.808 0.818 5 0.827 0.831 0.849 0.836 0.859 200 10 0.773 0.813 0.837 0.815 0.829 20 0.783 0.784 0.815 0.792 0.813 5 0.804 0.822 0.820 0.827 0.849 300 10 0.761 0.810 0.822 0.815 0.841 20 0.769 0.778 0.796 0.789 0.820

[7] T. Hofmann. Latent semantic models for collaborative ltering. ACM Trans. Info. Syst., Vol 22(1):89115, 2004. [8] Z. Huang, H. Chen, and D. Zeng. Applying associative retrieval techniques to alleviate the sparsity problem in collaborative ltering. ACM Trans. Inf. Syst., 22(1):116142, 2004. [9] R. Jin, J. Y. Chai, and L. Si. An automatic weighting scheme for collaborative ltering. In Proc. of SIGIR, 2004. [10] J. Kittler, M. Hatef, R. P. W. Duin, and J. Matas. On combining classiers. IEEE Trans. Pattern Anal. Mach. Intell., 20(3):226239, 1998. [11] G. Linden, B. Smith, and J. York. Amazon.com recommendations: Item-to-item collaborative ltering. IEEE Internet Computing, Jan/Feb.:7680, 2003. [12] D. M. Pennock, E. Horvitz, S. Lawrence, and C. Giles. Collaborative ltering by personality diagnosis: a hybrid memory and model based approach. In Proc. of UAI, 2000. [13] J. D. M. Rennie and N. Srebro. Fast maximum margin matrix factorization for collaborative prediction. In Proc. of ICML, 2005. [14] P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl. Grouplens: an open architecture for collaborative ltering of netnews. In Proc. of ACM CSCW, 1994. [15] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Item-based collaborative ltering recommendation algorithms. In Proc. of the WWW Conference, 2001. [16] B. M. Sarwar, G. Karypis, J. A. Konstan, and J. T. Riedl. Application of dimensionality reduction in recommender system a case study. In Proc. of ACM WebKDD Workshop, 2000. [17] L. Si and R. Jin. Flexible mixture model for collaborative ltering. In ICML, 2003. [18] J. Wang, A. P. de Vries, and M. J. Reinders. A user-item relevance model for log-based collaborative ltering. In Proc. of ECIR06, London, UK, 2006. [19] G.-R. Xue, C. Lin, Q. Yang, W. Xi, H.-J. Zeng, Y. Yu, and Z. Chen. Scalable collaborative ltering using cluster-based smoothing. In Proc. of SIGIR, 2005.

We normalize again by the average user rating: n(xa,b )I,U = n(xa,b )I = xa,b = xa,b 1 K 1 K 1 M n(xa,j )I
j

xi,b
i

1 M 1 M

xa,j
j

1 K

xi,j
i

xi,b
i

xa,j +
j

1 MK

xi,j
i,j

= xa,b x b x a + x where n(xa,b )I,U is the normalization of both item and user aspects. x a is the average rating from user a. x is the average of all the ratings. From here, we see that the result does not depend on the order of normalization (whether to normalize rst by user or by item). Treating each normalized individual rating as individual predictor results in: x k,m x m x k + x = xa,b x b x a + x pk,m (xa,b ) = x k,m = xa,b ( xa x k ) ( xb x m )

B.

A UNIFIED WEIGHTING FUNCTION

More specically, replacing three conditional probabilities with Eq. 15, the following can be derived from Eq. 13: xk,m
|r |

=
r =1

r
pk,m (xa,b )=r xa,b SUR

A+
xa,b SIR

B+
xa,b SUIR

=
xa,b :xa,b SUR

pk,m (xa,b )A +
xa,b :xa,b SIR

pk,m (xa,b )B +

pk,m (xa,b )C
xa,b :xa,b SUIR

where A= su (uk , ua ) (1 ) su (uk , ua )

xa,b :xa,b SUR

B=

si (im , ib ) (1 )(1 ) si (im , ib )

xa,b :xa,b SIR

APPENDIX A. NORMALIZATION
We rst normalize the matrix by subtracting the average item ratings: 1 n(xa,b )I = xa,b K xi,b = xa,b x b
i

C=

sui (xk,m , xa,b ) sui (xk,m , xa,b )

xa,b :xa,b SUIR

where A,B and C act as the weights to combine the predictors from three dierent sources. Unifying them we can obtain Eq. 16.

where n(xa,b )I normalizes ratings by subtracting the mean item rating. x b is the average rating of item b.