Cross-Domain Recommender Systems
Cross-Domain Recommender Systems
[email protected]
2nd ed. of the RSs Handbook
Cross-Domain Recommender Systems
2
Tutorial
Cross-Domain Recommender Systems
3
Today
Cross-Domain Recommender Systems
Paolo Cremonesi
4
Recommendations for single domains
5
User profiles in multiple systems
Nowadays, users…
• provide feedback for items of different types
• e.g., in Amazon we can rate books, DVDs, …
• express their opinions on different social media
and different providers
• e.g., Facebook, Twitter, Amazon, Netflix, TripAdvisor
definition of “domain”
definition of “better recommendations”
7
Problems related to Cross-Domain RSs
• Machine Learning
• Multi-Task Learning / Transfer Learning
• User Modeling
• aggregation user preferences for cross system
personalization, targeted adv., security
• Context Aware recommender
• different domains as different context
• Hybrid recommender (Ensemble learning)
• AdaBoost → Hybrid
• Bootstrap / Blending → Cross Domain
8
History of Cross-Domain RSs
• 2002: the term “cross-domain recommenders”
appear for the first time in a patent:
• Triplehop Technologies (now Oracle)
• 2005: some papers suggest “cross-domain” as an
interesting topic
• Mark van Setten, Sean M. McNee, Joseph A. Konstan.
2005
• Shlomo Berkovsky, Tsvi Kuflik, Francesco Ricci. 2005
• 2007: first papers with contributions on “cross-
domain”
• Ronald Chung, David Sundaram, Ananth Srinivasan. 2007
• Shlomo Berkovsky, Tsvi Kuflik, Francesco Ricci. 2007
• Ronald Chung, David Sundaram, Ananth Srinivasan. 2007
9
History of Cross-Domain RSs
• First papers trying to classify problems and
approaches
• Antonis Loizou. 2009
• Sinno Jialin Pan, and Qiang Yang. 2010
• Bin Li. 2011
• Paolo Cremonesi, Antonio Tripodi, and Roberto
Turrin. 2011
• Fernández-Tobías, Ignacio, Iván Cantador, Marius
Kaminskas, and Francesco Ricci. 2012
10
Cross-domain recommendations
• Single-Domain: Treat each domain
independently
• Collective-Domain: Merge domains an treat
them as a single domain baseline
12
Contents
13
Definition of the cross-domain problem
• Domains
• which types of domains exist?
• Goals
• why do we need cross-domain recommenders?
• Tasks
• which parts of the datasets are used?
• Scenarios
• which overlap of information exists between
domains?
14
Contents
16
Definition of domain
17
Definition of domain
source DS ↔ target DT
(auxiliary)
18
Domain levels
19
Domain levels in the literature…
20
Contents
Goal Percentage
Cold start 5%
New user 15%
New item 5%
Accuracy 55%
Diversity 5%
Privacy 5%
User model 10%
23
Cross-domain recommendation tasks
IS IT IS IT IS IT
S S S
UT
UT
UT
T US
T T
US
US
Multi-domain Linked-domain Cross-domain
S S S
UT
UT
UT
T US
T T
US
US
Multi-domain Linked-domain Cross-domain
• Recommend items in both source and target domains
• Goal: cross-selling, improve diversity, novelty,
serendipity
• Approach: sharing knowledge and linking domains
25
Linked-domain recommendation task
IS IT IS IT IS IT
S S S
UT
UT
UT
T US
T T
US
US
Multi-domain Linked-domain Cross-domain
• Recommend target items to users in the target domains
• Goal: improve accuracy of recommendations in the
target domain (e.g., reduce sparsity)
• Approach: all
26
Cross-domain recommendation task
IS IT IS IT IS IT
S S S
UT
UT
UT
T US
T T
US
US
Multi-domain Linked-domain Cross-domain
• Recommend items in the target domain to users in the
source domain
• Goal: solve cold-start, new users and new item probl.
• Approach: aggregating knowledge
27
Tasks in the literature …
Task Multi-domain
Multi-domain 20%
Linked-domain 55%
Cross-domain 25%
28
Contents
… users:
• FU(S) ∩ FU(T) ≠
• e.g., we have demographics of users in both domains
… items:
• FI(S) ∩ FI(T) ≠
• e.g., items share the same set of attributes in both
domains
31
Linking domains
Mapping attributes of …
… users:
• f : XU(S) → XU(T)
• e.g., friends of …
… items:
• f : XI(S) → XI(T)
• e.g., “vampire” in source and “zombie” in target are
both “horror”
32
Linking domains
Overlap of …
… items:
• I(S) ∩ I(T) ≠
• e.g., we have same common items between domains
… users:
• U(S) ∩ U(T) ≠
• e.g., we have same common users between domains
33
Cross-domain recommendation scenarios (I)
35
Contents
38
Cross-Domain: opportunity or problem?
is a matter of weights
39
Different approaches (I)
• Two types of cross-domain approaches, based on
how knowledge from the source domain is exploited
knowledge
linkage/transfer
Source Target Source Target
domain domain domain domain
knowledge
aggregation
+
target domain target domain
recommendations recommendations
Linking/Aggregating knowledge Sharing/Transferring knowledge
40
Different approaches (I)
• Two types of cross-domain approaches, based on
how knowledge from the source domain is exploited
knowledge
linkage/transfer
Source Target Source Target
domain domain domain domain
knowledge
aggregation
+
target domain target domain
recommendations recommendations
Linking/Aggregating knowledge Sharing/Transferring knowledge
Fuzzy !!! 41
Different approaches (II)
• Linking/Aggregating knowledge
• Merging user preferences
• Mediating user modeling data
• Combining recommendations
• Linking domains
• Sharing/Transferring knowledge
• Sharing latent features
• Transferring rating patterns
42
Contents
• Linking/aggregating knowledge
• Merging user preferences
• Mediating user modeling data
• Combining recommendations
• Linking domains
• Sharing/transferring knowledge
• Sharing latent features
• Transferring rating patterns
44
Merging user preferences (I)
• Aggregate user preferences
• ratings, tags, transaction logs, click-through data
IS IT
U DS DT
+ user preference
aggregation
active
user recsysST
target domain
recommendations
45
Merging user preferences (II)
• Pros:
• work well for the new-user problem
• robust (evolution of standard SD techniques)
• facilitate explanation
• Cons:
• need user-overlap between the source and target
domains
46
Merging user preferences: approaches
• On the aggregated matrix we can apply
“weighted” single-domain techniques
• User-based kNN
- Berkovsky et al. 2007; Shapira et al. 2013; Winoto & Tang
2008;
• Graph-based
- Nakatsuji et al. 2010; Cremonesi et al. 2011; Tiroshi et al.
2013
• Matrix Factorization / Factorization Machine
- Loni et al. 2014
47
Proposed categorization
• Linking/aggregating knowledge
• Merging user preferences
• Mediating user modeling data
• Combining recommendations
• Linking domains
• Sharing/transferring knowledge
• Sharing latent features
• Transferring rating patterns
48
Mediating user modeling data (I)
• Aggregate models (CF, CBF, Hybrid) from different
domains
• user similarities, user neighborhoods
IS IT
U DS DT
active
user
recsysS
recsysT
target domain
user neighborhood recommendations
(user similarities) 49
Mediating user modeling data (II)
• Pros:
• suited to the new-user problem and accuracy goals
• robust (evolution of standard SD techniques)
• Cons:
• need of either user- or item-overlap between the
source and target domains
50
Mediating modeling data: approaches
51
Proposed categorization
• Linking/Aggregating knowledge
• Merging user preferences
• Mediating user modeling data
• Combining recommendations
• Linking domains
• Sharing/transferring knowledge
• Sharing latent features
• Transferring rating patterns
52
Combining recommendations (I)
• Aggregate single-domain recommendations
• ratings, ranking, probability distributions
IS IT
U DS DT
active
user
recsysS
recsysT
target domain
user neighborhood recommendations
(user similarities)
53
Combining recommendations (II)
• Pros:
• easy to implement
• independent of the stand alone recommenders
• increase diversity
• independent of context
• Cons:
• need overlap of users
• difficult to tune weights assigned to
recommendations coming from different domains
54
Combining recommendations:
approaches
55
Proposed categorization
• Linking/aggregating knowledge
• Merging user preferences
• Mediating user modeling data
• Combining recommendations
• Linking domains
• Sharing/transferring knowledge
• Sharing latent features
• Transferring rating patterns
56
Linking domains (I)
• Linking domains by a common knowledge
• item attributes, user attributes, association rules,
semantic networks,
IS IT
U DS DT
recsysT
target domain
DS DT recommendations
multi-domain active
semantic network user
57
Linking domains (II)
• Pros:
• no need of user or item overlap
• bland with other technique
• Cons:
• difficult to generalize
• designed for particular cross-domain scenarios
58
Linking domains: approaches
• Overlap of user/item attributes
• Chung et al. 2007
• Overlap of social tags
• Szomszor et al. 2008; Abel et al. 2011; Abel et al. 2013;
Fernández-Tobias et al. 2013
• Overlap of text (BoW)
• Berkovsky et al. 2006
• Semantic networks
• Loizou 2009; Fernández-Tobias et al. 2011; Kaminskas et
al. 2013
• Knowledge-based rules
• Azak et al. 2010; Cantador et al. 2013
59
Contents
• Linking/aggregating knowledge
• Merging user preferences
• Mediating user modeling data
• Combining recommendations
• Linking domains
• Sharing/transferring knowledge
• Sharing latent features
• Transferring rating patterns
61
Sharing latent features (I)
• source and target domains are related by means
of shared latent features
IS IT
U DS DT
common
latent features
= x x B = x x B
DS A DT A
target domain
recommendations
active
user recsysT
62
Sharing latent features (II)
• Pros:
• work well to reduce sparsity and increase accuracy
for both source and target domains
• Cons:
• computationally expensive
• need overlap of users and/or items
63
Sharing latent features: approaches (I)
RS = U×BS×V’
RT = U×BT×V’
64
Sharing latent features: approaches (II)
S T → = × B × C
users
R A
items
65
Proposed categorization
• Linking/aggregating knowledge
• Merging user preferences
• Mediating user modeling data
• Combining recommendations
• Linking domains
• Sharing/transferring knowledge
• Sharing latent features
• Transferring rating patterns
66
Transferring rating patterns (I)
• rating patterns are transferred between domains
IS IT
U DS DT
rating
patterns
x RS x RS x RS x
DS A = B DT = A B
target domain
recommendations
active
user recsysT
67
Transferring rating patterns (II)
• Pros:
• apparently, no need of user or item overlap
• Cons:
• computationally expensive
68
Transferring rating patterns: approaches
RS = US×B×VS’
RT = UT×B×VT’
69
Transferring rating patterns: CBT
AUX
KNOWLEDGE 1. Extraction of knowledge
TRANSFER 1 EXTRACTION (codebook B) from
auxiliary domain
CODE
BOOK 2. Injection of knowledge in
0
AUX
KNOWLEDGE
EXTRACTION
TRANSFER 1
CODE
BOOK
0
TGT 2 INJECTION
3 RECOMMENDATION
71
Transferring rating patterns: CBT
AUX
KNOWLEDGE
EXTRACTION
TRANSFER 1
CODE Recommendation in
BOOK
0
target domain with
TGT 2 INJECTION user-based kNN
3 RECOMMENDATION
72
Transferring rating patterns: CBT
AUX: MovieLens
TGT: BookCrossing
kNN CBT
MAE 0,5216 0,5064
RMSE 0,4736 0,4492
73
Transferring rating patterns: CBT
AUX: MovieLens
TGT: BookCrossing
kNN CBT RND
MAE 0,5216 0,5064 0,4963
RMSE 0,4736 0,4492 0,4380
75
Data partitioning
Hold-out
Leave-some-users-out
76
Leave-all-users-out
Goal vs. partitioning technique
Leave-some- Leave-all-users-
Hold-out
users-out out
Cold start X
New user X
New item X
Accuracy X
Diversity X
77
Contents
78
Research issues (I)
79
Research issues (II)
• Cross-domain recommenders to reduce the user
model elicitation effort
• able to build detailed user profiles without the need
to collect explicit user preferences
[email protected]
References (I)
• Surveys
• Dong, G.: Cross Domain Similarity Mining: Research Issues and Potential
Applications Including Supporting Research by Analogy. ACM SIGKDD
Explorations Newsletter 14(1), pp. 43-47 (2012)
• Fernández-Tobías, I., Cantador, I., Kaminskas, M., Ricci, F.: Cross-domain
Recommender Systems: A Survey of the State of the Art. In: Proc. of the
2nd Spanish Conference on Information Retrieval, pp. 187-198 (2012)
• Li, B.: Cross-Domain Collaborative Filtering: A Brief Survey. In: Proc. of the
23rd IEEE International Conference on Tools with Artificial Intelligence, pp.
1085-1086 (2011)
• Pan, S. J., Yang, Q.: A Survey on Transfer Learning. IEEE Transactions on
Knowledge and Data Engineering 22(10), pp. 1345-1359 (2010)
82
References (II)
• Aggregating knowledge: merging user preferences
• Abel, F., Araújo, S., Gao, Q., Houben, G.-J.: Analyzing Cross-system User
Modeling on the Social Web. In: Proc. of the 11th International
Conference on Web Engineering, pp. 28-43 (2011)
• Cantador, I., Fernández-Tobías, I., Bellogín, A.: Relating Personality Types
with User Preferences in Multiple Entertainment Domains. In: Proc. of
the 1st Workshop on Emotions and Personality in Personalized Services,
CEUR workshop Proceedings, vol. 997 (2013)
• Cremonesi, P., Tripodi, A., Turrin, R.: Cross-domain Recommender
Systems. In: Proc. of the 11th IEEE International Conference on Data
Mining Workshops, pp. 496-503 (2011)
• Fernández-Tobías, I., Cantador, I., Plaza, L.: An Emotion Dimensional
Model Based on Social Tags: Crossing Folksonomies and Enhancing
Recommendations. In: Proc. of the 14th International Conference on E-
Commerce and Web Technologies, pp. 88-100 (2013)
• González, G., López, B., de la Rosa, J. LL.: A Multi-agent Smart User Model
for Cross-domain Recommender Systems. In: Proc. of Beyond
Personalization 2005 - The Next Stage of Recommender Systems Research,
pp. 93-94 (2005)
83
References (III)
84
References (IV)
• Aggregating knowledge: mediating user modeling data
• Berkovsky, S., Kuflik, T., Ricci, F.: Cross-Domain Mediation in Collaborative
Filtering. In: Proc. of the 11th International Conference on User Modeling,
pp. 355-359 (2007)
• Low, Y., Agarwal, D., Smola, A. J.: Multiple Domain User Personalization.
In: Proc. of the 17th ACM SIGKDD Conf. on Knowledge Discovery and Data
Mining, pp. 123-131 (2011)
• Pan, W., Xiang, E. W., Yang, Q.: Transfer Learning in Collaborative Filtering
with Uncertain Ratings. In: Proc. of the 26th AAAI Conference on Artificial
Intelligence, pp. 662-668 (2012)
• Shapira, B., Rokach, L., Freilikhman, S.: Facebook Single and Cross Domain
Data for Recommendation Systems. UMUAI 23(2-3), pp. 211-247 (2013)
• Stewart, A., Diaz-Aviles, E., Nejdl, W., Marinho, L. B., Nanopoulos, A.,
Schmidt-Thieme, L.: Cross-tagging for Personalized Open Social
Networking. In: Proc. of the 20th ACM Conference on Hypertext and
Hypermedia, pp. 271-278 (2009)
• Tiroshi, A., Kuflik, T.: Domain Ranking for Cross Domain Collaborative
Filtering. In: Proc. of the 20th International Conf. on User Modeling,
Adaptation, and Personalization, pp. 328-333 (2012) 85
References (V)
86
References (VI)
• Linking/transferring knowledge: linking domains
• Azak, M.: Crossing: A Framework to Develop Knowledge-based
Recommenders in Cross Domains. MSc thesis, Middle East Technical
University (2010)
• Berkovsky, S., Goldwasser, D., Kuflik, T., Ricci, F.: Identifying Inter-domain
Similarities through Content-based Analysis of Hierarchical Web-
Directories. In: Proc. of the 17th European Conference on Artificial
Intelligence, pp. 789-790 (2006)
• Cao, B., Liu, N. N., Yang, Q.: Transfer Learning for Collective Link
Prediction in Multiple Heterogeneous Domains. Proc. of 27th
International Conf. on Machine Learning, pp. 159-166 (2010)
• Chung, R., Sundaram, D., Srinivasan, A.: 2007. Integrated Personal
Recommender Systems. In: Proc. of the 9th International Conference on
Electronic Commerce, pp. 65-74 (2007)
87
References (VII)
• Linking/transferring knowledge: linking domains
• Fernández-Tobías, I., Cantador, I., Kaminskas, M., Ricci, F.: 2011. A Generic
Semantic-based Framework for Cross-domain Recommendation. In: Proc.
of the 2nd International Workshop on Information Heterogeneity and
Fusion in Recommender Systems, pp. 25-32 (2011)
• Loizou, A.: How to Recommend Music to Film Buffs: Enabling the
Provision of Recommendations from Multiple Domains. PhD thesis,
University of Southampton (2009)
• Shi, Y., Larson, M., Hanjalic, A.: Tags as Bridges between Domains:
Improving Recommendation with Tag-induced Cross-domain
Collaborative Filtering. In: Proc. of the 19th International Conference on
User Modeling, Adaption, and Personalization, pp. 305-316 (2011)
• Zhang, Y., Cao, B., Yeung, D.-Y.: Multi-Domain Collaborative Filtering. In:
Proc. of the 26th Conference on Uncertainty in Artificial Intelligence, pp.
725-732 (2010)
88
References (VIII)
• Linking/transferring knowledge: sharing latent features
• Enrich, M., Braunhofer, M., Ricci, F.: Cold-Start Management with Cross-
Domain Collaborative Filtering and Tags. In: Proc. of the 14th Intl.
Conference on E-Commerce and Web Technologies, pp. 101-112 (2013)
• Fernández-Tobías, I., Cantador, I.: Exploiting Social Tags in Matrix
Factorization Models for Cross-domain Collaborative Filtering. In: Proc. of
the 1st Intl. Workshop on New Trends in Content-based Recommender
Systems (2013)
• Hu, L., Cao, J., Xu, G., Cao, L., Gu, Z., Zhu, C.: Personalized
Recommendation via Cross-domain Triadic Factorization. In: Proc. of the
22nd International Conference on World Wide Web, pp. 595-606 (2013)
• Pan, W., Liu, N. N., Xiang, E. W., Yang, Q.: Transfer Learning to Predict
Missing Ratings via Heterogeneous User Feedbacks. In: Proc. of the 22nd
Intl. Joint Conference on Artificial Intelligence, pp. 2318-2323 (2011)
• Pan, W., Xiang, E. W., Liu, N. N., Yang, Q.: Transfer Learning in
Collaborative Filtering for Sparsity Reduction. Proc. of the 24th AAAI Conf.
on Artificial Intelligence, pp. 210-235 (2010)
89
References (IX)
90
References (X)
91