Market

Basket
Analysis
Anticipating Customer
Behavior

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Introduction
• Machine Learning is rewarding the retail industry in a unique way. It
supports the retail sector in all areas, from predicting sales success to
locating customers.
• Market basket analysis (MBA) is one such top retail
application of machine learning. It helps retailers know what
products people are purchasing together so that the store/website
layout can be designed in the same manner.
• It is mainly done by studying their previous purchasing activity.
Companies also leverage it in cross-selling their products on their
online platform.
• However, it is not only used in the retail sector; fraudulent insurance
claims and credit card transactions also make use of it.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Introduction
• Amazon is a great example that leverages this analysis to cross-sell
products.
• These are the products that come under the suggested item list
which might interest you along with your current purchase.
• Your browsing history, what other customers have bought with a
given product, and other factors determine which products appear in
the suggested category.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Introduction
• Some of us go to the grocery with a standard list; while some of us
have a hard time sticking to our grocery shopping list, no matter how
determined we are.
• No matter which type of person you are, retailers will always be
experts at making various temptations to inflate your budget.
• Remember the time when you had the “Ohh, I might need this as
well.” moment? Retailers boost their sales by relying on this one
simple intuition.
• People that buy this will most likely want to buy that as well.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Introduction
• People who buy bread will have a higher chance of buying butter
together, therefore an experienced assortment manager will
definitely know that having a discount on bread pushes the sales on
butter as well.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Data-driven strategies
• Huge retailers pivot on a detailed market basket analysis to uncover
associations between items. Using this valuable information, they are
able to carry out various strategies to improve their revenue:
• Associated products are placed close to each other, so that buyers
of one item would be prompted to buy the other.
• Discounts can be applied to only one of the associated products.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Data-driven strategies
• Imagine that you are a shop owner. These are the questions that you
may want to answer:
• “When arranging the shelves in the store, what should be placed
close to what?”
• “ For the online shopping application, what products will be
recommended for our customers?”
• “ Which products should be recommended together in the
promotion?”

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Association Rule Mining
• Data scientists are able to carry out Market Basket Analysis by
implementing Association Rule Mining.
• Association Rule Mining is a rule-based machine learning method
that helps to uncover meaningful correlations between different
products according to their co-occurrence in a data set.
• An association rule for market basket analysis should be performed
to extract insight from large transaction data to answer these
questions.
• The association rule can identify the relationship between items
using machine learning (or statistics) with items in the basket.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Association Rule Mining
• I will cover simple metrics in the association rule and the apriori
algorithm that will reduce the computational workload for the
association rule.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Example
• There are 5 items in our shop including bread, butter, jam, milk, and
egg. There are 5 transactions (baskets) in the database.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Example
• The computer will translate to a matrix like this..

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Example
• From the transaction dataset, we will calculate the metrics that can
guide us to business actions. There are 3 famous metrics covered :
1.support,
2.confidence,
3.lift,

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Metrics
• 1. Support
• How many transaction contain this itemset?
• Support gives an idea of how frequent an itemset is in all the
transactions.
• The support value helps us identify the rules worth considering for
further analysis.

• freq(X) is the number of occurrences of X in all transactions.

• N is the total number of transactions.
• Ex. Support of bread = 4/5 = 0.8

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Metrics
• 1. Support
• These tables show the support of all itemset with size 1–2. You can
try calculating along.
High support means
that most customers
buy this item, so this
item is important for
our shop.

We can filter out item

sets with low support
since the number of
occurrences is too low
to find any insight
(rule).

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Metrics
• 2. Confidence
• Considering only transaction containing item A. How many
transaction contain item B?
• Confidence is a conditional probability. It defines the probability of
the occurrence of the following item(s) in the same transaction given
some item(s) (antecedents) are already in that transaction.

• Ex.
Confidence(Bread →Milk) = 2/4 = 0.5
Confidence(Milk →Jam) = 1/3 = 0.33

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Metrics
• The below tables show the confidence of some rules.

High confidence
means that the
basket containing
A will likely contain
B also.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Metrics
• However, considering only the value of confidence is not enough to
make any business decision.
• If B (following itemset) is a very frequent itemset, confidence related
to B will always be high.
• Lift is another metric to be considered together with confidence.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Metrics
• 3. Lift
• With and without item A is in the transaction, mow much it affect
item B?
• Lift is the ratio of the probability of B occurrence given A is
present and the probability of B occurrence without knowing about
A.

• Ex.
Lift(Bread →Milk) = 0.5/0.6 = 0.83
Lift(Milk →Jam) = 0.33/0.2 = 1.65

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Metrics
• The below tables show the lift of some rules.

If Lift is higher than 1,

the presence of A in
this transaction
causes a higher
probability of B in the
same transaction.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm
• As an increment of the number of itemsets leads to an exponential
growth of the association rule, it is not efficient to calculate all rules.
• If we have 10 items, the number of rules is 57,000!!! Then, what
about a supermarket with more than 100 items? Calculating all rules
can be computationally expensive.
• The Apriori algorithm is designed to find only important association
rules with minimum computation.
• There are 2 steps in association rule mining
• Itemset Generation
• Rule Generation
• We will see how the Apriori algorithm helps us in these 2 steps

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Itemset
Generation
• 1. Itemset Generation
• We want only frequent itemsets.
• Frequent itemsets’ supports are higher than minimum support. To
make it simple, they are itemsets that occur frequently in all
transactions.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Itemset
Generation
• 1. Itemset Generation
• For example, if we have 4 items, the number of itemsets is 2⁴ -1 = 15.
if we have 10 items, the number of itemsets is 2¹⁰ -1 = 1023.
• Therefore, it is computationally expensive and time-consuming to
calculate support for all itemsets.
• We can remove many itemsets from support calculation by following
the Apriori principle.
• “Support-Based Pruning”
• It is intuitive because it is harder to find a transaction that has many
required items.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Itemset
Generation
• 1. Itemset Generation
• Example
• The store has 4 items including bread, butter, jam, and milk.
• We can create 15 itemsets as illustrated below.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Itemset
Generation
• 1. Itemset Generation
• Example
• Assume we set acceptable minimum support = 0.3.
• If the support of bread is less than 0.3, all supersets of bread will also
have support of less than 0.3.
• Then, we can exclude all supersets of bread from the support
calculation. We call this “Support-Based Pruning”

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Itemset
Generation
• 1. Itemset Generation
• Example
• Then, we can exclude all supersets of bread from the support
calculation. We call this “Support-Based Pruning”

The computation
can be greatly
reduced by as much
as half.

After we have the frequent

itemsets, we can now create an
association rule.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm
• 2. Rule Generation
• We want only rules with high confidence.
• A way to identify important rules is to filter only rules with high
confidence since it means that the correlation between items A and
B is high.
• However, it is also not efficient to calculate the confidence of all rules
(the number of rules is even higher than the number of itemsets).
• We can remove many rules from confidence calculation by following
the Apriori principle.
• “Confidence-Based Pruning”

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm
• 2. Rule Generation
• For example, if we have 4 items in the itemset: A, B, C, and D.
the confidence of (A,B,C→ D) ≥ (A,B → C,D) ≥ (A → B,C,D).
• To prove this, I will recall the formula of confidence.

• The difference between these 2 rules is only the denominator.

As we talk about support in step 1, more items will cause lower
support (lower frequency), so the first rule has a lower denominator
and higher confidence.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm
• 2. Rule Generation
• We can perform “Confidence-Based Pruning” as we do in step1. We
will start with a rule that has only 1 consequence.
• It may be confusing at first. My tip is to focus only on the
consequence (item(s) behind an arrow) and do it as we used to do
with support.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm
• 2. Rule Generation
• Ex.
• The store has 4 items including bread, butter, jam, and milk.
• We can create 14 rules as illustrated below.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm
• 2. Rule Generation
• Assume we set acceptable minimum confidence = 0.3.
• If the confidence of {butter, jam, milk} → {bread} is less than 0.3, all
rules that the consequence is a superset of bread will also have the
confidence of less than 0.3.
• Then, we can exclude all rules that the consequence is a superset of
bread from the confidence calculation.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm
• 2. Rule Generation
• Assume we set acceptable minimum confidence = 0.3.
• If the confidence of {butter, jam, milk} → {bread} is less than 0.3, all
rules that the consequence is a superset of bread will also have the
confidence of less than 0.3.
• Then, we can exclude all rules that the consequence is a superset of
bread from the confidence calculation.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm
• After we finish 1 itemset, we move to other frequent itemsets.
• Finally, we will calculate the lift from the remaining rules and make
business decisions with that data!

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• For the given dataset apply Apriori algorithm to discover strong
association rules among the items bought.
• Assume that min_support = 40% and min_confidence = 70%
• Generate association rules from the frequent item sets. Calculate
the confidence of each rule and identify all the strong association
rules.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• Step # 01: Generate frequent itemsets step by step until the largest
frequent itemset is generated
• Unique Products : Bread, Butter, Milk, Cookies, Beer, Diapers

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise

min_support = 40%
min_support_count = min_suppot * itemset_count
min_support_count = 40% * 5
min_support_count = 2

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• Therefore, except cookies all are 1 frequent itemset

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• Now, we have 5 unique items: Bread, Butter, Diapers, Milk, Beer
• We have to make combinations of 2 item set by considering the new
unique items

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• We neglect the combination whose support is less than given
min_support i.e. 2.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• Now, we need to generate the 3 frequent itemset considering the
items got selected by 2 frequent itemset
• Unique Item: Bread, Butter, Milk, Diapers, Beer

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• Now, we need to generate the 3 frequent itemset considering the
items got selected by 2 frequent itemset
• Unique Item: Bread, Butter, Milk, Diapers, Beer

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• Now, we need to generate the 3 frequent itemset considering the
items got selected by 2 frequent itemset
• Unique Item: Bread, Butter, Milk, Diapers, Beer

We stop here, because no 4-itemset

can be generated further

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• Next step is to write association rules and select only those rules
which are strong based on the confidence score.

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• min_confidence = 70%
• Confidence:

• We have 5 frequent item set

• Therefore, candidate rules are:

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• min_confidence = 70%
• Confidence:

• We have 5 frequent item set

• Therefore, candidate rules are:

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• min_confidence = 70%
• Confidence:

• We have 5 frequent item set

• Therefore, candidate rules are:

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• min_confidence = 70%
• Confidence:

• We have 5 frequent item set

• Therefore, candidate rules are:

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• min_confidence = 70%
• Confidence:

• We have 5 frequent item set

• Therefore, candidate rules are:

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise
• min_confidence = 70%
• Confidence:

• We have 5 frequent item set

• Therefore, candidate rules are:

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Thank You

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.
 Apriori algorithm : Exercise

© 2016 Cengage Learning. All Rights Reserved. May not be copied, scanned, or duplicated, in whole or in part, except for use as permitted in a license distributed with a certain product or service or otherwise on a password-protected website for classroom use.