APRIORI

ALGORITHM
 Motivation: Association Rule Mining
• Given a set of transactions, find rules that will predict the
occurrence of an item based on the occurrences of other
items in the transaction

Market-Basket transactions
Example of Association Rules
TID Items
{Diaper}  {Beer},
1 Bread, Milk {Milk, Bread}  {Eggs,Coke},
2 Bread, Diaper, Beer, Eggs {Beer, Bread}  {Milk},
3 Milk, Diaper, Beer, Coke
4 Bread, Milk, Diaper, Beer
5 Bread, Milk, Diaper, Coke
 Applications: Association Rule Mining

• *  Maintenance Agreement
– What the store should do to boost Maintenance
Agreement sales
• Home Electronics  *
– What other products should the store stocks up?
• Attached mailing in direct marketing
• Detecting “ping-ponging” of patients
• Marketing and Sales Promotion
• Supermarket shelf management
 Definition: Frequent Itemset
• Itemset
– A collection of one or more items
•Example: {Milk, Bread, Diaper}
– k-itemset
•An itemset that contains k items TID Items
• Support count () 1 Bread, Milk
– Frequency of occurrence of an itemset 2 Bread, Diaper, Beer, Eggs
– E.g. ({Milk, Bread,Diaper}) = 2 3 Milk, Diaper, Beer, Coke
• Support 4 Bread, Milk, Diaper, Beer
– Fraction of transactions that contain an 5 Bread, Milk, Diaper, Coke
itemset
– E.g. s({Milk, Bread, Diaper}) = 2/5
• Frequent Itemset
– An itemset whose support is greater
than or equal to a minsup threshold
 Definition: Association Rule
• Association Rule TID Items

– An implication expression of the form 1 Bread, Milk

X  Y, where X and Y are itemsets 2 Bread, Diaper, Beer, Eggs
– Example: 3 Milk, Diaper, Beer, Coke
{Milk, Diaper}  {Beer} 4 Bread, Milk, Diaper, Beer
5 Bread, Milk, Diaper, Coke
• Rule Evaluation Metrics
– Support (s) Example:
•Fraction of transactions that contain both {
Milk 
,Diaper
} Be


X and Y
– Confidence (c) (
Milk
,
DiapBee
)2
•Measures how often items in Y 
s 
0
.4
|T| 5

appear in transactions that
contain X


(Milk,
DiapBee
)2

c 0
.
6
(Milk
,Diap
) 3
 Association Rule Mining Task

• Given a set of transactions T, the goal of

association rule mining is to find all rules having
– support ≥ minsup threshold
– confidence ≥ minconf threshold

• Brute-force approach:
– List all possible association rules
– Compute the support and confidence for each rule
– Prune rules that fail the minsup and minconf
thresholds
 Computationally prohibitive!
 Computational Complexity
• Given d unique items:
– Total number of itemsets = 2d
– Total number of possible association rules:

d dk
k  
d
1 d
k
R 
   j 
k
1 j
1

3 2d
1 d
1

If d=6, R = 602 rules

 Mining Association Rules: Decoupling
TID Items Example of Rules:
1 Bread, Milk
2 Bread, Diaper, Beer, Eggs
{Milk,Diaper}  {Beer} (s=0.4, c=0.67)
{Milk,Beer}  {Diaper} (s=0.4, c=1.0)
3 Milk, Diaper, Beer, Coke
{Diaper,Beer}  {Milk} (s=0.4, c=0.67)
4 Bread, Milk, Diaper, Beer
{Beer}  {Milk,Diaper} (s=0.4, c=0.67)
5 Bread, Milk, Diaper, Coke {Diaper}  {Milk,Beer} (s=0.4, c=0.5)
Observations: {Milk}  {Diaper,Beer} (s=0.4, c=0.5)
• All the above rules are binary partitions of the same itemset:
{Milk, Diaper, Beer}
• Rules originating from the same itemset have identical support but
can have different confidence
• Thus, we may decouple the support and confidence requirements
 Mining Association Rules

• Two-step approach:
1. Frequent Itemset Generation
– Generate all itemsets whose support  minsup

2. Rule Generation
– Generate high confidence rules from each frequent itemset,
where each rule is a binary partitioning of a frequent itemset

• Frequent itemset generation is still

computationally expensive
 Frequent Itemset Generation
• Brute-force approach:
– Each itemset in the lattice is a candidate frequent itemset
– Count the support of each candidate by scanning the
database
Transactions List of
Candidates
TID Items
1 Bread, Milk
2 Bread, Diaper, Beer, Eggs
N 3 Milk, Diaper, Beer, Coke M
4 Bread, Milk, Diaper, Beer
5 Bread, Milk, Diaper, Coke
w

– Match each transaction against every candidate

– Complexity ~ O(NMw) => Expensive since M = 2d !!!
Reducing Number of Candidates: Apriori
• Apriori principle:
– If an itemset is frequent, then all of its subsets must also
be frequent

• Apriori principle holds due to the following property

of the support measure:

X,
Y :
(X
Y
) s
(X
) s
(
Y)
– Support of an itemset never exceeds the support of its
subsets
– This is known as the anti-monotone property of support
 Illustrating Apriori Principle
null

A B C D E

AB AC AD AE BC BD BE CD CE DE

Found to be
Infrequent
ABC ABD ABE ACD ACE ADE BCD BCE BDE CDE

ABCD ABCE ABDE ACDE BCDE

Pruned
supersets
ABCDE
 Illustrating Apriori Principle

Item Count Items (1-itemsets)

Bread 4
Coke 2
Milk 4 Item set Count Pairs (2-itemsets)
Beer 3 {Bread,M ilk} 3
Diaper 4 {Bread,Beer} 2 (No need to generate
Eggs 1
{Bread,Diaper} 3 candidates involving Coke
{M ilk,Beer} 2 or Eggs)
{M ilk,Diaper} 3
{Beer,Diaper} 3
Minimum Support = 3
Triplets (3-itemsets)

If every subset is considered, Item set Count

6C + 6C + 6C = 41 {B read,Milk,Diaper} 3
1 2 3
With support-based pruning,
6 + 6 + 1 = 13
Apriori: Reducing Number of Comparisons
• Candidate counting:
– Scan the database of transactions to determine the support of
each candidate itemset
– To reduce the number of comparisons, store the candidates in a
hash structure
• Instead of matching each transaction against every candidate, match
it against candidates contained in the hashed buckets

Transactions Hash Structure

TID Items
1 Bread, Milk
2 Bread, Diaper, Beer, Eggs
N 3 Milk, Diaper, Beer, Coke k
4 Bread, Milk, Diaper, Beer
5 Bread, Milk, Diaper, Coke
Buckets
Apriori: Implementation Using Hash Tree
Suppose you have 15 candidate itemsets of length 3:
{1 4 5}, {1 2 4}, {4 5 7}, {1 2 5}, {4 5 8}, {1 5 9}, {1 3 6}, {2 3 4}, {5 6 7}, {3 4 5}, {3
5 6}, {3 5 7}, {6 8 9}, {3 6 7}, {3 6 8}
You need:
• Hash function
• Max leaf size: max number of itemsets stored in a leaf node
(if number of candidate itemsets exceeds max leaf size, split the node)

Hash function 234

3,6,9 567
1,4,7
145 136
2,5,8 345 356 367
357 368
124 159 689
125
457 458
 Apriori: Implementation Using Hash Tree
1 2 3 5 6 transaction

1+ 2356
2+ 356
12+ 356
3+ 56
13+ 56
234
15+ 6 567

145 136
345 356 367
357 368
124 159 689
125
457 458
Match transaction against 11 out of 15 candidates
REFERENCES :

• Fast algorithms for mining association rules in large databases