CSE@HCST 1

ASSOCIATION RULE MINING

9/5/2019 UNIT-II PART-2

 Mining Association Rules in Large Databases

 Association rule mining.

 Mining single-dimensional Boolean association rules from

transactional databases.

 Mining multilevel association rules from transactional

databases.

 Mining multidimensional association rules from transactional

databases and data warehouse.

 From association mining to correlation analysis.

 Constraint-based association mining.

CSE@HCST 9/5/2019
 What Is Association Mining?
3

 Association rule mining

 Finding frequent patterns, associations, correlations, or causal

structures among sets of items or objects in transaction
databases, relational databases, and other information
repositories.

 Applications

 Basket data analysis, cross-marketing, catalog design, loss-

leader analysis, clustering, classification, etc.
CSE@HCST 9/5/2019
 Association Mining
4

 Ruleform-
Prediction (Boolean variables) =>Prediction
(Boolean variables) [support, confidence]
 Computer => antivirus _software
[support =2%, confidence = 60%]
 buys(x, “computer”)  buys (x,
“antivirus_software”) [S=0.5%, C=60%]

CSE@HCST 9/5/2019
 Association Rule: Basic Concepts
5

 Given a database of transactions each transaction is a

list of items (purchased by a customer in a visit).
 Find all rules that correlate the presence of one set of
items with that of another set of items.
 Find frequent patterns.
 Example for frequent item-set mining is market basket
analysis.

CSE@HCST 9/5/2019
 Association rule performance measures
6

 Confidence
 Support
 Minimum support threshold
 Minimum confidence threshold

CSE@HCST 9/5/2019
 Rule Measures: Support and
7
Confidence
Customer
buys both
Customer  Find all the rules X & Y  Z with minimum
buys napkin confidence and support
 support, s, probability that a transaction
contains {X  Y  Z}
 confidence, c, conditional probability that a
transaction having {X  Y} also contains Z
Customer
buys beer

Let minimum support 50%, and

Transaction ID Items Bought minimum confidence 50%, we have
2000 A,B,C  A  C (50%, 66.6%)
1000 A,C  C  A (50%, 100%)
4000 A,D
5000 B,E,F
CSE@HCST 9/5/2019
 Martket Basket Analysis
8

 Shopping baskets
 Each item has a Boolean variable representing the
presence or absence of that item.
 Each basket can be represented by a Boolean vector
of values assigned to these variables.
 Identify patterns from Boolean vector.
 Patterns can be represented by association rules.

CSE@HCST 9/5/2019
 Association Rule Mining: A Road Map
9

 Boolean vs. quantitative associations

- Based on the types of values handled
 buys(x, “SQLServer”) ^ buys(x, “DMBook”) =>buys(x,
“DBMiner”) [0.2%, 60%]
 age(x, “30..39”) ^ income(x, “42..48K”) =>buys(x, “PC”) [1%,
75%]
 Single dimension vs. multiple dimensional associations
 Single level vs. multiple-level analysis

CSE@HCST 9/5/2019
10

Mining single-dimensional Boolean

association rules from transactional
databases

CSE@HCST 9/5/2019
 Apriori Algorithm
11

 Single dimensional, single-level, Boolean frequent

item sets.
 Finding frequent item sets using candidate
generation.
 Generating association rules from frequent item
sets.

CSE@HCST 9/5/2019
 Mining Association Rules—An Example
12

Transaction ID Items Bought Min. support 50%

2000 A,B,C Min. confidence 50%
1000 A,C
4000 A,D Frequent Itemset Support
{A} 75%
5000 B,E,F
{B} 50%
{C} 50%
For rule A  C: {A,C} 50%
support = support({A C}) = 50%
confidence = support({A C})/support({A}) = 66.6%
The Apriori principle:
Any subset of a frequent itemset must be frequent

CSE@HCST 9/5/2019
 Mining Frequent Itemsets: the Key Step
13

 Find the frequent itemsets: the sets of items that have

minimum support
 A subset of a frequent itemset must also be a frequent itemset
 i.e., if {AB} is a frequent itemset, both {A} and {B} should be a
frequent itemset
 Iteratively find frequent itemsets with cardinality from 1 to k
(k-itemset)
 Use the frequent itemsets to generate association rules.

CSE@HCST 9/5/2019
 The Apriori Algorithm
14

 Join Step
 Ck is generated by joining Lk-1with itself

 Prune Step
 Any (k-1)-itemset that is not frequent cannot be a subset
of a frequent k-itemset

CSE@HCST 9/5/2019
 The Apriori Algorithm
15

 Pseudo-code:
Ck: Candidate itemset of size k
Lk : frequent itemset of size k
L1 = {frequent items};
for (k = 1; Lk !=; k++) do begin
Ck+1 = candidates generated from Lk;
for each transaction t in database do
increment the count of all candidates in Ck+1 that are
contained in t
Lk+1 = candidates in Ck+1 with min_support
end
return k Lk;

CSE@HCST 9/5/2019
 The Apriori Algorithm — Example
Database D itemset sup.
16 L1 itemset sup.
TID Items C1 {1} 2 {1} 2
100 134 {2} 3 {2} 3
200 235 Scan D {3} 3 {3} 3
300 1235 {4} 1 {5} 3
400 25 {5} 3
C2 itemset sup C2 itemset
L2 itemset sup {1 2} 1 Scan D {1 2}
{1 3} 2 {1 3} 2 {1 3}
{2 3} 2 {1 5} 1 {1 5}
{2 3} 2 {2 3}
{2 5} 3
{2 5} 3 {2 5}
{3 5} 2
{3 5} 2 {3 5}
C3 itemset Scan D L3 itemset sup
{2 3 5} {2 3 5} 2
CSE@HCST 9/5/2019
 How to Generate Candidates?
17

 Suppose the items in Lk-1 are listed in an order

 Step 1: self-joining Lk-1
insert into Ck
select p.item1, p.item2, …, p.itemk-1, q.itemk-1
from Lk-1 p, Lk-1 q
where p.item1=q.item1, …, p.itemk-2=q.itemk-2, p.itemk-1 < q.itemk-1

 Step 2: pruning
forall itemsets c in Ck do
forall (k-1)-subsets s of c do
if (s is not in Lk-1) then delete c from Ck
CSE@HCST 9/5/2019
 How to Count Supports of Candidates?
18

 Why counting supports of candidates a problem?

 The total number of candidates can be very huge

 One transaction may contain many candidates

 Method
 Candidate itemsets are stored in a hash-tree

 Leaf node of hash-tree contains a list of itemsets and

counts
 Interior node contains a hash table

 Subset function: finds all the candidates contained in a

transaction

CSE@HCST 9/5/2019
 Example of Generating Candidates
19

 L3={abc, abd, acd, ace, bcd}

 Self-joining: L3*L3
 abcd from abc and abd
 acde from acd and ace

 Pruning:
 acde is removed because ade is not in L3

 C4={abcd}

CSE@HCST 9/5/2019
 Methods to Improve Apriori’s Efficiency
20

 Hash-based itemset counting

 A k-itemset whose corresponding hashing bucket count is below the
threshold cannot be frequent.

CSE@HCST 9/5/2019
 Methods to Improve Apriori’s Efficiency
21

 Transaction reduction
 A transaction that does not contain any frequent k-itemset is useless in
subsequent scans

 Partitioning
 Any itemset that is potentially frequent in DB must be frequent in at least one
of the partitions of DB

CSE@HCST 9/5/2019
 Methods to Improve Apriori’s Efficiency
22

 Sampling
 mining on a subset of given data, lower support threshold +
a method to determine the completeness.

CSE@HCST 9/5/2019
 Mining Frequent Patterns Without Candidate
Generation[Not in the syllabus]
23

 Compress a large database into a compact, Frequent-Pattern

tree (FP-tree) structure
 highly condensed, but complete for frequent pattern mining
 avoid costly database scans
 Develop an efficient, FP-tree-based frequent pattern mining
method
 A divide-and-conquer methodology: decompose mining tasks into smaller
ones
 Avoid candidate generation: sub-database test only

CSE@HCST 9/5/2019
24

Mining multilevel association rules

from transactional databases

CSE@HCST 9/5/2019
 Mining various kinds of association rules
25

 Mining Multilevel association rules

 Concepts at different levels

 Mining Multidimensional association rules

 More than one dimensional
 Mining Quantitative association rules
 Numeric attributes

CSE@HCST 9/5/2019
26 CSE@HCST 9/5/2019
 Multiple-Level Association Rules
27

Food
 Items often form hierarchy.
 Items at the lower level are milk bread
expected to have lower support.
 Rules regarding itemsets at skim 2% wheat white
appropriate levels could be quite
useful. Fraser Sunset
 Transaction database can be
encoded based on dimensions
and levels.
 We can explore shared multi-
level mining.
CSE@HCST 9/5/2019
28 CSE@HCST 9/5/2019
 Multi-level Association
29

 Uniform Support- the same minimum support for all

levels
+ One minimum support threshold. No need to examine
itemsets containing any item whose ancestors do not have
minimum support.
 – Lower level items do not occur as frequently. If support
threshold
 too high  miss low level associations
 too low  generate too many high level
associations

CSE@HCST 9/5/2019
 Multi-level Association
30

 Reduced Support- reduced minimum support at

lower levels
 There are 4 search strategies:
 Level-by-level independent
 Level-cross filtering by k-itemset
 Level-cross filtering by single item
 Controlled level-cross filtering by single item

CSE@HCST 9/5/2019
31 CSE@HCST 9/5/2019
 Uniform Support
32

Multi-level mining with uniform support

Level 1 Milk
min_sup = 5%
[support = 10%]

Level 2 2% Milk Skim Milk

min_sup = 5% [support = 6%] [support = 4%]

CSE@HCST 9/5/2019
Back
 Reduced Support
33

Multi-level mining with reduced support

Level 1 Milk
min_sup = 5%
[support = 10%]

Level 2 2% Milk Skim Milk

min_sup = 3% [support = 6%] [support = 4%]

CSE@HCST 9/5/2019
34 CSE@HCST 9/5/2019
 Multi-level Association: Redundancy Filtering
35

 Some rules may be redundant due to “ancestor”

relationships between items.
 Example-

 We say the first rule is an ancestor of the second rule.

 A rule is redundant if its support is close to the
“expected” value, based on the rule’s ancestor.

CSE@HCST 9/5/2019
36

Mining multidimensional association

rules from transactional databases
and data warehouse

CSE@HCST 9/5/2019
 Multi-Dimensional Association
37

 Single-dimensional rules
 Intra-dimension association rules
buys(X, “milk”) buys(X, “bread”)

 Multi-dimensional rules
 Inter-dimension association rules -no repeated predicates
age(X,”19-25”)  occupation(X,“student”) buys(X,“coke”)
 hybrid-dimension association rules -repeated predicates
age(X,”19-25”)  buys(X, “popcorn”)  buys(X, “coke”)

CSE@HCST 9/5/2019
 Multi-Dimensional Association
38

 Categorical Attributes
 finite
number of possible values, no ordering among
values
 Quantitative Attributes
 numeric, implicit ordering among values

CSE@HCST 9/5/2019
 Techniques for Mining MD Associations
39

 Search for frequent k-predicate set:

 Example: {age, occupation, buys} is a 3-predicate set.
 Techniques can be categorized by how age are treated.
1. Using static discretization of quantitative attributes
 Quantitative attributes are statically discretized by using
predefined concept hierarchies.
2. Quantitative association rules
 Quantitative attributes are dynamically discretized into
“bins” based on the distribution of the data.
3. Distance-based association rules
 This is a dynamic discretization process that considers the
distance between data points.

CSE@HCST 9/5/2019
 Static Discretization of Quantitative Attributes
40

 Discretized prior to mining using concept hierarchy.

 Numeric values are replaced by ranges.
 In relational database, finding all frequent k-predicate sets
will require k or k+1 table scans.
 Data cube is well suited for mining.
 The cells of an n-dimensional cuboid correspond to
the predicate sets.
 Mining from data cube scan be much faster.

CSE@HCST 9/5/2019
41 CSE@HCST 9/5/2019
 Quantitative Association Rules
42

 Numeric attributes are dynamically discretized

 Such that the confidence or compactness of the rules
mined is maximized.
 2-D quantitative association rules:
Aquan1  Aquan2  Acat
 Cluster “adjacent” association rules to form general
rules using a 2-D grid.

CSE@HCST 9/5/2019
 Example:

43 CSE@HCST 9/5/2019
44 CSE@HCST 9/5/2019
45

From association mining to

correlation analysis

CSE@HCST 9/5/2019
 Interestingness Measurements
46

 Objective measures-
 Two popular measurements
 support
 confidence

 Subjective measures-
A rule (pattern) is interesting if
*it is unexpected (surprising to the user); and/or
*actionable (the user can do something with it)

CSE@HCST 9/5/2019
 Criticism to Support and Confidence
 Example
47

 Among 5000 students

 3000 play basketball
 3750 eat cereal
 2000 both play basket ball and eat cereal

 play basketball  eat cereal [40%, 66.7%] is misleading because

the overall percentage of students eating cereal is 75% which is higher
than 66.7%.
 play basketball  not eat cereal [20%, 33.3%] is far more
accurate, although with lower support and confidence

basketball not basketball sum(row)

cereal 2000 1750 3750
not cereal 1000 250 1250
sum(col.) 3000 2000 5000
CSE@HCST 9/5/2019
 Criticism to Support and Confidence

48

 Example
 X and Y: positively correlated,
 X and Z, negatively related
X 1 1 1 1 0 0 0 0
 support and confidence of Y 1 1 0 0 0 0 0 0
X=>Z dominates
 We need a measure of dependent or
Z 0 1 1 1 1 1 1 1
correlated events
P( A B)
corrA, B =
P( A) P( B)
 Rule
Support Confidence
P(B|A)/P(B) is also called the lift of rule A
=> B X=>Y
25% 50%
X=>Z 37.50% 75%
CSE@HCST 9/5/2019
 Other Interestingness Measures: Interest
49

 Interest (correlation, lift) P( A  B)

P( A) P( B)
 taking both P(A) and P(B) in consideration

 P(A^B)=P(B)*P(A), if A and B are independent events

 A and B negatively correlated, if the value is less than 1;
otherwise A and B positively correlated

X 1 1 1 1 0 0 0 0 Itemset Support Interest

X,Y 25% 2
Y 1 1 0 0 0 0 0 0
X,Z 37.50% 0.9
Z 0 1 1 1 1 1 1 1 Y,Z 12.50% 0.57
CSE@HCST 9/5/2019
 END

50 CSE@HCST 9/5/2019