The Apriori Algorithm: Basics

The Apriori Algorithm is an influential algorithm for

mining frequent itemsets for boolean association rules.

Key Concepts :
• Frequent Itemsets: The sets of item which has minimum
support (denoted by Li for ith-Itemset).
• Apriori Property: Any subset of frequent itemset must be
frequent.
• Join Operation: To find Lk , a set of candidate k-itemsets is
generated by joining Lk-1 with itself.
 The Apriori Algorithm in a
Nutshell
• Find the frequent itemsets: the sets of items that have
minimum support
– A subset of a frequent itemset must also be a
frequent itemset
• i.e., if {AB} is a frequent itemset, both {A} and {B}
should be a frequent itemset
– Iteratively find frequent itemsets with cardinality
from 1 to k (k-itemset)
• Use the frequent itemsets to generate association rules.
 The Apriori Algorithm : Pseudo
code
• Join Step: Ck is generated by joining Lk-1with itself
• Prune Step: Any (k-1)-itemset that is not frequent cannot be a
subset of a frequent k-itemset
• Pseudo-code:
Ck: Candidate itemset of size k
Lk : frequent itemset of size k
L1 = {frequent items};
for (k = 1; Lk !=; k++) do begin
Ck+1 = candidates generated from Lk;
for each transaction t in database do
increment the count of all candidates in Ck+1
that are contained in t
= candidates in Ck+1 with min_support
L
end
k+1
return k Lk;
The Apriori Algorithm: Example

TID List of Items

• Consider a database, D ,
T100 I1, I2, I5 consisting of 9 transactions.
• Suppose min. support count
T100 I2, I4
required is 2 (i.e. min_sup = 2/9 =
T100 I2, I3 22 % )
T100 I1, I2, I4 • Let minimum confidence required
is 70%.
T100 I1, I3
• We have to first find out the
T100 I2, I3
frequent itemset using Apriori
T100 I1, I3 algorithm.
T100 I1, I2 ,I3, I5 • Then, Association rules will be
generated using min. support &
T100 I1, I2, I3
min. confidence.
 Step 1: Generating 1-itemset Frequent Pattern

Itemset Sup.Count Itemset Sup.Count

Compare candidate
Scan D for support count with
count of each
{I1} 6 {I1} 6
minimum support
candidate {I2} 7 count {I2} 7
{I3} 6 {I3} 6
{I4} 2 {I4} 2
{I5} 2 {I5} 2

C1

L1

•The set of frequent 1-itemsets, L1 , consists of the candidate 1-

itemsets satisfying minimum support.
•In the first iteration of the algorithm, each item is a member of the set of
candidate.
 Step 2: Generating 2-itemset Frequent Pattern

Itemset Itemset Sup. Itemset Sup

Generate Compare
C2 {I1, I2} Scan D for Count candidate Count
candidates count of {I1, I2} 4 support {I1, I2} 4
from L1
{I1, I3} each count with
{I1, I4} candidate {I1, I3} 4 minimum {I1, I3} 4
support {I1, I5} 2
{I1, I5} {I1, I4} 1 count
{I2, I3} {I1, I5} 2 {I2, I3} 4

{I2, I4} {I2, I4} 2

{I2, I3} 4
{I2, I5} {I2, I5} 2
{I2, I4} 2
{I3, I4} {I2, I5} 2 L2
{I3, I5}
{I3, I4} 0
{I4, I5}
{I3, I5} 1

C2 {I4, I5} 0

C2
Step 2: Generating 2-itemset Frequent Pattern

• To discover the set of frequent 2-itemsets, L2 , the algorithm

uses L1 Join L1 to generate a candidate set of 2-itemsets, C2.
• Next, the transactions in D are scanned and the support
count for each candidate itemset in C2 is accumulated (as
shown in the middle table).
• The set of frequent 2-itemsets, L2 , is then determined,
consisting of those candidate 2-itemsets in C2 having
minimum support.
• Note: We haven’t used Apriori Property yet.
 Step 3: Generating 3-itemset Frequent Pattern

Compare
Scan D for Scan D for Itemset Sup. candidate Itemset Sup
count of Itemset count of support
Count count with
Count
each each
{I1, I2, I3} {I1, I2, I3} 2 min support {I1, I2, I3} 2
candidate candidate
{I1, I2, I5} count
{I1, I2, I5} 2 {I1, I2, I5} 2

C3 C3 L3

•The generation of the set of candidate 3-itemsets, C3 , involves use of the

Apriori Property.
• In order to find C3, we compute L2 Join L2.
•C3 = L2 Join L2 = {{I1, I2, I3}, {I1, I2, I5}, {I1, I3, I5}, {I2, I3, I4}, {I2, I3,
I5}, {I2, I4, I5}}.
•Now, Join step is complete and Prune step will be used to reduce the size of
C3. Prune step helps to avoid heavy computation due to large Ck.
 Step 3: Generating 3-itemset Frequent Pattern

• Based on the Apriori property that all subsets of a frequent itemset must also
be frequent, we can determine that four latter candidates cannot possibly be
frequent. How ?
• For example , lets take {I1, I2, I3}. The 2-item subsets of it are {I1, I2}, {I1, I3}
& {I2, I3}. Since all 2-item subsets of {I1, I2, I3} are members of L2, We will
keep {I1, I2, I3} in C3.
• Lets take another example of {I2, I3, I5} which shows how the pruning is
performed. The 2-item subsets are {I2, I3}, {I2, I5} & {I3,I5}.
• BUT, {I3, I5} is not a member of L2 and hence it is not frequent violating
Apriori Property. Thus We will have to remove {I2, I3, I5} from C3.
• Therefore, C3 = {{I1, I2, I3}, {I1, I2, I5}} after checking for all members of
result of Join operation for Pruning.
• Now, the transactions in D are scanned in order to determine L3, consisting of
those candidates 3-itemsets in C3 having minimum support.
Step 4: Generating 4-itemset Frequent Pattern

• The algorithm uses L3 Join L3 to generate a candidate set of

4-itemsets, C4. Although the join results in {{I1, I2, I3, I5}},
this itemset is pruned since its subset {{I2, I3, I5}} is not
frequent.
• Thus, C4 = φ , and algorithm terminates, having found all
of the frequent items. This completes our Apriori
Algorithm.
• What’s Next ?
These frequent itemsets will be used to generate strong
association rules ( where strong association rules satisfy both
minimum support & minimum confidence).
 Step 5: Generating Association Rules from Frequent
Itemsets

• Procedure:
• For each frequent itemset “l”, generate all nonempty subsets of l.
• For every nonempty subset s of l, output the rule “s  (l-s)” if
support_count(l) / support_count(s) >= min_conf where
min_conf is minimum confidence threshold.

• Back To Example:
We had L = {{I1}, {I2}, {I3}, {I4}, {I5}, {I1,I2}, {I1,I3}, {I1,I5}, {I2,I3},
{I2,I4}, {I2,I5}, {I1,I2,I3}, {I1,I2,I5}}.
– Lets take l = {I1,I2,I5}.
– Its all nonempty subsets are {I1,I2}, {I1,I5}, {I2,I5}, {I1}, {I2}, {I5}.
 Step 5: Generating Association Rules from
Frequent Itemsets
• Let minimum confidence threshold is , say 70%.
• The resulting association rules are shown below,
each listed with its confidence.
– R1: I1 ^ I2  I5
• Confidence = sc{I1,I2,I5}/sc{I1,I2} = 2/4 = 50%
• R1 is Rejected.
– R2: I1 ^ I5  I2
• Confidence = sc{I1,I2,I5}/sc{I1,I5} = 2/2 = 100%
• R2 is Selected.
– R3: I2 ^ I5  I1
• Confidence = sc{I1,I2,I5}/sc{I2,I5} = 2/2 = 100%
• R3 is Selected.
Step 5: Generating Association Rules from
Frequent Itemsets
– R4: I1  I2 ^ I5
• Confidence = sc{I1,I2,I5}/sc{I1} = 2/6 = 33%
• R4 is Rejected.
– R5: I2  I1 ^ I5
• Confidence = sc{I1,I2,I5}/{I2} = 2/7 = 29%
• R5 is Rejected.
– R6: I5  I1 ^ I2
• Confidence = sc{I1,I2,I5}/ {I5} = 2/2 = 100%
• R6 is Selected.
In this way, We have found three strong
association rules.
Methods to Improve Apriori’s Efficiency

• Hash-based itemset counting: A k-itemset whose corresponding

hashing bucket count is below the threshold cannot be frequent.
• Transaction reduction: A transaction that does not contain any
frequent k-itemset is useless in subsequent scans.
• Partitioning: Any itemset that is potentially frequent in DB must be
frequent in at least one of the partitions of DB.
• Sampling: mining on a subset of given data, lower support threshold
+ a method to determine the completeness.
• Dynamic itemset counting: add new candidate itemsets only when all of
their subsets are estimated to be frequent.
Mining Frequent Patterns Without Candidate
Generation
• Compress a large database into a compact,
Frequent- Pattern tree (FP-tree) structure
– highly condensed, but complete for frequent pattern
mining
– avoid costly database scans
• Develop an efficient, FP-tree-based frequent pattern
mining method
– A divide-and-conquer methodology: decompose
mining tasks into smaller ones
– Avoid candidate generation: sub-database test
only!