CS 316: ALGORITHMS

Lecture 5
Analysis Of Recursive Algorithms
Dr. Ensaf Hussein
Department Of Computer Science
ANALYZING RECURSIVE
ALGORITHMS

MASTER METHOD
 3. MASTER METHOD
The idea is to solve a class of recurrences that have the
form
T (n)  aT (n / b)  f (n)
a >= 1 and b > 1, and f is asymptotically positive!

• a’s subproblems of size n/b are solved recursively, each in time

T(n/b)
• f(n) is the cost of dividing the problem and combining the results.

• Ex: In merge-sort T (n)  2T (n / 2)  (n)

a=2 b=2 f(n) = n

 MASTER METHOD

Split problem into a parts at logbn levels.

logb n logb a
There are a  n
leaves
 MASTER METHOD
log b n log b a
Number of leaves: a n
Iterating the recurrence, expanding the tree yields
T ( n)  f ( n)  aT (n / b)
 f (n)  af ( n / b)  a 2T ( n / b 2 )
 f (n)  af ( n / b)  a 2T ( n / b 2 )  ...
 a logb n1 f (n / b logb n 1 )  a logb nT (1)
Thus,
logb n 1
T ( n)   j 0
a j f (n / b j )  ( nlogb a )

The first term is a division/recombination cost (totaled across all levels of

the tree)
The second term is the cost of doing all subproblems of size 1 (total of
all work pushed to leaves) n logb a
 MASTER METHOD INTUITION

Three common cases:

• Running time dominated by cost at leaves
• Running time evenly distributed throughout the
tree
• Running time dominated by cost at root
Consequently, to solve the recurrence, we need only
to characterize the dominant term
In each case compare f with
( n) O(nlogb a )
MASTER THEOREM SUMMARIZED
Given a recurrence of the form T (n)  aT (n / b)  f (n)
1. 
f (n)  O n logb a  

 T (n)   n logb a 
2. 
f (n)   nlogb a 

 T (n)   n logb a lg n 
3.  
f (n)   nlogb a  and af (n / b)  cf (n), for some c  1, n  n0
 T (n)    f (n) 
The master method cannot solve every recurrence of this
form; there is a gap between cases 1 and 2, as well as
cases 2 and 3
MASTER METHOD CASE 1

f (n)  O(n logb a  )for some constant  0

• f (n) grows polynomially (by factor n) slower than nlogb a
The work at the leaf level dominates
• Summation of recursion-tree levels log b a
O(n )
• Cost of all the leaves
• Thus, the overall cost (nlogb a )
(nlogb a )
MASTER METHOD CASE 2
logb a
f ( n )  ( n lg n)
• f (n) and n logbare
a
asymptotically the same
The work is distributed equally throughout the
tree
• (level cost) * (number of levels)

log b a
T ( n )  ( n lg n)
MASTER METHOD CASE 3

f (n)  (n logb a  )for some constant  0

• Inverse of Case 1
log b a
• f (n) grows polynomially faster than n
• Also need a regularity condition
c  1 and n0  0 such that af (n / b)  cf (n) n  n0
The work at the root dominates
T (n)  ( f (n))
STRATEGY
Extract a, b, and f (n) from a given recurrence
logb a
Determine n
logb a
Compare f (n) and n asymptotically
Determine appropriate MT case, and apply
Example: merge sort

T (n)  2T (n / 2)  (n)
a  2, b  2; nlogb a  nlog2 2  n  (n)
Also f (n)  (n)
 
 Case 2: T (n)   nlogb a lg n    n lg n 
EXAMPLES

T(n) = 2T(n/2) + n

a = 2, b = 2, log22 = 1

Compare nlog22 with f(n) = n

 f(n) = (n)  Case 2

 T(n) = (nlgn)
 EXAMPLES
Ex: 1 Binary-search(A, p, r, s):
T (n)  T (n / 2)  1 q¬(p+r)/2
if A[q]=s then return q
a  1, b  2; n log2 1  1 else if A[q]>s then
also f ( n)  1, f ( n)  (1) Binary-search(A,p,q-1,s)
else
 Case 2: T (n)  (lg n) Binary-search(A,q+1,r,s)

Ex: 2
T (n)  9T (n / 3)  n
a  9, b  3;
log3 9 
f ( n )  n, f ( n )  O ( n ) with   1
 Case 1: T (n)    n 2 
EXAMPLES
Ex: 3
T (n)  3T (n / 4)  n lg n
a  3, b  4; n log 4 3  n 0.793
f (n)  n lg n, f (n)  (n log 4 3 ) with   0.2
 Case 3:
Regularity condition
af (n / b)  3(n / 4) lg(n / 4)  (3 / 4)n lg n  cf (n) for c  3 / 4
T (n)  (n lg n)

Ex: 4
T (n)  2T (n / 2)  n lg n
a  2, b  2; n log2 2  n1
f (n)  n lg n, f (n)  (n1 ) with  ?
also n lg n / n1  lg n
 neither Case 3 nor Case 2!
 EXAMPLES (3)
Ex: 5
T (n)  4T (n / 2)  n 3
a  4, b  2; n log 2 4  n 2
f (n)  n3 ; f (n)  (n 2 )
 Case 3: T (n)    n3 
Checking the regularity condition
4 f (n / 2)  cf (n)
4n3 / 8  cn3
n3 / 2  cn3
c  13// 24  1
RECURSIVE ALGORITHMS

 Factorial
 Fibonacci Sequence
 Tower of Hanoi
ANALYSIS OF RECURSIVE ALGORITHMS
FACTORIAL
ALGORITHM F(n)
// Output: n!
Input size: n
if n = 0 Basic operation: ×
return 1
M(n) = M(n-1) + 1 for n > 0
else
return n×F(n-1) to compute
F(n-1) to multiply
n and F(n-1)
ANALYSIS OF RECURSIVE FACTORIAL
T(0) = 1 (1)
T(n) = c + T(n - 1) (2)
= c + c + T(n - 2) by subtituting T(n – 1) in (2)
= c +c +c + T(n - 3) by substituting T(n – 2) in (2)
…
= kc + T(n - k)

The base case is reached when n – k = 0  k = n, we then

have:
T(n) = nc + T(n - n)
= cn + T(0)
= cn + 1
Therefore the method factorial is O(n)
ANALYSIS OF RECURSIVE (CONTD.):
TOWER OF HANOI
 TOWER OF HANOI (CONTD.)

M(n-1
)

M(n) = M(n-1) + 1 + M(n-1)

M(1) = 1 1

M(n-1)
TOWER OF HANOI (CONTD.)
ALGORITHM ToH(n, s, m, d)
if n = 1 M(1) = 1

print “move from s to d”

else
ToH(n-1, s, d, m) M(n-1)
print “move from s to d” M(1)

ToH(n-1, m, s, d) M(n-1)
Expanding:
T(1) = 1 (1)
T(n) = 2T(n – 1) + c if n > 1 (2)
= 2[2T(n – 2) + c] + c = 22 T(n – 2) + 2c + c by substituting T(n – 1) in (2)

= 22 [2T(n – 3) + c] + 2c + c = 23 T(n – 3) + 22c + 2c + c by substituting T(n-2) in (2)

= 23 [2T(n – 4) + c] + 22 c+ 2c + c = 24 T(n – 4) + 23 c + 22c + 21c + 20c

by substituting T(n – 3) in (2)

= ……
= 2k T(n – k) + c[2k- 1 + 2k– 2 + . . . 21 + 20]

The base case is reached when n – k = 1  k = n – 1, we then have:

Therefore, The method hanoi is O(2n)

FIBONACCI SEQUENCE
Leonardo Fibonacci, the Italian mathematician
of the middle ages, first used the sequence
1, 1, 2, 3, 5, 8, 13, 21, 34, 55, ......
in his example of Breeding Rabbits.
Fibonacci used the sequence to explain the breeding pattern of
rabbits under ideal circumstances.

The set up for the problem is as follows:

 Rabbits mate when they are one month old and at the end
of the second month, the female can reproduce exactly one
pair of rabbits, one male and one female.
 One pair of newly born rabbits (one male and one female) is
observed initially for the study.
 It is assumed that the newborns of rabbits never die.
FIBONACCI SEQUENCE
ANALYSIS OF RECURSIVE FIBONACCI
fibonacci (n) { // Recursively calculates Fibonacci number
if( n == 1 || n == 2)
return 1;
else
return fibonacci(n – 1) + fibonacci(n – 2);
}

T(n) = c if n = 1 or n = 2 (1)
T(n) = T(n – 1) + T(n – 2) + b if n > 2 (2)

We determine a lower bound on T(n):

Expanding: T(n) = T(n - 1) + T(n - 2) + b

≥ T(n - 2) + T(n-2) + b
 = 2T(n - 2) + b
= 2[T(n - 3) + T(n - 4) + b] + b by substituting T(n - 2) in (2)
 2[T(n - 4) + T(n - 4) + b] + b
= 22T(n - 4) + 2b + b
= 22[T(n - 5) + T(n - 6) + b] + 2b + b
by substituting T(n - 4) in (2)
≥ 23T(n – 6) + (22 + 21 + 20)b
...
 2kT(n – 2k) + (2k-1 + 2k-2 + . . . + 21 + 20)b
= 2kT(n – 2k) + (2k – 1)b
The base case is reached when n – 2k = 2  k = (n - 2) / 2
Hence T(n) ≥ 2 (n – 2) / 2 T(2) + [2 (n - 2) / 2 – 1]b
= (b + c)2 (n – 2) / 2 – b
= [(b + c) / 2]*(2)n/2 – b  Recursive Fibonacci is exponential
REFERENCES

Introduction to Algorithms, Second Edition by Thomas H.

Cormen, Charles E. Leiserson, Ronald L. Rivest and Clifford
Stein, The MIT Press © 2001
Chapter 4.3