Advanced Analysis of Algorithms

Cormen (4.3-4.6)
Levetin (2.4-2.5)
Fibonacci Sequences
(Natural Models)
 Fibonacci Sequence
• By studying Fibonacci numbers and constructing
Fibonacci sequence we can imagine how
mathematics is connected to apparently unrelated
things in this universe.
• Even though these numbers were introduced in
1202 in Fibonacci’s book Liber abaci, but these
numbers and sequence are still fascinating and
mysterious to people of today.
• Fibonacci, gave a problem in his book whose
solution was the Fibonacci sequence as we will
discuss it today.
 Fibonacci’s Problem
Statement:
• Start with a pair of rabbits, one male and one female,
born on January 1.
• Assume that all months are of equal length and that
rabbits begin to produce two months after their own
birth.
• After reaching age of two months, each pair produces
another mixed pair, one male and one female, and then
another mixed pair each month, and no rabbit dies.
How many pairs of rabbits will there be after one year?

Answer: The Fibonacci Sequence!

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, . . .
 Construction of Mathematical
Model
F0 = 0

F1 = 1 end of month 1

end of month 2

F2 = 1
end of month 3
F3 = 2

F4 = 3
end of month 4

end of month 5
F5 = 5

end of month 6
F6 = 8

end of month 7

F7 = 13 end of month 12
... ... ...
 Construction of Mathematical Model
• Total pairs at level k = Total pairs at level k-1 + Total
pairs born at level k (1)
• Since
Total pairs born at level k = Total pairs at level k-2 (2)
• Hence by equation (1) and (2)
Total pairs at level k = Total pairs at level k-1 + Total
pairs at level k-2
• Now let us denote
Fk = Total pairs at level k
• Now our recursive mathematical model will become
Fk = Fk-1 + Fk-2
 Computing Values using Mathematical Model
Since Fk = Fk-1 + Fk-2 F0 = 0, F1= 1
• F 2 = F 1 + F 0= 1 + 0 = 1
• F 3 = F 2 + F 1= 1 + 1 = 2
• F 4 = F 3 + F 2= 2 + 1 = 3
• F 5 = F 4 + F 3= 3 + 2 = 5
• F 6 = F 5 + F 4= 5 + 3 = 8
• F7 = F6 + F5= 8 + 5 = 13
• F8 = F7 + F6= 13 + 8 = 21
• F9 = F8 + F7= 21 + 13 = 34
• F10 = F9 + F8= 34 + 21 = 55
• F11 = F10 + F9= 55 + 34 = 89
• F12 = F11 + F10= 89 + 55 = 144 . . .
 Explicit Formula Computing
Theorem: Fibonacci Numbers
The fibonacci sequence F0,F1, F2,…. Satisfies
the recurrence relation
Fk Fk  1  Fk  2  k 2
with initial condition F0 F1 1

Find the explicit formula for this sequence.

Solution:
The given fibonacci sequence
Fk  Fk  1  Fk  2
Let tk is solution to this, then characteristic equation
t 2  t  1 0
 Fibonacci Sequence
1 1 4
t 
2
1 5 1 5
t1  , t2 
2 2
For some real C and D fibonacci sequence satisfies the relation
n n
 1 5   1 5 
Fn C    D
 

  n 0
 2   2 
n 0
0 0
 1 5   1 5 

F0 C    D  
  2 
 2   
 F0 C  D
 C  D 0 1  F0 0
 Fibonacci Sequence
Now n 1
1 5  1 5 
F1 C    D
 


 2   2 
1 5 1 5
 C  D 1 2  F1 1
2 2

Solving 1and 2 simultaneo usly we get

1 1
C  , D 
5 5
Hence
n n
 1   1  5   1   1  5 
Fn    
 5   2   5   2 5 

 Fibonacci Sequence
After simplifying we get
n n
1  1 5  1  1 5 
Fn      
5  2  5  2 

which is called the explicit formula for the

Fibonacci sequence recurrence relation.

 1 5   1 5 
Let   
 and    then
2   2 
   
1 n 1 n
Fn    
5 5
 Verification of the Explicit
Example: Compute F Formula 3

1 n 1 n  1 5   1 5 
Since Fn     where   
 and    then
  2 
5 5  2   
3 3
1  1 5  1  1 5 
F3    

 
5 2  5  2 


1  1  3.12. 5  3.1.5  5 5  1  1  3.12. 5  3.1.5  5 5 

Now, F3   

 
5 8  5  8 


F3 
1
8. 5

1  3.1. 5  3.1.5  5 5  1
8. 5

1  3.1. 5  3.1.5  5 5 
F3 
1
8. 5
 
1  3.1. 5  3.1.5  5 5  1  3.1. 5  3.1.5  5 5 2
 Recursive Algorithm Computing Fibonacci Numbers

Fibo-R(n)
if n = 0
then 0
Terminating conditions
if n = 1
then 1
else Fibo-R(n-1) + Fibo-R(n-2) Recursive calls
 Running Time of Recursive Fibonacci Algorithm
• Least Cost: To find an asymptotic bound of computational
cost of this algorithm, we can use a simple trick to solve this
recurrence containing big oh expressions
• Simply drop the big O from the recurrence, solve the
recurrence, and put the O back. Our recurrence

O (1) if n  2
T (n) 
T (n  1)  T (n  2) n 2

will be refined to
1 if n  2
T (n) 
T (n  1)  T (n  2) n 2
 Running Time of Recursive Fibonacci Algorithm

• Guess that Fn+1 is the least cost to solve this

recurrence. Why this guess?
 n  0, T(n)  Fn+1
then Fn+1 will be minimum cost for this recurrence
• We prove it by mathematical induction
Base Case
There are two base cases
For n = 0, T(0) = 1 and F1 = 1, hence T(0)  F1
For n = 1, T(1) = 1 and F2 = 1, hence T(1)  F2
 Running Time of Recursive Fibonacci Algorithm
• Inductive Hypothesis
Let us suppose that statement is true some k  1
T(k)  Fk+1 , for k =0, 1, 2,. . . and k  1
• Now we show that statement is true for k + 1
• Now, T(k + 1) = T(k) + T(k -1) By definition on T(n)
T(k + 1) = T(k) + T(k -1)  Fk+1 + Fk = Fk+2 Assumption
T(k + 1)  Fk+2

• Hence the statement is true for k + 1.

• We can now say with certainty that running time of
this recursive Fibonacci algorithm is at least (Fn+1).
Running Time of Recursive Fibonacci Algorithm
• Now we have proved that
T(n)  Fn+1 , n  0 (1)
• We already proved in solution to recursive relation that

1 n 1 n  1 5   1 5 
Fn    
 where    
and    (2)
5 5 2  2 
   

It can be easily verified that Fn  n/5  (3/2)n

From the equations (1) and (2), T(n)  Fn+1  Fn  (3/2)n

Hence we can conclude that r unning time of our recursive

Fibonacci Algorithm is:
T(n) =  (3/2)n
 Golden Ratio

• W say that two quantities, x and y, (x < y), are in

the golden ratio if the ratio between the sum, x + y,
of these quantities and the larger one, y, is the
same as the ratio between the larger one, y, and
the smaller one x.

x y y
 1.62
y x

• Mathematicians have studied the golden ratio

because of its unique and interesting properties.
 Golden Ratio

1 5
 1.62
2
x   1 0.62,
 1
y 1


of course x  y and
xy y  1 1
 i.e. 
y x 1  1
2
     1 0
 Drawback in Recursive Algorithms
Recursion Tree
F(n)

F(n-1) F(n-2)

F(n-2) F(n-3) F(n-3) F(n-4)

F(0) F(1) F(1) F(0)

 Generalization of Rabbits
Statement: Problem
• Start with a pair of rabbits, one male and one female,
born on January 1.
• Assume that all months are of equal length and that
rabbits begin to produce two months after their own
birth.
• After reaching age of two months, each pair produces
two other mixed pairs, two male and two female, and
then two other mixed pair each month, and no rabbit dies.
How many pairs of rabbits will there be after one year?

Answer: Generalization of Fibonacci Sequence!

0, 1, 1, 3, 5, 11, 21, 43, 85, 171, 341, 683, . . .
Construction of Mathematical Model

F0 = 0

F1 = 1

F2 = 1

F3 = 3

F4 = 5

F5 = 11

F6 = 21
 Construction of Mathematical Model
• Total pairs at level k =
Total pairs at level k-1 + Total pairs born at level k (1)
• Since
Total pairs born at level k =
2 x Total pairs at level k-2
(2)
• By (1) and (2), Total pairs at level k =
Total pairs at level k-1 + 2 x Total pairs at level k-2
• Now let us denote
Fk = Total pairs at level k
• Our recursive mathematical model:
Fk = Fk-1 + 2.Fk-2
• General Model (m pairs production): Fk = Fk-1 + m.Fk-2
 Generalization
• Recursive mathematical model
(one pair production)
Fk = Fk-1 + Fk-2
• Recursive mathematical model
(two pairs production)
Fk = Fk-1 + 2.Fk-2
• Recursive mathematical model
(m pairs production)
Fk = Fk-1 + m.Fk-2
 Computing Values using Mathematical Model
Since Fk = Fk-1 + 2.Fk-2 F0 = 0, F1 = 1
• F2 = F1 + 2.F0= 1 + 0 = 1
• F3 = F2 + 2.F1= 1 + 2 = 3
• F4 = F3 + 2.F2= 3 + 2 = 5
• F5 = F4 + 2.F3= 5 + 6 = 11
• F6 = F5 + F4= 11 + 10 = 21
• F7 = F6 + F5= 21 + 22 = 43
• F8 = F7 + F6= 43 + 42 = 85
• F9 = F8 + F7= 85 + 86 = 171
• F10 = F9 + F8= 171 + 170 = 341
• F11 = F10 + F9= 341 + 342 = 683
• F12 = F11 + F10= 683 + 682 = 1365 . . .
 Another Generalization of Rabbits Problem
Statement:
• Start with a different kind of pair of rabbits, one male and
one female, born on January 1.
• Assume all months are of equal length and that rabbits
begin to produce three months after their own birth.
• After reaching age of three months, each pair produces
another mixed pairs, one male and other female, and then
another mixed pair each month, and no rabbit dies.
How many pairs of rabbits will there be after one year?

Answer: Generalization of Fibonacci Sequence!

0, 1, 1, 1, 2, 3, 4, 6, 9, 13, 19, 28, 41, 60, . . .
Construction of Mathematical
Model
F0 = 0

F1 = 1

F2 = 1

F3 = 1

F4 = 2

F5 = 3

F6 = 4

F7 = 6

F8 = 9

F9 = 13

F10 = 19
 Construction of Mathematical Model
• Total pairs at level k =
Total pairs at level k-1 + Total pairs born at level k (1)
• Since
Total pairs born at level k = Total pairs at level k-3 (2)
• By (1) and (2)
Total pairs at level k =
Total pairs at level k-1 + Total pairs at level k-3
• Now let us denote
Fk = Total pairs at level k
• This time mathematical model: Fk = Fk-1 + Fk-3
 Computing Values using Mathematical Model
Since Fk = Fk-1 + Fk-3 F0 = 0, F1= F2= 1
• F 3 = F 2 + F 0= 1 + 0 = 1
• F 4 = F 3 + F 1= 1 + 1 = 2
• F 5 = F 4 + F 2= 2 + 1 = 3
• F 6 = F 5 + F 3= 3 + 1 = 4
• F 7 = F 6 + F 4= 4 + 2 = 6
• F 8 = F 7 + F 5= 6 + 3 = 9
• F9 = F8 + F6= 9 + 4 = 13
• F10 = F9 + F7= 13 + 6 = 19
• F11 = F10 + F8= 19 + 9 = 28
• F12 = F11 + F9= 28 + 13 = 41 . . .
 More Generalization
• Recursive mathematical model
(one pair, production after three months)
Fk = Fk-1 + Fk-3
• Recursive mathematical model
(two pairs, production after three months)
Fk = Fk-1 + 2.Fk-3
• Recursive mathematical model
(m pairs, production after three months)
Fk = Fk-1 + m.Fk-3
• Recursive mathematical model
(m pairs, production after n months)
Fk = Fk-1 + m.Fk-n
 Applications of Fibonacci Sequences and Golden
Ratio
Fibonacci sequences
• Are used in trend analysis
• By some pseudorandom number generators
• The number of petals is a Fibonacci number.
• Many plants show the Fibonacci numbers in the
arrangements of the leaves around the stems.
• Seen in arrangement of seeds on flower heads
• Consecutive Fibonacci numbers give worst case
behavior when used as inputs in Euclid’s algorithm.
• As n approaches infinity, the ratio F(n+1)/F(n)
approaches the golden ratio:
 =1.6180339887498948482...
 Applications of Fibonacci Sequences

Fibonacci sequences
• The Greeks felt that rectangles whose sides are in
the golden ratio are most pleasing
• Sum of the first n Fibonacci numbers is F(n+2)-1.
• The shallow diagonals of Pascal’s triangle sum to
Fibonacci numbers.
• Except n = 4, if F(n) is prime, then n is prime.
• Equivalently, if n not prime, then F(n) is not prime.
• gcd( F(n), F(m) ) = F( gcd(n, m) )
Pascal’s Triangle
 Euclid’s algorithm
• The Euclidean Algorithm for finding GCD(A,B) is as
follows:
• If A = 0 then GCD(A,B)=B, since the GCD(0,B)=B, and
we can stop.
• If B = 0 then GCD(A,B)=A, since the GCD(A,0)=A, and
we can stop.
• Write A in quotient remainder form (A = B⋅Q + R)
• Find GCD(B,R) using the Euclidean Algorithm since
GCD(A,B) = GCD(B,R)
 Example
• Find the GCD of 270 and 192
• A=270, B=192A ≠0B ≠0
• Use long division to find that 270/192 = 1 with a
remainder of 78.
• We can write this as: 270 = 192 * 1 +78
• Find GCD(192,78), since GCD(270,192)=GCD(192,78)
A=192, B=78 A ≠0 B ≠0 Use long division to find that
192/78 = 2 with a remainder of 36.
• We can write this as:192 = 78 * 2 + 36
• Find GCD(78,36), since GCD(192,78)=GCD(78,36)
A=78, B=36 A ≠0 B ≠0 Use long division to find that
78/36 = 2 with a remainder of 6.
• We can write this as:78 = 36 * 2 + 6
• Find GCD(36,6), since GCD(78,36)=GCD(36,6) A=36,
B=6
• And so on……
Golden Ratio
Golden Ratio
Golden Ratio
Golden Ratio
 What is Recursion?
• More concisely, a recursive function or definition
is defined in terms of itself.
• Recursion is a computer algorithm that calls itself
in a steps having a termination condition.
• The successive repetitions are processed up to
the critical step where the condition is met.
• In recursive algorithms, each repetition is
processed from the last one called to the first.
• Recursion is a wonderful technique for dealing
with many problems where problems and sub-
problems have same mechanism to solve it.
 Merits and Demerits of Recursion
• Recursive solutions are much easier to conceive of
and code than their iterative counterparts.
• Every problem is not solvable using this approach
What kinds of problems are solved with recursion?
• Generally, problems which are defined in terms of
themselves are usually good candidates for
recursive techniques.

Example
• Finding factorial of a number is an easiest example
one can think using recursion
 Recursive Mathematical Model
• Since n! can be computed as: 5! = 5*4*3*2*1.
• If we have a close look at this, we notice that
5! = 5*4!.
• Now if denote F(n) = n! then it can be written as
F(n) = n.F(n-1)
• Assume 0! = 1 i.e. F(0) = 1, and solve till termination
F(n) = n.F(n-1) = n.(n-1).F(n-2) = n.(n-1).(n-2).F(n-3)
...
F(n) = n.(n-1).(n-2). . . 2.1.F(0) = n.(n-1).(n-2). . . 2.1.1

1 if n 0
F (n) 
n.F (n  1) otherwise
 Methods for solving Recurrence

– Assumptions in solving recurrence

– The Substitution Method
– The Recursion Tree Method
– The Master Theorem
Assumption in Solving Recurrence Relation
• Neglect certain technical details solve recurrences
• Assume integer arguments to functions
– Because running time T (n) is always defined
when n is an integer
• Consequently, for convenience, we shall omit
statements of boundary conditions of recurrences
and assume that T (n) is constant for small n
• Recurrence for worst-case time of MERGE-SORT

(1) if n 1

T (n)   n   n
T  2   T  2   (n) otherwise
    
Assumption in Solving Recurrence Relation
• We do not give any explicit value for small n
Because changing n, T (1) changes solution to
recurrence. Solution typically doesn’t change by more
than a constant factor, so order of growth is unchanged
• Solve recurrences, often omit floors, ceilings, etc.
• First analyze without these details and later determine
whether such assumptions matter or not.
• Usually it dose not matter but it is important to know
when it does and when it does not

(1) if n 1

T (n)  n
2.T ( 2 )  (n) otherwise
Methods Solving Recurrence Relation

The Substitution Method

 Substitution Method
• Substitution method has two steps
• Guess the form of the solution
• Use mathematical induction to find constants and
show that the solution does work
• The name Substitution comes from the substitution
of guessed answer for the function when the
inductive hypothesis is applied to smaller values.
• Method is powerful, but it can be applied only in
cases when it is easy to guess the form of answer
• The substitution method can be used to establish
either upper or lower bounds on a recurrence.
Some Important Rules used in this Section
Prove that log a b log b c log a c

Proof
let us suppose that
b c
log  s a log t
b

s t
 a b and b c

Now b t c, ( a s ) t c
st
a c, log ca  s t
s t logca , log ba log bc log ca proved
Some Important Rules used in this Section
log n4 log 34
Prove that 3 n
Proof
log n4 log 34
3 n
logn4 log34
 log 3 (3 ) log 3 (n )
n 3 3 n
 log log log log
4 3 4 3

n 3 n
 log log log
4 4 3

n n
 log log 4 4
b
 log log log
a
c
b
c
a
  0 n 0
s (n) 
c  s (n  1) n  0
• s(n) =
c + s(n-1)
c + c + s(n-2)
2c + s(n-2)
2c + c + s(n-3)
3c + s(n-3)
…
kc + s(n-k) = ck + s(n-k)
  0 n 0
s (n) 
c  s (n  1) n  0
• So far for n >= k we have
– s(n) = ck + s(n-k)
• What if k = n?
– s(n) = cn + s(0) = cn
  0 n 0
s (n) 
c  s (n  1) n  0
• So far for n >= k we have
– s(n) = ck + s(n-k)
• What if k = n?
– s(n) = cn + s(0) = cn
• So  0 n 0
s (n) 
c  s (n  1) n  0
• Thus in general
– s(n) = cn
  0 n 0
s (n) 
n  s (n  1) n  0
• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
=…
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
  0 n 0
s (n) 
n  s (n  1) n  0
• s(n)
= n + s(n-1)
= n + n-1 + s(n-2)
= n + n-1 + n-2 + s(n-3)
= n + n-1 + n-2 + n-3 + s(n-4)
=…
= n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k)
n

i
=i n  k 1
 s (n  k )
  0 n 0
s (n) 
n  s (n  1) n  0
• So far for n >= k we have
n

i
i n  k 1
 s (n  k )
  0 n 0
s (n) 
n  s (n  1) n  0
• So far for n >= k we have
n

i
i n  k 1
 s (n  k )

• What if k = n?
  0 n 0
s (n) 
n  s (n  1) n  0
• So far for n >= k we have
n

i
i n  k 1
 s (n  k )

• What if k = n?
n
n 1
n


i 1
i  s (0)   i  0  n
i 1 2
  0 n 0
s (n) 
n  s (n  1) n  0
• So far for n >= k we have
n

i
i n  k 1
 s (n  k )

• What if k = n?
n
n 1
n


i 1
i  s (0)   i  0  n
i 1 2
• Thus in general
n 1
s ( n)  n
2
 The Substitution Method
Solve the recurrence relation given below.
1 if n 1

T ( n)  n
3 T ( 4 )  n otherwise
Solution:
n
(1) T (n) 3 T ( )  n
4
replace the value of n by n 4 in (1)
n n n
(2) T ( ) 3 T ( 2 ) 
4 4 4
 The Substitution Method
n n n
(3) T ( 2 ) 3 T ( 3 )  2 and so on
4 4 4
n n n
(4) T ( k1
) 3 T ( ) 
4 4k 4k  1
n
Now substitute the value of T ( ) from (2) to (1)
4
 n n
(5) T (n) 3 3T ( 2 )    n
 4 4
n 3
3 2 T ( 2 )  ( ) n  n
4 4
n
substitute the value of T ( 2
) from (3) to (5) equation, we have
4
 2n n  3
T ( n) 3  3 T ( 3 )  2   ( ) n  n
 4 4  4
 The Substitution Method

After continuing this process

k n 3 k1 3 k 2 3 3 0
T (n) 3 T ( k )  ( ) n  ( )  ( )n  ( ) n
4 4 4 4 4
Let us suppose that n can be expressed as n 4 k
kn  3 3 2 3 k1
T (n) 3 T ( )  n 1  ( )  ( )  ....  ( ) 
n  4 4 4 
 3 k 
 1 ( ) 
k 4 2 k1 1 xk
T ( n) 3 T (1)  n 1 ( )  1  x  x  ...  x 1 ( )
 3  1 x
1
 4 
k 3k
T (n) 3 1  4n(1  k )  T (1) 1
4
 The Substitution Method
k k
k 3 k 3
T (n) 3  4n (1  k ) 3  4n (1  )
4 n
k
k n 3
3  4n( )
n
k k k k
3  4(n  3 ) 1 3  4n  4 3

log n
4 n  3 3 4n  3 3
k 4

 4 k n, k log n4
 The Substitution Method
logn4
T (n) 4n  3 3
log34
T (n) 4n  3 n

T (n) 4n  3 n  0  1

 T (n) 4n  3 n

Hence T (n)   (n) let  log 34

 A Hard Example
Solve the recurrence relation given below
1 if n 1
 where 0  a  b
T ( n)  n
aT ( b )  n otherwise
n n n
Solution : T (n) aT ( )  n a a T ( 2 )  a   n
b b b
2 n a 2 n n a
a T ( 2 )  ( )n  n a  aT ( 3 )  2   ( )n  n
b b  b b  b
3 n a 2 a
a T ( 3 )  ( ) n  ( ) n  n
b b b
 A Hard Example
Continuing this process we get
k n a k1 a k 2 a 1
T (n) a T ( k )  ( ) n  ( ) n  ....  ( ) n  n
b b b b
k n  a a 2 a k1 
a T ( k )  n 1  ( )  ( )  ....  ( ) 
b  b b b 

k
Let us suppose that, n b

k  a a k1
T (n) a T (1)  n 1  ( )  ....  ( ) 
 b b 
 A Hard Example

 a k
 1 ( )  k
b k b a
 a T (1)  n( )(1  k )
k
a T (1)  n 
 1 a  b a b
 b 
k k
k b b  a
a 1  n( )( k
)
b a b
k k
b b  a k
k
a  n( )( )  n b
b a n
 A Hard Example

k b k k
a  ( )(b  a )
b a

k b k b k
a  b  a
b a b a

b k b a b k
 b  ( )a
b a b a
 A Hard Example
b k a logbn
T ( n ) ( ) b  a
b a b a
b k a logba let  logba
 b  n
b a b a

 a  b  log  1  0    1
a
b

b a
 T ( n)  n  n  T (n)  (n)

b a b a
 More Hard Example using Substitution
Method
Solve the recurrence relation given below.

1 if n 1

T ( n)  n 2
3 T ( 4 )  cn otherwise
Solution:
n 2
(1) T (n) 3 T ( )  cn
4
n n n 2
(2) T ( ) 3 T ( 2 )  c( )
4 4 4
 Contd..

n n n 2
(3) T ( 2 ) 3 T ( 3 )  c( 2 )
4 4 4

n n n 2
(4) T ( 3 ) 3 T ( 4 )  c( 3 )
4 4 4

and so on
n n n 2
(5) T ( k  1 ) 3 T ( k )  c( k  1 )
4 4 4
 Contd..

n
Now substitute the value of T ( ) from (2) to (1)
4
 n n 2
(6) T (n) 3 3T ( 2 )  c( )   cn 2
 4 4 
2 n n 2 2
3 T ( 2 )  3c( )  cn
4 4
n
substitute the value of T ( 2 ) from (3) to (6), we have
4
 n n 2 n 2
T ( n) 3  3T ( 3 )  c ( 2 )   3c ( )  cn 2
2

 4 4  4
 Contd..
3 n 2 n 2 n 2 2
3 T ( 3 )  3 c( 2 )  3c ( )  cn
4 4 4

3 n 3 2 2 3 1 2 2
3 T ( 3 )  ( 2 ) cn  ( 2 ) cn  cn
4 4 4

After continuing this process, we get

k n  3 k1 3 k 2 3 0 2
T (n) 3 T ( k )   ( 2 )  ( 2 )  ( 2 )  cn
4  4 4 4 
 Contd..
k
Let us suppose that n can be expressed as n 4

k 2 3 3 2 3 k1 
T (n) 3 T (1)  cn 1   ( )  ( ) 
 16 16 16 
 3 k 
 1 ( ) 
T (n) 3 k 1  cn 2  ( 16 )
 1 3 
 16 
k
16 3
3 k  cn 2  (1  k )
13 16
 Contd..
k 2 2
since 4 n  (4 ) n
k

k 2
 (4 ) n 2 k 2
 16 n
k
k16 3 2
T (n) 3  cn  (1  k )
13 16
16 n 2
 3 k
16
k 2 k
3 k  cn 2  ( 2
) 3  c (n  3 )
13 n 13
16 2 16 k
 c n  (1  c)3
13 13
 Contd..

16 2 16 log n
T (n)  cn  (1  c)3 4
13 13
16 2 16 log 3
 cn  (1  c)n 4
13 13
let log34  where 0    1
16 2 16
T (n)  cn  (1  c)n 
13 13

Hence T (n)  (n ) 2
 Observations
1 if n 1

T (n)  n k
 a T ( )  cn otherwise
b
k1
k
en  fn log a
b suppose b a

k
k log b 1
en  fn b

k k1 . log b k k1
en  f .n b en  f .n
max( k , k1 )
(n )
Recursion Tree Method
 Recursion Tree Method
• Although substitution method can provide a
sufficient proof that a solution to a recurrence is
correct, sometimes difficult to give a good guess.
• Drawing out a recursion tree, is a straight forward
way to devise a good guess.
• In recursion tree, nodes represent costs of a sub-
problems in the set of recursive function
invocations.
• We sum costs within each level of the tree to obtain
a set of per-level costs.
• And then we sum all per-level costs to determine
the total cost of all levels of the recursion.
• Recursion trees are particularly useful when
recurrence describes running time of divide and
conquer algos.
 Recursion Tree Method
• Recursion tree is best one used to generate a good
guess, which is then verified by substitution method
• When using a recursion tree to generate a good
guess, we can often tolerate a small amount of
sloppiness since we have to verify it later on.
• If we are careful when drawing out a recursion tree
and summing costs, then we can use a recursion
tree as a direct proof of a solution to any
recurrence of any problem.
• Here, we will use recursion trees directly to prove
theorem that forms the basis of the master method.
 Recursion Tree

• We simplify the recurrence by dropping the floor and

replacing θ(n) by n to get T(n) = 2T(n /3) + n .
• Each node in a recursion tree represents one term in the
calculation of T(n) obtained by recursively substituting
the expression for T into itself.
• We construct a sequence of such trees of increasing
depths
• By summing the nodes in any one of these trees, we
obtain an expression for T(n) .
• After k iterations of this process we reach a tree in which
all bottom level nodes are T(n /3k )
• Note that there are 2i nodes at depth i, each of which has
value n /3i (for 0 <= i<= k -1).
• The sequence of trees terminates when all bottom level
nodes are T(1) , i.e. when n /3k =1, which implies
• The number of nodes at this bottom level is therefore

• Summing all nodes in the final recursion tree gives us

the following expression for T(n) .
• If we seek an asymptotic upper bound we may drop the
negative term to obtain
• Since log3 (2) < 1 , the first term dominates, and so we
guess: T(n) = O(n) .
 Recursion Tree Method
Example:
Solve the following recurrence using recurrence tree method

(1) if n 1
T (n) 
n
3.T ( )  (n 2 ) if otherwise
 4
Solution: The above recurrence can be written in the form


1 if n 1
T (n) 
n
 3.T ( )  cn 2 if otherwise
 4
Assumption: We assume that n is exact power of 4.
The recurrence tree is given in the next slide
 Recursion Tree Method
c.n2
T(n) = 3.T(n/4)+c.n2

T(n/4) T(n/4) T(n/4)

c.n2

c.(n/4)2 c.(n/4)2 c.(n/4)2

T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16)

 Recursion Tree Method

c.n2 c.n2
Suppose n=4k

c.(n/4)2 c.(n/4)2 c.(n/4)2 (3/16).c.n2

2
c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)
(3/16)2.c.n2

T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k)

 Recursion Tree Method
c.n2 c.n2
Suppose n = 4k
Log4n

c.(n/4)2 c.(n/4)2 c.(n/4)2 (3/16).c.n2

2 2 2
c(n/16)2 c(n/16)2 c(n/16) c(n/16)2 c(n/16)2 c(n/16) c(n/16) c(n/16) c(n/16)
2 2

(3/16)2.c.n2

(3/42)k-1 cn2
T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k) T(n/4k)
T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) (nlog43)
 Recursion Tree Method
• Now the total computation cost would be
= Cost of Childs + Cost of tree excluding childes
= Cost of Child x total number of Childs + Cost of
all levels excluding childes level
= total number of Childs + sum of costs at each
level excluding childes level.

logn4
T (n) 3  cost at Levels above child level

log3 4  3 0 3 1 3 k1 2
T (n) (n )   ( 2 )  ( 2 )  ( 2 )  cn
 4 4 4 
 Recursion Tree Method
Now total computational cost can be calculated as
log3 4  3 0 3 1 3 k1 2
T (n) (n )   ( 2 )  ( 2 )  ( 2 )  cn
 4 4 4 
Where 4 k n  k log 4 n

log3 4  3 0 3 1  2
T (n) (n )   ( 2 )  ( 2 )   cn
 4 4 
log3 4 1 2 log3 4 16 2
T (n) (n )( )cn (n )  cn
3 13
1 ( )
16
Hence T (n) (n 2 )
 Practice Problems

• Solve the following recurrences by Recursion

Tree method
• 1. T(n)=T(n/3)+T(2n/3)+n
• 2. T(n)=T(n/5)+T(4n/5)+n
• 3. T(n)=T(n/2)+n2
Master Theorem
 The Master Theorem
• Given: a divide and conquer algorithm
– An algorithm that divides the problem of size
n into a subproblems, each of size n/b
– Let the cost of each stage (i.e., the work to
divide the problem + combine solved
subproblems) be described by the function
f(n)
 The Master Theorem
• if T(n) = aT(n/b) + f(n) then
 

  
n 
logb a
 
f (n) O n logb a   

 
   0
T (n)  n  logb a
log n  f (n)  n 
logb a

  c 1
 
 f (n)   
f (n)  n logb a  AND 
 
 af (n / b)  cf (n) for large n
 Using The Master Method
• T(n) = 9T(n/3) + n
– a=9, b=3, f(n) = n
– nlog a = nlog 9 = (n2)
b 3

– Since f(n) = O(nlog 9 - ), where =1, case 1

3

applies:

T (n)  n log a
b
when f (n) On
log a  
b


– Thus the solution is T(n) = (n2)

Practice Problems
 Master Theorem
Lemma 1:
Let a  1, b > 1 be constants, f(n) a non-negative
function defined on exact power of b by recurrence


(1) if n 1
T (n)  n
a.T ( )  f (n) if n b i
 b
where i is a positive integer. Then
logb n  1
logb a n
T (n) (n ) 
j 0
j
a .f ( j )
b
 Master Theorem
Lemma 2: Let a  1, b > 1 be constants, let f(n) be a
non-negative function defined on exact power of b. A
function g(n) defined over exact powers of b by
logb n  1
n
g ( n)  j 0
j
a .f ( j )
b

can be bounded asymptotically for exact powers of b as

logb a 
1. If f (n) (n logb a
), for some constants   0, then g (n) (n )
a a
2. If f (n) (n log ), then g (n) (n log . lg n )
b b

3. If a.f(n/b) ≤ c.f(n), for some constant c < 1 and for all

n  b, then g(n) = (f(n))
 Master Theorem
Proof:
logb a  
Case 1: f (n) (n )
n n logb a 
Which implies that f ( j ) (( j ) ) (1)
b b

logb n  1
n
We are given that: g ( n)  
j 0
j
a .f ( j )
b
(2)

Substituting value from Equation (1) in Equation (2)

logb n  1
n logb a 
g (n) ( 
j 0
j
a .( j )
b
)
 Master Theorem
logb n  1 logb n  1 
n logb a.b

a 


a 
• Consider j
a .( j ) n logb
. ( ) j
lo gb a
j 0 b j 0 b
a
• Assume that:
logb a
x
b
• Taking log on both sides: log ( a ) log x
b lo g a b b
b
lo gb a
a b a a b
log b  log b log b x  log b  log b . log b log b x
a a
log b  log b log b x  0 log b x  x 1
 Master Theorem

n n logb n  1
logb  1
n logb logb  1
a.b
 j
a .( j )
a 
n logb a 
.  ( lo gb a
) n
j logb a  
.  )
(b  j

j 0 b j 0 b j 0

logb a    0  1  logb n  1
n .((b )  (b )  ...  (b ) )
• It is a geometric series with first term 1,
common ratio b and number of terms as logbn.
 logb n
Hence .g(n),  . logb b
a  (b ) 1 a   b logb n
1 logb a   n 1
n logb
( 
) n logb
(  ) n (  )
b 1 b 1 b 1
a  n  1 logb a logb a   logb a
n logb (  ) c.n  c.n (n )
b 1
Hence
 Master Theorem

logb a
Case 2: f (n) (n )
n n logb a
Which implies that: f ( j ) (( j ) ) (3)
b b

logb n  1
We are given that: g ( n)  n

j 0
j
a .f ( j )
b
(4)

Substituting value from Equation (3) in Equation (4)

logb n  1
n logb a
g (n) ( 
j 0
j
a .( j ) )
b
 Master Theorem
logb n  1
a
n
logb  1
n logb a
g (n) (  j
a .( j ) )
b
(n logb a
.  ( lo gb a
j
) )
j 0 j 0 b

a
We have already proved that logb a
1
b
logb n  1
logb a
g (n) (n logb a
.  1) (n . (1  1  ...  1 )
    
j 0
logb n number of terms

logb a n
(n . log b ) case is proved
 Master Theorem
Case 3: n n c
Given that: a. f ( ) c. f (n)  f ( )  . f (n)
b b a
n c n c 2 n c 2
f ( 2 )  . f ( ) ( ) . f (n)  f ( 2 ) ( ) . f (n)
b a b a b a
n c 2 n c 3 n c 3
f ( 3 ) ( ) . f ( ) ( ) . f (n)  f ( 3 ) ( ) . f (n)
b a b a b a
In general: f ( n ) ( c ) j . f (n)
bj a
Equivalently: a j . f ( n ) c j . f (n)
bj
 Master Theorem
logb n  1
n
We are given that: g (n)  
j 0
j
a .f ( j )
b
(5)

n
We have proved that: a . f ( j ) c j . f (n)
j

(6) b
From equation (5) and (6)
logb n  1 logb n  1
n 
g ( n)  
j 0
j
a .f ( j ) 
b  c . f ( nj
)  c . f (n)
j

j 0 j 0

1
g (n)  f (n)( ) ( f (n))
1 c
(7)
 Master Theorem
Since f(n) appears in definition of g(n) and all
terms of g(n) are non-negative, we can conclude
easily that

g (n) ( f (n)) (8)

We have already proved, equation (7), that

(9)
g (n) ( f (n))

From Equations (8) and (9) g (n) ( f (n))

Hence it proves the lemma
 Master Theorem
Lemma 3 Let a  1, b > 1 be constants, let f(n) be a non-
negative function defined on exact power of b. Define T(n) on
exact powers of b by the recurrence

(1) if n 1
T (n)  n
a.T ( )  f (n) if n b i
 b
where is a positive integer. Then T(n) can be bounded
asymptotically for exact powers of b as
a 
1. If f (n) (n log ), for some constants   0, then T (n) (n log )
a
b b

logb a logb a
2. If f (n) (n ), then T (n) (n . lg n )
3. If f (n) (n log  )
b
a
for some  > 0, and a.f(n/b) ≤ c.f(n) for
some constant c < 1and sufficiently large n, then T(n) =
(f(n))
 Master Theorem
Proof: Case 1
Given that 
(1) if n 1
T (n)  n
a.T ( )  f (n) if n b i
 b
logb n  1
logb a n
By Lemma 4.2: T (n) (n ) 
j 0
j
a .f ( j )
b

By Lemma 4.3: T ( n) ( n logb a )  ( n logb a )

Hence for case 1 it is proved

 Master Theorem
Case 2
Again given that 
(1) if n 1
T (n)  n
a.T ( )  f (n) if n b i
 b
logb n  1
logb a n
By Lemma 4.2: T (n) (n ) 
j 0
j
a .f ( j )
b

By Lemma 4.3: T ( n) ( n logb a )  ( n logb a . lg n)

Hence for case 2 it is also proved

 Master Theorem
Case 3
logb n  1
a n
By Lemma 4.2: T (n) (n logb )  
j 0
j
a .f ( j )
b

By Lemma 4.3: T ( n) ( n logb a

)  ( f (n))

Since f (n) (n logb a 

)

Hence
T (n) ( f (n))
This proves the Lemma 3