DESIGN & ANALYSIS OF ALGORITHM

(BCSC1012)

Chapter 3: Recurrence Relation

Prof. Anand Singh J a l a l

Department of Computer Engineering & Applications
 Recurrence: Introduction
• A recurrence is an equation or inequality that describes a function in
terms of its value on smaller inputs.
• An algorithm contains a recursive call to itself,
• We can often describe its running time by a recurrence equation or
recurrence.
• Example:
 c n 1

T (n)  
2T    cn n  1
 n
  2 
 Recurrence: Introduction …

Solving Methods
• Iterative Method (back-substitution)
• Master Method
• Recursion Tree
• Substitution Method
 Recurrence: Iterative Method
• Convert the recurrence into a summation and try to bound it
using known series
• Iterate the recurrence until the initial condition is reached.
• Expand the recurrence k times
• Work some algebra to express as a summation
• Use back-substitution (unrolling and summing) to express the
recurrence in terms of n and the initial (boundary) condition.
 Recurrence: Iterative Method
Analysis of Recursive Factorial method
Example1: Form and solve the recurrence relation for the running time of factorial
method and hence determine its big-O complexity:
T(0) = c (1)
T(n) = b + T(n - 1) (2)
= b + b + T(n - 2) by subtituting T(n – 1) in (2)
= b +b +b + T(n - 3) by substituting T(n – 2) in (2)

= kb + T(n - k)
The base case is reached when n – k = 0  k = n,
we then have: long factorial (int n) {
T(n) = nb + T(n - n) if (n == 0)
= bn + T(0) return 1;
= bn + c else
Therefore the Complexitymethod factorial is return n * factorial (n – 1);
O(n) }
 Recurrence: Iterative Method
public int binarySearch (int target, int[] array,
int low, int high) {
if (low > high)
return -1;
else {
Analysis Of int middle = (low + high)/2;
if (array[middle] == target)
Recursive Binary return middle;
Search else if(array[middle] < target)
return binarySearch(target, array, middle + 1, high);
else
return binarySearch(target, array, low, middle - 1);
}
}

The recurrence relation for the running time of the method is:
T(1) = 1 if n = 1 (one element array)
T(n) = T(n / 2) + b if n > 1
 Recurrence: Iterative Method
Without loss of generality, assume n, the problem size, is a multiple of 2, i.e., n = 2k
Expanding:
T(1) = 1 (1)
T(n) = T(n / 2) + b (2)
= [T(n / 22) + b] + b = T (n / 22) + 2b by substituting T(n/2) in (2)
= [T(n / 23) + b] + 2b = T(n / 23) + 3b by substituting T(n/22) in (2)
= ……..
= T( n / 2k) + kb
The base case is reached when n / 2k = 1  n = 2k  k = log2 n,
we then have:
T(n) = T(1) + b log2 n
= 1 + b log2 n
Therefore, Recursive Binary Search is O(log n)
 Recurrence: Iterative Method
 0 n0
T (n)  
c  T (n 1) n  0
• T(n) = c + T(n-1) = c + c + T(n-2)
= 2c + T(n-2) = 2c + c + T(n-3)
= 3c + T(n-3) … kc + T(n-k)
= ck + T(n-k)
• So far for n  k we have
• T(n) = ck + T(n-k)

• To stop the recursion, we should have

• n-k=0  k=n
• T(n) = cn + T(0) = cn Thus in general T(n) = O(n)
 Recurrence: Iterative Method
 0 n0
T (n)  
 n  T (n 1) n  0
• T(n) = n + T(n-1)
= n + n-1 + T(n-2)
= n + n-1 + n-2 + T(n-3)
= n + n-1 + n-2 + n-3 + T(n-4)
=… n

= n + n-1 + n-2 + n-3 + … + (n-k+1) + T(n-k) = i  T (n  k)

ink 1
for n  k
• To stop the recursion, we should have n - k = 0  k = n

n n
n(n 1) n(n 1)
i  T (0)  i  0 
2
T (n) 
2
 O(n 2 )
i 1 i 1
Recurrence: Master Method
 Recurrence: Master Method …
Ex1. T(n) = 9 T(n/3) + n
Compare with standard formula T(n) = a T(n/b) + f(n)
a=9, b=3 and f(n)=n
Compute logb a  log3 9 2 
n n n

It is the form of Case I then T(n)=ѳ(n2)

 Recurrence: Master Method …

Ex2: T(n)=4T(n/2) + n2 [T(n)=aT(n/b) + f(n)]

a=4, b=2, f(n)= n2

nlogba = nlog24 = n2=f(n)

It is the form of Case II then

T(n) = θ (nlogba lg n)
T(n) = θ (nlog24 lg n)
T(n) = θ (n2 lg n)
 Recurrence: Master Method …

Ex3: T(n)=3T(n/2) + n2

a=3, b=2, f(n)= n2

nlogba = nlog23 = n1.5+ = f(n)
where  =0.5

It is the form of Case III then

T(n) = θ (f(n))
T(n) = θ (n2)
 Recurrence: Master Method …

Ex. T(n) = 2T(n/2)+n logn

 Recurrence-Recursion Tree

Steps
• Convert the recurrence into a tree.
• Each node represents the cost of a single sub problem somewhere
in the set of recursive function invocations.
• Sum the costs within each level of the tree to obtain a set of per-
level costs.
• Sum all the per-level costs to determine the total cost of all levels of
the recursion.
Recurrence-Recursion Tree …
 Recurrence-Recursion Tree …
T (n) = 2T(n/2) + c n>1
=1 n=1
c

2c

4c

T(n)= c + 2c + 22c + …………………2kc

The recursion will end when n/2k=1 (2k 1 1) k 1
So T(n)= c(1+2+2 +2 ………2
2 3 k= c  c(2 1)  c(2n 1)  O(n)
2 1
 Recurrence-Recursion Tree …
T (n) = 2T(n/2) + n n>1
=1 n=1
n

T(n)= n + n + n + n …………………upto k
where base case n/2k =1 => n=2k => k=log2n
T(n)= n*k=n*logn
T(n) = O(n lg n)
 Recurrence: Substitution Method

• Guess a bound and then use mathematical induction to

prove our guess correct.
• The substitution method for solving recurrences
comprises two steps:
 Guess the form of the solution.
 Use mathematical induction to find the constants and
show that the solution works.
 Recurrence: Substitution Method
• Substitution Method
T(n)=2T(n/2)+n
Guess the solution is: O(n log n)
Prove that T(n)<= c. (n log n) for c>0
T(n)=2T(n/2)+n………………………………..(1)
Compute for T(n/2)<= c. (n/2).log(n/2) and put into (1)
T(n)<= 2. c. (n/2).log(n/2) +n
T(n)<= c. n. log(n/2) +n
T(n)<= c.n. log n –c.n.log2 +n
T(n)<= c.n.log n –c.n + n for c>=1
 Recurrence: Substitution Method

• Substitution Method
T(n)<= c.n.log n –c.n + n for c=1
T(n)<= 1.n.log n -1.n +n= n.log n
T(n)<= 2. n. log n -2. n +n = n.logn + n.logn –n
.
.
.
T(n)=O(n log n)
Any Questions ?
Dr. Anand Singh Jalal
Professor
Email: [email protected]