MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Module 1

Session 1 Characteristics of algorithm

Session 2 Analysis of algorithm - Time and Space
Session 3 Asymptotic analysis of complexity bounds
Session 4 Best, average and worst-case behaviour
Performance measurements of Algorithm: Time and
Session 5
space trade-off
Analysis of recursive algorithms through recurrence
Session 6
relations - Substitution method
Analysis of recursive algorithms through Recursion tree
Session 7
method and Masters’ theorem

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 1

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

SESSION 6: Analysis of Recursive Algorithms through

Recurrence Relations - Substitution Method

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 2

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Time complexity - Recursive algorithms

Recursion - A function calling itself again and again

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 3

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Time complexity - Recursive algorithms

General Procedure for analyzing Recursive Algorithms

• Decide the parameter that represents the input’s size

• Identify the basic operation in the algorithm
• Check whether the number of times the basic operation is executed
depends only on the size of input or on some additional property –
else investigate the worst case, average case, best-case efficiencies
• Set up a recurrence relation with a suitable initial condition for the
number of times basic operation is executed
• Solve the recurrence using backward substitution

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 4

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

A recurrence relation can be solved using the following methods:

• Substitution Method − In this method, we guess a bound and using
mathematical induction we prove that our assumption was correct.
• Iterative Method - It means to expand the recurrence and express it
as a summation of terms of n and initial condition.
• Recursion Tree Method − In this method, a recurrence tree is
formed where each node represents the cost.
• Master’s Theorem − This is another important technique to find the
complexity of a recurrence relation.

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 5

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Substitution method

• The Substitution Method Consists of two main steps:

 Guess the Solution

 Use the mathematical induction to find the boundary

condition and shows that the guess is correct

Example: Solve the equation by Substitution Method.

,
• We have to show that it is asymptotically bound by O (log n)

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 6

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 7

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Iterative method

• It means to expand the recurrence and express it as a summation of

terms of n and initial condition.
T(n)=
T(n) = 2T(n-1), T(n-1) = 2T(n-2)
= 2[2T(n-2)] = 22T(n-2)
= 4[2T(n-3)] = 23T(n-3)
= 8[2T(n-4)] = 24T(n-4)

Repeat the procedure for ‘i’ times

T(n)=2i T(n-i)
Put n-i=1 or i= n-1
T(n) = 2n-1 T(n-(n-1)) = 2n-1 T(1)
= 2n-1.1 (T(1) =1 .....given)
Note: The other 2 methods will be = 2n-1
discussed in the next session

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 8

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Example 1: Factorial of a given value of ‘n’

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 9

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Time complexity - Recursive algorithms

Example 1 : Factorial (Recursive)
 Compute the factorial function F(n) = n! for an arbitrary nonnegative
integer ‘n’.
n! = 1· ... ·(n-l)·n = (n-l)!·n, for n ≥ 1 and 0! = 1
F(n) = F(n- 1) · n

int factorial(int n)
{
if(n==0 || n==1)
return 1;
return (n * factorial(n - 1));
}
• Input size: n
• Basic operation: Multiplication
• M(n): Number of multiplications

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 10

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Time complexity - Recursive algorithms

• The function F(n) is computed based on the formula

F(n) = n. F(n-1), for n > 0
F(0)=1, F(1)=1
• The number of multiplications M(n) needed to compute it must satisfy
the equality
• M(n-1) multiplications are spent to compute F(n-1) and one more
multiplication is needed to multiply the result by ‘n’

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 11

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

factorial(1) 1
factorial(2) 2*factorial(1) 2
factorial(3) 3*factorial(2) 6
factorial(4) 4*factorial(3) 24
factorial(5) 5*factorial(4) 120

int factorial(int n)
{
if(n==0 || n==1)
return 1;
return(n * factorial(n - 1));
}

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 12

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Time complexity - Recursive algorithms

• Initial condition tells the value with which the sequence starts
• Condition that makes the algorithm stop its recursive calls: if n = 0 return 1
• When n = 0, the algorithm performs no multiplications. The initial condition
is M(0)=0
There are 2 recursively defined functions
• The first is the factorial function F(n) itself; it is defined by the recurrence
F(n) = n. F(n-1) for every n > 0, F(0)=1, F(1)=1
• The second is the number of multiplications M(n) needed to compute F(n)
by the recursive algorithm.
M(n) = M(n- 1) + 1 for n > 0, M(0)=0, M(1)=0

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 13

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Time complexity - Recursive algorithms

Solving using Backward Substitution

M(0)=0, M(1)=0
M(n) = M(n-1) + 1 substitute M(n-1) = M(n-2) + 1
= [M(n-2) + 1] + 1 = M(n-2) + 2 substitute M(n-2)= M(n-3) + 1
= [M(n-3) + 1] + 2 = M(n-3) + 3

General formula for the pattern is M(n) = M(n- i) + i

M(n) = M(n-1) + 1 = · · · = M(n-i) + i
n-i=0, Substitute i=n,
M(n) = M(n-n) + n = M(0) + n = n

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 14

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Example 2: Number of Digits in n's Binary Representation

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 15

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Example 2: Find the number of Binary Digits in n's Binary

Representation

• Input size - 1 (a number)

int BinRec(n)
{ • Basic operation - Addition
if(n==1) • A(n) - Number of Additions
return 1; • The number of additions made in
else
return(BinRec(n/2) + 1); computing BinRec, plus one more
} addition is made by the algorithm to
increase the returned value by 1.
• This leads to the recurrence
B(1)=1
A(1)=0

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 16

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

• The recursive calls end when n=1

• There are no additions made
• Initial condition: A(1) = 0
• Backward Substitution cannot be used as n is given by and not a power
of 2.
• Substitute n= and use the theorem called the smoothness rule which
claims that the order of growth observed for n= gives a correct answer for
the order of growth for all values of n.

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 17

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

A=A+1 Substitute A= A+1

= [A + 1] + 1= A + 2
Substitute A = A + 1
= [A + 1] + 2= A + 3
In general,
A=A+i
k-i=0, Substitute i=k
A = A + k = A + k = kn

Mathematically, if n=, then k=

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 18

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

Time complexity - Recursive algorithms

Try yourself

1. How will you find the basic operation in the recursive algorithms?
2. What is the basic operation of the algorithm written for generating the Fibonacci
series?
3. Write the recursive equations for finding the factorial.
4. Identify the basic operation in Towers of Hanoi problem.
5. Write the recursive algorithms and analyse the same using Backward
substitution.
 Finding factorial of ‘n’
 Counting the number of digits in the binary representation of a number

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 19

MODULE 01: SESSION 06 19CS403 - DESIGN AND ANALYSIS OF ALGORITHMS

SESSION 7: Analysis of Recursive Algorithms through

Recursion Tree Method and Masters’ Theorem

RECURSIVE ALGORITHMS - SUBSTITUTION METHOD 20