1.

6 MATHEMATICAL ANALYSIS OF RECURSIVE ALGORITHMS

Solving recurrence relations

1. Substitution Method - This method repeatedly makes substitution for each
occurrence of the function T in the right-hand side until all such occurrences
disappear.

2. Master Theorem - The efficiency analysis of many

divide-and-conquer algorithms is greatly simplified by
the master theorem. It states that, in recurrence
equation T(n) = aT(n/b) + f (n), If f (n)∈ Θ (nd) where d ≥ 0
then

3. Recursion Tree Method - While substitution method works well for many recurrence
relations, it is not a suitable technique for recurrence relations that model divide
and conquer paradigm based algorithms. Recursion Tree Method is a popular
technique for solving such recurrence relations, in particular for solving
unbalanced recurrence relations. For example, in case of modified merge Sort, to
solve a problem of size n (to sort an array of size n), the problem is divided into
two problems of size n/3 and 2n/3 each. This is done recursively until the
problem size becomes 1. Consider the following recurrence relation. 1. T(n) =
2T(n/2) + 1 Here the number of leaves = 2log n = n and the sum of effort in each
level except leaves is
Master Theorem

T(n) = aT(n/b) + f(n),

where,
n = size of input
a = number of subproblems in the recursion
n/b = size of each subproblem. All subproblems are assumed
to have the same size.
f(n) = cost of the work done outside the recursive call,
which includes the cost of dividing the problem and
 cost of merging the solutions

Here, a ≥ 1 and b > 1 are constants, and f(n) is an

asymptotically positive function.

If a ≥ 1 and b > 1 are constants and f(n) is an asymptotically

positive function, then the time complexity of a
recursive relation is given by

Example:

Let T(n) = 2T(n/2) + 1, solve using master theorem.

Solution:

Here: a = 2,b = 2,d = 0

f(n) = Θ(1)

Therefore:
a > bd i.e., 2 > 20
Case 3 of master theorem holds good. Therefore:
T(n) Є Θ (nlogba )
Є Θ (nlog22 )
Є Θ (n)
1.6 MATHEMATICAL ANALYSIS OF RECURSIVE ALGORITHMS

General plan:

1. Decide the input size based on the parameter n.

2. Identify algorithm’s basic operation.
3. Check how many times the basic operation is executed. Then find whether the
execution of basic operation depends upon the input size n. Determine worst, best and
average case for input of size n. If the basic operation depends upon those three cases
then that has to be analyzed separately.
4. Setup the recurrence relation with some initial condition and expressing the
basic operation.
5. Solve the recurrence or at least determine the order of growth. While solving
the recurrence we will use the forward and backward substitution method. And then
correctness of formula can be proved with the help of mathematical induction method.
Let us analyze some recursive algorithm mathematically.
Example 1:
Computing factorial of some n numbers:

• The factorial of some numbers can be obtained by performing repeated multiplication.

• For instance n=5, then
Step 1: n! = 5!
Step 2: 4!*5
Step 3: 3! * 4! * 5
Step 4: 2! * 3! * 4! * 5
Step 5: 1! * 2! * 3! * 4! * 5
Step 6: 0! * 1! * 2! * 3! * 4! * 5
Step 7: 1 * 2 * 3 * 4 * 5
Pseudocode:
If (n=0)
Return 1
Else
Return Factorial (n-1)*n
Mathematical Analysis:
1. The factorial algorithm works for input size n.
2. The basic operation in computing factorial is multiplication.
3. The recursive function can be formulated as F(n) = F(n-1) * n where n >0.
Then basic operation multiplication is given as M(n).
M(n) = M(n-1) + 1
M(n-1) → these multiplications are required to compute factorial
(n-1)
+1 → to multiply factorial (n-1) by n.
4. In step 3 the recurrence relation is obtained. M(n) = M(n-1) + 1.
Now we will solve recurrence using
Forward Substitution:
M(n) = M(n-1)+1, with M(0)=0.
M(1) = M(0) + 1 =1
M(2) = M(1)+1 = 1+1=2
M(3) = M(2)+1 = 2+1=3
Backward Substitution:
M(n)= M(n-1)+1
=[M(n-2)+1]+1 = M(n-2)+2
=[M(n-3)+2]+1 = M(n-3)+3
From the substitution methods we can establish a general formula as:
M(n)= M(n-i) + i.
Now let us prove the correctness of this formula using mathematical induction as follows:
Prove M(n) = n by using mathematical induction.
Basis: Let n=0 then
M(n) = 0
M(0) = 0 =n.

Thus, the recurrence relation and initial condition for the algorithm’s number of
multiplications

M(n):
M(n) = M(n − 1) + 1 for n > 0,
M(0) = 0 for n = 0.

Method of backward substitutions

M(n) = M(n − 1) + 1 substitute M(n − 1) = M(n − 2) + 1
= [M(n − 2) + 1]+ 1
= M(n − 2) + 2 substitute M(n − 2) = M(n − 3) + 1
= [M(n − 3) + 1]+ 2
= M(n − 3) + 3
…
= M(n − i) + i

= M(n − n) + n
= n.
 Therefore M(n)=n
Thus the time complexity of factorial function is Θ(n).

Example 2:
Towers of Hanoi

• Classic example of recursive function.

• There are 3 pegs named A, B and C. 5 disks of different sizes are placed on peg A. The
arrangements of the disks is such that every smaller disk is placed on the larger disk. •
The Problem of “Towers of Hanoi” states that move the 5 disks from peg A to peg C
using peg B as an auxiliary.

Fig : Recursive solution to the Tower of Hanoi puzzle.

• The given conditions are:
1. Only the top disk on any peg may be moved to any other peg.
2. A larger disk can never rest on the smaller one.
• Solution for the given problem is as follows:
1. Move disk 1 from A to B
2. Move disk 2 from A to C
3. Move disk 1 from B to C
4. Move disk 3 from A to B
5. Move disk 1 from C to A
6. Move disk 2 from C to B
7. Move disk 1 from A to B
8. Move disk 4 from A to C
9. Move disk 1 from B to C
10. Move disk 2 from B to A
11. Move disk 1 from C to A
12. Move disk 3 from B to C
13. Move disk 1 from A to B
14. Move disk 2 from A to C
15. Move disk 1 from B to C.
Thus actually we have moved n-1 disk from peg A to C.
Pseudocode:
If (n=1) then
{
Write (“The peg moved from A to C”)
Return
}
Else
{
// move top n-1 disk from A to B using C
TOH(n-1,A,B,C);
// move remaining disk from B to C using A
TOH(n-1, B,C,A)
}
Mathematical Analysis:
Step 1: The input size is n.
Step 2: The basic operation in this problem is moving the disk from one peg to another.
When n>1, then to move these disks from peg A to peg C using auxiliary peg B, we first
move recursively n-1 disks from peg A to peg B using auxiliary peg C. Then we move
the largest disk directly from peg A to peg C and finally move n-1 disks from peg B to
peg C.
If n=1, then we simply move the disk from peg A to peg C.
Step 3: The move of disks are denoted by M(n). M(n) depends on the number of disks n.
The recursive relation can then setup as M(1)=1.
If n>1 then we need two recursive calls plus one move. Hence
M(n) = M(n-1) + 1 + M(n-1)

Here M(n-1) → to move (n-1) disks from peg A to B.

1 → To move largest disk from peg A to C.

M(n-1) → to move (n-1) disks from peg B to C.

Therefore M(n) = 2M(n-1) + 1 →(equ 1)

Step 4: solving recurrence M(n) = 2M(n-1) + 1 using two substitution methods: Forward
Substitution:
For n > 1,
M(2) = 2M(1) + 1
= 2*1+1
M(2) = 3
M(3) = 2M(2) + 1
= 2*3+1
M(3) = 7
M(4) = 2M(3) + 1
=2 * 7 + 1
M(4) = 15
Backward Substitution:
M(n) = 2M(n-1) + 1
Put M(n-1) = 2M(n-2)+1
=2[2M(n-2)+1]+1
=4M(n-2)+3
Put M(n-2) = 2M(n-3)+1
=4[2M(n-3)+1]+3
=8M(n-3)+7

• From this we can establish a general formula as:

M(n) = 2i M(n-i) + 2i-1 + 2i-2 +…… + 2 +1

This can also be written as:

M(n) = 2i M(n-i) + 2i– 1 → (equ 2)

• Thus for obtaining M(n) we substitute n by n- i in the equation 2.

• Let us use mathematical induction to establish correctness of equation 2. Basis:
As in equation 2 we can obtain M(n) by substituting n = n – i , assume initially n=1 then,
n–i=1
i = n-1
M(n) = 2i M(n-i) + 2i-1 with i=n-1 can become
= 2n-1 M(n-(n-1))+ 2n-1– 1
= 2n-1 M(1) + 2n-1– 1 put M(1)=1
= 2n-1 + 2n-1-1
M(n) = 2n– 1
If n=1 then,
M(1) = 2-1=1 is proved.
Induction:
Substitute M(n-1) = 2n-1– 1 in M(n) = 2M(n-1) +1
M(n) = 2(2n-1-1) +1
M(n) = 2n-2+1
M(n) = 2n-1
Put n=1
M(1) = 2-1 = 1 is proved
Therefore we get recurrence as M(n) = 2n-1
From this we can conclude the time complexity of the Towers of Hanoi problem as Θ(2n-
1) = Θ(2n).

EXAMPLE 3: An investigation of a recursive version of the algorithm which finds

the number of binary digits in the binary representation of a positive decimal integer.
ALGORITHM BinRec(n)
//Input: A positive decimal integer n

//Output: The number of binary digits in n’s binary representation if n = 1 return 1 /

else return BinRec( n 2 )+ 1
Algorithm Analysis

Since the recursive calls end when n is equal to 1 and there are no additions made then,
the initial condition is A(1) = 0.
The standard approach to solving such a recurrence is to solve it only for n
= 2k A(2k) = A(2 k−1) + 1 for k > 0,
A(20) = 0.

Backward substitutions

A(2k) = A(2k−1) + 1

substitute A(2k−1) = A(2k−2) + 1

= [A(2k−2) + 1]+ 1

= A(2k−2) + 2

substitute A(2k−2) = A(2k−3) + 1

= [A(2k−3) + 1]+ 2

= A(2k−3) + 3
...

...

= A(2k−i) + i

...

= A(2k−k) + k.
 Thus, we end up with A(2 k) = A(1) + k = k, or, after returning to the original variable n =
2k and hence k = log2 n,
A(n) = log2 n ϵ Θ (log2 n).

Example 4 : Fibonacci Numbers Sequence

A classic example for more elaborate recursion relations.
Fibonacci Sequence: 0, 1, 1 2, 3, 5, 8, 13, 21, 34, ...
Fibonacci (1202) proposed for the growth of rabbits
Can be defined by the simple recurrence
F(n) = F(n-1) + F(n-2) for n > 1
IC: F(0) = 0 and F(1) = 1 (why do we need two ICs)
Homogenous second-order linear recurrence with constant coefficients
ax(n) + bx(n-1) + cx(n-2) = 0
Homogenous because the recurrence equals zero.
Why second-order linear? Substitute the proposed solution x(n) = rn
a rn + b rn-1 + c rn-2 = 0
divide by rn
a + b r + c r2 = 0, characteristic equation is a second order polynomial.
The real roots are solutions
x(n) = αr1n + βr2n
α and β are determined from the

Apply to Fibonacci Recursion

F(n) - F(n-1) - F(n-2) = 0, homogenous second order with constant coefficients

r2 - r - 1 = 0, characteristic equation
r1,2 = (1 ± √5)/2 = φ or φ' { φ = (1+√5)/2 and φ' = (1-√5)/2}
The general form of the solution
F(n) = αφn + βφ' n where α and β are unknows
Using the ICs
α + β =0
φα + φ'β = 1
Solve by substituting the first equation into the second, get α = 1/√5 and β = -1/√5
So
F(n) = 1/√5 (φn - φ' n) = φn/√5 rounded to the nearest integer

Example: Recursive Algorithm for Fibonacci Numbers

Algorithm F(n)
if n ≤ 1 then return n
else return F(n-1) + F(n-2)

1. Problem size is n, the sequence number for the Fibonacci number

2. Basic operation is the sum in recursive call
3. No difference between worst and best case
4. Recurrence relation
A(n) = A(n-1) + A(n-2) + 1
IC: A(0) = A(1) = 0
or
A(n) - A(n-1) - A(n-2) = 1, Inhomogeneous recurrences because of the 1

In general solution to the inhomogeneous problem is equal to the sum of solution to

homogenous problem plus solution only to the inhomogeneous part. The
undetermined coefficients of the solution for the homogenous problem are used to
 satisfy the IC.

In this case A(n) = B(n) + I(n) where

A(n) is solution to complete inhomogeneous problem
B(n) is solution to homogeneous problem
I(n) solution to only the inhomogeneous part of the problem

We guess at I(n) and then determine the new IC for the homogenous problem for B(n)

For this problem the correct guess is I(n) = 1

substitute A(n) = B(n) -1 into the recursion and get

B(n) - B(n-1) - B(n-2) = 0 with IC B(0) = B(1) = 1

The same as the relation for F(n) with different IC

We do not really need the exact solution; We can conclude
A(n) = B(n) -1 = F(n+1) - 1 ε Θ(φn), exponential

There is the Master Theorem that give the asymptotic limit for many common problems.

Iterative algorithm
Algorithm Fib(n)
F[1] ← 0; F[1] ← 1

for i ← 2 to n do

F[i] ← F[i-1] + F[i-2]

return F[n]
Order of growth is Θ(n).