Data Structure & Algorithms (CS-102)

Recurrence Relation

Subject Coordinator: Dr. Sonal Telang Chandel

Department : Computer Science & Engineering
E-Mail ID: [email protected]
Google Scholar ID: https://scholar.google.com/citations?hl=en&user=pG-SWjIAAAAJ
Scopus ID: 57194155544
Orchid ID: https://orcid.org/0000-0003-4344-1642
 Recurrence
• In an analysis of algorithm recurrence relations are used to analyze
the running time of a recurrence function.

• A recurrence relation (or recurrence) is an equation or inequality that

describes a function in terms of its value on smaller inputs.

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 The running time of a recursive function is denoted by T(n) where n is the size of the input.

In recurrence relation, the running time of a recursive function of input size n is expressed in terms of
the running time of the lower value of n. For example

T(n)=T(n−1)+O(1)
• Here, the running time for size n is equal to the running time for size n−1 plus a constant time.
Usually, the running time T(n) is expressed as a conditional statement having the following two
conditions:
1. Base Case: The base case is the running time of an algorithm when the value of n is small such that
the problem can be solved trivially.

2. Recursive case: The recursive running time is given by a recurrence relation for a large value of n.

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 how to write recurrence relation looking at the code?

• The first thing to look in the code is the base condition and note down the running time of the base
condition. Remember: every recursive function must have a base condition.

• For each recursive call, notice the size of the input passed as a parameter.

• Calculate the running time of operations that are done after the recursion calls.

• Finally, write the recurrence relation.

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Example 1

The base condition happens when the value of n is 0. We can find the value of 0! in constant time O(1)
trivially i.e. for base condition

T(n) = O(1)
Next, we look for the recursive call. There is only one recursive call with input size n−1 and after the
recursive function returns we do a simple multiplication that takes O(1) time i.e. the total running time is

T(n)=T(n−1)+O(1)

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Example 2
The base condition happens when the value of n is 0. We can find the
value of 0! in constant time O(1) trivially i.e. for base condition

1 T(n) = O(1)

n+1

T(n-1)

Total = T(n-1)+n+n+1+1
= T(n-1)+2n+2

= T(n-1)+n

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Example 3

The base condition happens when the value of n is 0. We can find the
value of 0! in constant time O(1) trivially i.e. for base condition
1
T(n) = O(1)
log n +1

log n

T(n-1)

Total = T(n-1)+2 log n+2

= T(n-1) + log n

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 T(n) = 2 T(n-1)+ 1 T(n) = T(n/2)+ 1

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Methods for solving Recurrence
• Substitution Method
We make a guess for the solution and then we use mathematical induction to prove the guess is correct or
incorrect.

• Recurrence Tree Method

In this method, we draw a recurrence tree and calculate the time taken by every level of the tree. Finally, we sum
the work done at all levels. To draw the recurrence tree, we start from the given recurrence and keep drawing till
we find a pattern among levels.

• Master Method
Master Method is a direct way to get the solution. The master method works only for the following type of
recurrences or for recurrences that can be transformed into the following type.

T(n) = aT(n/b) + f(n) where a >= 1 and b > 1

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Substitution Method
• The Substitution Method Consists of two main steps:

Guess the Solution.

Use the mathematical induction to find the boundary condition and shows that the guess is correct.

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Example 1

T(n) = 2T(n/2) + 1
T(n) = 2T(n/2) + 1 = 2[2T(n/4) + 1] + 1 = 4T(n/4) + 2 + 1
T(n) = 4T(n/4) + 3
T(n) = 4T(n/4) + 3 = 4[2T(n/8) + 1] + 3 = 8T(n/8) + 4 + 3
T(n) = 8T(n/8) + 7
T(n) = 8T(n/8) + 7 = 8[2T(n/16) + 1] + 7 = 16T(n/16) + 8 + 7
T(n) = 16T(n/16) + 15
T(n) = 16T(n/16) + 15 = 16[2T(n/32) + 1] + 15 = 32T(n/32) + 16 + 15
T(n) = 32T(n/32) + 31

So now rewrite these five equations and look for a pattern:

T(n) = 2T(n/2) + 1 = 21 T(n/ 21) + 21 -1 1st step of recursion

T(n) = 4T(n/4) + 3 = 22 T(n/ 22) + 22 -1 2nd step of recursion
T(n) = 8T(n/8) + 7 = 23 T(n/ 23) + 23 -1 3rd step of recursion
T(n) = 16T(n/16) + 15 = 24 T(n/ 24) + 24 -1 4th step of recursion
T(n) = 32T(n/32) + 31 = 25 T(n/ 25 ) + 25 -1 5th step of recursion

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 In general, after k iterations, we have:
𝑇(𝑛) = 2𝑘 𝑇 (𝑛/ 2𝑘 )+ 2𝑘 − 1

We’re not done since we still have T(…)’s on the right side of the equation. We need to get down to T(1). How?

We have T(n/ 2𝑘 ), and we want T(1). So let n = 2𝑘 We will then have T(2𝑘 / 2𝑘 ), which equals T(1).

So use that substitution (n = 2𝑘 ) throughout the entire generalized, kth recurrence relation.
𝑇(𝑛) = 2𝑘 𝑇 (𝑛/ 2𝑘 ) + 2𝑘 − 1
= 𝑛 ∗ 𝑇 (2𝑘 / 2𝑘 )+ 𝑛 − 1 = 𝑛 ∗ 𝑇(1) + 𝑛 − 1
𝑇(𝑛) = 𝑛 ∗ 1 + 𝑛 − 1 = 2𝑛 − 1

So, T(n) = 2n – 1 and runs in O(n) time

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Example 2

Solution:
T(n) = 2T(n/2) + n = 2[2T(n/4) + n/2] + n = 4T(n/4) + n + n
T(n) = 4T(n/4) + 2n

T(n) = 4T(n/4) + 2n = 4[2T(n/8) + n/4] + 2n = 8T(n/8) + n + 2n

T(n) = 8T(n/8) + 3n

T(n) = 8T(n/8) + 3n = 8[2T(n/16) + n/8] + 3n = 16T(n/16) + n + 3n

T(n) = 16T(n/16) + 4n

T(n) = 16T(n/16) + 4n = 16[2T(n/32) + n/16] + 4n = 32T(n/32) + n + 4n

T(n) = 32T(n/32) + 5n

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
So now rewrite these five equations and look for a pattern:
T(n) = 2T(n/2) + n = 21 T(n/ 21) + 1n 1st step of recursion
T(n) = 4T(n/4) + 2n = 22 T(n/ 22 ) + 2n 2nd step of recursion
T(n) = 8T(n/8) + 3n = 23 T(n/ 23) + 3n 3rd step of recursion
T(n) = 16T(n/16) + 4n = 24 T(n/ 24 ) + 4n 4th step of recursion
T(n) = 32T(n/32) + 5n = 25 T(n/ 25 ) + 5n 5th step of recursion

Generalized recurrence relation at the kth step of the recursion:

T(n) = 2k T(n/2k ) + kn
We want T(1). So we let n = 2k .
Solving for k, we get k = logn.
T(n) = 2lognT(2k /2k ) + (logn)n = n*T(1) + (logn)n
= n + nlogn

T(n) = n + nlogn runs in O(nlogn) time

 Example 3

Solution: T(n) = T(n-1) + 2

= [T(n-2) + 2] + 2
= T(n-2) + 2 + 2
T(n) = T(n-2) + 2*2

T(n) = T(n-2) + 2*2

= [T(n-3) + 2] + 2*2
= T(n-3) + 2 + 2*2
T(n) = T(n-3) + 2*3

T(n) = T(n-3) + 2*3

= [T(n-4) + 2] + 2*3
= T(n-4) + 2 + 2*3
T(n) = T(n-4) + 2*4

Do it one more time…

T(n) = T(n-4) + 2*4

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 So now rewrite these five equations and look for a pattern:

T(n) = T(n-1) + 2*1 1st step of recursion

T(n) = T(n-2) + 2*2 2nd step of recursion
T(n) = T(n-3) + 2*3 3rd step of recursion
T(n) = T(n-4) + 2*4 4th step of recursion
T(n) = T(n-5) + 2*5 5th step of recursion
Generalized recurrence relation at the kth step of the recursion:
T(n) = T(n-k) + 2*k

We want T(1). So we let n-k = 1. Solving for k, we get k = n - 1.

T(n) = T(n-k) + 2*k

T(n) = T(1) + 2*(n-1), and we know T(1) = 1

T(n) = 2*(n-1) = 2n-1

Right side does not have any T(…)’s. This recurrence relation is now solved
it runs in O(n) time

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Master Theorem

The Master Theorem provides a systematic way of solving recurrence relations of the form:

T(n) = aT(n/b) + f(n)

Where,
n = size of the problem
a = number of subproblems in the recursion and a >= 1
n/b = size of each subproblem
f(n) = cost of work done outside the recursive calls like dividing into subproblems and cost of combining them
to get the solution.

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Continue……..

This theorem is an advance version of master theorem that can be used to determine running time of divide
and conquer algorithms if the recurrence is of the following form :-

Where,
n = size of the problem
a = number of subproblems in the recursion and a >= 1
n/b = size of each subproblem
b > 1, k >= 0 and p is a real number.

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Case 1: if a > bk, then T(n) = θ(nlogba)

Case 2: if a = bk, then

(a) if p > -1, then T(n) = θ(nlogba logp+1n)
(b) if p = -1, then T(n) = θ(nlogba loglogn)
(c) if p < -1, then T(n) = θ(nlogba)

Case 3: if a < bk, then

(a) if p >= 0, then T(n) = θ(nk logpn)
(b) if p < 0, then T(n) = θ(nk)

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Example 1

Here a = 1, b = 2, k = 0 and p = 0
bk = 1. So, a = bk and p > -1 [Case 2.(a)]

T(n) = θ(nlogba logp+1n)

T(n) = θ(logn)

Example 2

a = 3, b = 2, k = 2, p = 0
bk = 4. So, a < bk and p = 0 [Case 3.(a)]

T(n) = θ(nk logpn)

T(n) = θ(n2)

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Practice questions on Master Theorem

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
Recurrence Tree Method
• In this method a recurrence relation is converted into recursive tree.
• To find the total cost , cost of all levels are summed up.
Steps to solve recurrence relation

1. Draw a recursion tree on the given recurrence relation.

2. Determine:
• Cost of each level
• Total number of levels in the recursion tree
• Number of nodes in the last level
• Cost of the last level

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
Example 1
Step 1: Draw the recursion tree
When n= n/2
T(n) = 2T(n/4)+n/2
T(n/20) n Level 0

T(n/21) Level 1
n/2 n/2

T(n/22) n/4 n/4 n/4 n/4 Level 2

T(n/2k) Level k

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 2. Calculate the cost at each level

• Cost at Level-0 = n, two sub-problems are merged.

• Cost at Level-1 = n + n = 2*n, two sub-problems

are merged two times.

• Cost at Level-2 = n + n + n + n = 4*n, two sub-

problems are merged four times. and so on....

• Cost at kth level= kn……………….(1)

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Calculate the Height of the Tree

Since we know that when we continuously divide a number by 2, there comes a time when this number is
reduced to 1. Same as with the problem size N, suppose after K divisions by 2, N becomes equal to 1,
which implies, (n / 2k) = 1
Here n / 2^k is the problem size at the last level and it is always equal to 1.
Now we can easily calculate the value of k from the above expression by taking log() to both sides. Below is
a more clear derivation,
n = 2k
•log(n) = log2 (2k)
•log(n) = k * log2 (2)
•k = log2(n) / log2 (2)
•k = log2 (n) ………………..(2)
So the height of the tree is log (n)
base 2.

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Calculate the number of nodes at each level

Let's first determine the number of nodes in the last level. From the recursion tree, we can deduce
this
Level-0 have 1 (2^0) node

Level-1 have 2 (2^1) nodes

Level-2 have 4 (2^2) nodes

Level-3 have 8 (2^3) nodes

So the level log(n) should

have 2^(log2(n)) nodes i.e. n
nodes.

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Sum up the cost of all the levels

The total cost can be written as,

Total Cost = Cost of all levels except last level + Cost of last level

Total Cost = Cost for level-0 + Cost for level-1 + Cost for level-2 +.... + Cost for level-log(n) + Cost for last
level

The cost of the last level is

n*T(1)= n

Total Cost = (n+n+n+………) + O(n)

for log2n levels

= n log2n +O(n)

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II
 Thank You

Subject Coordinator: Dr. Sonal Chandel Subject Name & Code: DSA (CS-102) Semester: II