Recurrence Relation

Introduction
} Many algorithms (divide and conquer) are recursive in nature.
} When we analyze them, we get a recurrence relation for time complexity.
} We get running time as a function of 𝒏 (input size) and we get the running time on inputs of
smaller sizes.
} A recurrence is a recursive description of a function, or a description of a function in terms of
itself.
} A recurrence relation recursively defines a sequence where the next term is a function of the
previous terms.
Methods to Solve Recurrence
} Substitution
} Recurrence tree
} Master method
Substitution Method – Example 1
} We make a guess for the solution and then we use mathematical induction to prove the guess
is correct or incorrect.
Time to solve the
Example 1: instance of size 𝑛 − 1

Time to solve the 𝑻(𝒏) = 𝑻(𝒏 − 𝟏) + 𝒏 1

instance of size 𝑛
} Replacing 𝑛 by 𝑛 − 1 and 𝑛 − 2, we can write following equations.

𝑻 𝒏−𝟏 =𝑻 𝒏−𝟐 +𝒏−𝟏 2

𝑻 𝒏−𝟐 =𝑻 𝒏−𝟑 +𝒏−𝟐 3

} Substituting equation 3 in 2 and equation 2 in 1 we have now,

𝑻(𝒏) = 𝑻(𝒏 − 𝟑) + 𝒏 − 𝟐 + 𝒏 − 𝟏 + 𝒏 4
Substitution Method – Example 1
𝑻(𝒏) = 𝑻(𝒏 − 𝟑) + 𝒏 − 𝟐 + 𝒏 − 𝟏 + 𝒏 4

} From above, we can write the general form as,

𝑻 𝒏 = 𝑻 𝒏 − 𝒌 + (𝒏 − 𝒌 + 𝟏) + (𝒏 − 𝒌 + 𝟐) + … + 𝒏

} Suppose, if we take 𝑘 = 𝑛 then,

𝑻 𝒏 = 𝑻 𝒏 − 𝒏 + (𝒏 − 𝒏 + 𝟏) + (𝒏 − 𝒏 + 𝟐) + … + 𝒏

𝑻 𝒏 = 𝟎 + 𝟏 + 𝟐 + …+ 𝒏

𝒏 𝒏+𝟏
𝑻 𝒏 = = 𝑶 𝒏𝟐
𝟐
Substitution Method – Example 2
𝑐1 𝑖𝑓 𝑛 = 0
𝑡 𝑛 =6
𝑐2 + 𝑡 𝑛 − 1 𝑜/𝑤

} Rewrite the equation, 𝑡 𝑛 = 𝑐2 + 𝑡(𝑛 − 1)

} Now, replace 𝐧 by 𝐧 – 𝟏 and 𝐧 − 𝟐
𝑡 𝑛 − 1 = 𝑐2 + 𝑡(𝑛 − 2) ∴ 𝑡 𝑛 − 1 = 𝑐2 + 𝑐2 + 𝑡(𝑛 − 3)
𝑡 𝑛 − 2 = 𝑐2 + 𝑡(𝑛 − 3)
} Substitute the values of 𝐧 – 𝟏 and 𝐧 − 𝟐
𝑡 𝑛 = 𝑐2 + 𝑐2 + 𝑐2 + 𝑡(𝑛 − 3)
} In general,
𝑡 𝑛 = 𝑘𝑐2 + 𝑡(𝑛 − 𝑘)
} Suppose if we take 𝑘 = 𝑛 then,
𝑡 𝑛 = 𝑛𝑐2 + 𝑡 𝑛 − 𝑛 = 𝑛𝑐2 + 𝑡 0
𝑡(𝑛) = 𝑛𝑐2 + 𝑐1 = 𝑶 𝒏
Substitution Method Exercises
} Solve the following recurrences using substitution method.

1 if n = 0 or 1
1. T n = 6
T n − 1 + n − 1 o/w

2. T (n) = T (n − 1) + 1 and T (1) = θ (1).

Recurrence Tree Method
} In recurrence tree, each node represents the cost of a single sub-problem in the set of
recursive function invocations.
} We sum the costs within each level of the tree to obtain a set of per level costs.
} Then we sum the all the per level costs to determine the total cost of all levels of the recursion.
} Here while solving recurrences, we divide the problem into sub-problems of equal size.
} E.g., 𝑇(𝑛) = 𝑎 𝑇(𝑛/𝑏) + 𝑓(𝑛) where 𝑎 > 1 , 𝑏 > 1 and 𝑓(𝑛) is a given function.
} 𝐹(𝑛) is the cost of splitting or combining the sub problems.

𝒏⁄𝒃 𝒏⁄𝒃
 Example 1: 𝑻(𝒏) = 𝟐𝑻(𝒏/𝟐) + 𝒏
Recurrence Tree Method
The recursion tree for this recurrence is § When we add the values across the levels of
the recursion tree, we get a value of 𝑛 for
every level.
𝑛
§ The bottom level has 2&'( ) nodes, each
contributing the cost 𝑇(1).
𝑛⁄2 𝑛⁄2 𝒏
𝒍𝒐𝒈𝟐 𝒏 § We have 𝑛 + 𝑛 + 𝑛 + …… log 𝑛
𝑡𝑖𝑚𝑒𝑠
𝒍𝒐𝒈𝟐 𝒏1𝟏
𝑛⁄23 𝑛⁄23 𝑛⁄23 𝑛⁄23 𝒏
𝑻(𝒏) = T 𝒏 + 𝟐𝒍𝒐𝒈 𝒏 𝑻(𝟏)
𝒊+𝟎

1 1 1 1 𝑻 𝒏 = 𝒏 𝒍𝒐𝒈 𝒏 + 𝒏

𝑻 𝒏 = 𝑶(𝒏 log 𝒏)
 Example 2: 𝑻(𝒏) = 𝑻(𝒏/𝟑) + 𝑻(𝟐𝒏/𝟑) + 𝒏
Recurrence Tree Method
The recursion tree for this recurrence is § When we add the values across the levels of
the recursion tree, we get a value of 𝑛 for
every level.

&'("/$ )16
𝑛
𝑇(𝑛) = T 𝑛 + 𝑛&'("/$ 3𝑇(1)
4+5

𝑛⁄3 2𝑛⁄3 𝒏
𝒍𝒐𝒈𝟑 𝒏 𝒍𝒐𝒈𝟑/𝟐 𝒏 𝑻(𝒏) ∈ 𝒏 log 𝟑/𝟐 𝒏

1𝑛 2𝑛 1 2𝑛 2 2𝑛 𝒏
33 33 3 3 3 3
 Example 3: 𝑻(𝒏) = 𝟐𝑻(𝒏/𝟐) + 𝒄. 𝒏𝟐
Recurrence Tree Method
The recursion tree for this recurrence is § Sub-problem size at level 𝑖 is )⁄3%
) 3
§ Cost of problem at level 𝑖 Is ⁄3%
§ Total cost,
𝒍𝒐𝒈𝟐 𝒏1𝟏 𝒊
𝟏
𝑛3 𝑻 𝒏 ≤ 𝒏𝟐 T
𝟐
𝒊+𝟎

7
𝑛⁄2 %
𝑛⁄2 3 1⁄2 𝑛3 𝟏 𝒊
𝟐
𝑻 𝒏 ≤𝒏 T
𝟐
𝒊+𝟎

𝑛⁄4 3 𝑛⁄4 3 𝑛⁄4 3 𝑛⁄4 3 1⁄4 𝑛3 𝑻 𝒏 ≤ 𝟐𝒏𝟐

𝑻 𝒏 = 𝑶 𝒏𝟐
𝑂 𝑛3
Recurrence Tree Method - Exercises
} Example 1: 𝑇(𝑛) = 𝑇(𝑛/4) + 𝑇(3𝑛/4) + 𝑐. 𝑛
} Example 2: 𝑇(𝑛) = 3𝑇(𝑛/4) + 𝑐. 𝑛2
} Example 3: 𝑇(𝑛) = 𝑇(𝑛/4) + 𝑇(𝑛/2) + 𝑛2
} Example 4: 𝑇(𝑛) = 𝑇(𝑛/3) + 𝑇(2𝑛/3) + 𝑛
Master Theorem
} The master theorem is a cookbook method for solving recurrences.
Time to divide &
} Suppose you have a recurrence of the form recombine
𝑇(𝑛) = 𝑎𝑇(𝑛/𝑏) + 𝑓(𝑛)

Number of sub- Time required to

problems solve a sub-problem

} This recurrence would arise in the analysis of a recursive algorithm.

} When input size 𝒏 is large, the problem is divided up into 𝒂 sub-problems each of size 𝒏/𝒃.
Sub-problems are solved recursively and results are recombined.
} The work to split the problem into sub-problems and recombine the results is 𝒇(𝒏).
Masters Theorem for Dividing Functions

𝑻(𝒏) = 𝒂𝑻(𝒏/𝒃) + 𝒇(𝒏)

where 𝑎 >= 1 and 𝑏 > 1, 𝑓(𝑛) = 𝜃(𝑛+ 𝑙𝑜𝑔, 𝑛)
First find out the value of : 1. 𝑙𝑜𝑔-.
2. 𝑘
Based on the values of 𝑙𝑜𝑔-. and 𝑘, we define 3 cases:
"
Case 1: if 𝑙𝑜𝑔-. > 𝑘, then 𝜃(𝑛 /01!
)
Case 2: if 𝑙𝑜𝑔-. =𝑘,
if p > -1 then 𝜃 𝑛+ 𝑙𝑜𝑔,23 𝑛
if p = -1 then 𝜃 𝑛+ 𝑙𝑜𝑔𝑙𝑜𝑔𝑛
if p < -1 then 𝜃 𝑛+
Case 3: if 𝑙𝑜𝑔-. < 𝑘,
if p>=0, then 𝜃 𝑛+ 𝑙𝑜𝑔, 𝑛
if p<0, then O(𝑛+ )
Example 1 : 𝑻(𝒏) = 𝟐𝑻(𝒏/𝟐) + 𝟏

Given, a = 2, b = 2,
f(n) = 𝜃(1)
= 𝜃 𝑛4 (log 𝑛)4
𝑘 = 0, 𝑝 = 0
𝑙𝑜𝑔55 = 1 > 𝑘 = 0
/01##
Case1 :𝜃(𝑛 ) = 𝜃(𝑛3 )
Example 2: 𝑇(𝑛) = 4𝑇(𝑛/2) + 𝑛

𝑙𝑜𝑔56 = 2 > 𝑘 = 1 , p = 0
therefore case 1: 𝜃(𝑛5 )
Solve Examples:
1. 𝑇(𝑛) = 4𝑇(𝑛/2) + 𝑛
2. 𝑇(𝑛) = 4𝑇(𝑛/2) + 𝑛5
Master Theorem for dividing function

Case 1:

𝑇(𝑛) = 2𝑇(𝑛/2) + 1 𝑂(𝑛)

𝑇(𝑛) = 4𝑇(𝑛/2) + 1 𝑂(𝑛3 )
𝑇(𝑛) = 4𝑇(𝑛/2) + 𝑛 𝑂(𝑛5 )
𝑇(𝑛) = 8𝑇(𝑛/2) + 𝑛5 𝑂(𝑛7 )
𝑇(𝑛) = 16𝑇(𝑛/2) + 𝑛5 𝑂(𝑛6 )
Case 3:

𝑇(𝑛) = 𝑇(𝑛/2) + 𝑛 𝑂(𝑛)

𝑇(𝑛) = 2𝑇(𝑛/2) + 𝑛5 𝑂(𝑛5 )
𝑇(𝑛) = 2𝑇(𝑛/2) + 𝑛5 𝑙𝑜𝑔𝑛 𝑂(𝑛5 𝑙𝑜𝑔𝑛)
Case 2:

𝑇(𝑛) = 𝑇(𝑛/2) + 1 O(𝑙𝑜𝑔𝑛)

𝑇(𝑛) = 2𝑇(𝑛/2) + 𝑛 O(𝑛𝑙𝑜𝑔𝑛)

Solve for:
𝑇(𝑛) = 2𝑇(𝑛/2) + 𝑛𝑙𝑜𝑔𝑛
 Master Theorem For Subtract and Conquer/Decreasing
Recurrences
Let T(n) be a function defined on positive n as shown below:

For some constants c, a>0, b>0, k>=0 and function f(n). If f(n) is O(nk), then
1. If a<1 then T(n) = O(nk)
2. If a=1 then T(n) = O(nk+1)
3. If a>1 then T(n) = O(nkan/b)
Examples :