Lecture04-Analyzing Recursive Algorithms
Lecture04-Analyzing Recursive Algorithms
Lecture04
By Mohammed Alqmase
Outline
➢ What is Recursion?
➢ How Recursion Works (Call Stack)
➢ How to analyze Recursive algorithms
➢ Recurrence Relation
➢ Recursion Tree
➢ Substitution Methods
➢ Master Theorem
Objectives
➢ To grasp the process of analyzing recursive algorithms.
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
Key Idea of Recursion
✓ Break a problem into smaller sub-problems.
✓ Solve the smallest problem directly (base case).
✓ Combine solutions to sub-problems to solve the original problem.
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
How Recursion Works - Call Stack
Recursion uses a call stack to keep track of function calls.
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
x=factorial(4)
print(x)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
n=4
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
n=3
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else: n=2
return n * factorial(n-1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else: n=2
return n * factorial(n-1) return 2 * factorial(1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0: n=1
return 1
else: n=2
return n * factorial(n-1) return 2 * factorial(1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0: n=1
return 1 return 1 * factorial(0)
else: n=2
return n * factorial(n-1) return 2 * factorial(1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
n=0
def factorial(n):
if n == 0: n=1
return 1 return 1 * factorial(0)
else: n=2
return n * factorial(n-1) return 2 * factorial(1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
n=0
def factorial(n): return 1
if n == 0: n=1
return 1 return 1 * factorial(0)
else: n=2
return n * factorial(n-1) return 2 * factorial(1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
n=0
def factorial(n): return 1
if n == 0: n=1
return 1 return 1 * factorial(0)
else: n=2
return n * factorial(n-1) return 2 * factorial(1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
n=0
def factorial(n): return
if n == 0: n=1
return 1 return 1 *1 1
else: n=2
return n * factorial(n-1) return 2 * factorial(1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0: n=1
return 1 return 1
else: n=2
return n * factorial(n-1) return 2 * factorial(1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0: n=1
return 1 return
else: n=2
return 2 2* 1
return n * factorial(n-1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else: n=2
return 2
return n * factorial(n-1)
n=3
return 3 * factorial(2)
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else: n=2
return n * factorial(n-1) return
n=3
return 3 *6 2
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
n=3
return 6
n=4
return 4 * factorial(3)
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
n=3
return
n=4
return 4 24
* 6
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
n=4
return 24
x=factorial(4)
print(x) x= factorial(4)
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
x=factorial(4)
print(x) x= 24
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
x=factorial(4)
print(x) x= 24
Call Stack
How recursion works
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
x=factorial(4)
print(x)
def test(n):
if n <= 0:
return 1
print(n)
test(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def test(n):
if n <= 0:
return 1
i=0
while i<n: 𝑂 1 𝑛≤0
print(n) 𝑇 𝑛 =ቐ
i=i+1 𝑇 𝑛 − 1 + 𝑂(𝑛) 𝑛>0
test(n-1)
Construct recurrence relations for given recursive algorithms.
def test(n):
if n <= 0:
return 1
print(n)
test(n-1)
test(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def test(n):
if n <= 0:
return 1
print(n)
test(n-1)
test(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
2𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example1(n):
if n <= 1:
return 1
return example1(n-1) + 1
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example2(n):
if n <= 0:
return 1
return 2 * example2(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example4(n):
if n <= 1:
return 1
for i in range(n):
print(i)
return example4(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(𝑛) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example5(n):
if n <= 1:
return 1
for i in range(n):
for j in range(n):
print(i, j)
return example5(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(𝑛2 ) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example7(n):
if n <= 1:
return 1
example7(n-1)
example7(n-2)
example7(n-3)
𝑂 1 𝑛≤1
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + 𝑇 𝑛 − 3 + 𝑂(1) 𝑛>1
Construct recurrence relations for given recursive algorithms.
def example8(n):
if n <= 1:
return 1
for i in range(n):
example8(n-1)
𝑂 1 𝑛≤1
𝑇 𝑛 =ቐ
𝑛 ∗ 𝑇 𝑛 − 1 + 𝑂(1) 𝑛>1
Construct recurrence relations for given recursive algorithms.
def example9(n):
if n <= 1:
return 1
return 2 * example9(n-1) + n
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example10(n):
if n <= 0:
return 1
for i in range(n):
for j in range(n):
example10(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑛2 ∗ 𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example11(n):
if n <= 1:
return 1
example11(n-1)
example11(n-1)
example11(n-1)
𝑂 1 𝑛≤1
𝑇 𝑛 =ቐ
3𝑇 𝑛 − 1 + 𝑂(1) 𝑛>1
Construct recurrence relations for given recursive algorithms.
def example12(n):
if n <= 0:
return 1
return example12(n/2) + 1
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛/2 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example13(n):
if n <= 0:
return 1
return example13(n/2) + example13(n/2) + example13(n/2)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
3𝑇 𝑛/2 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example14(n):
if n <= 0:
return 1
return 2 * example14(n-1) + 2 * example14(n-2)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example15(n):
if n <= 1:
return 1
return example15(n-1) + n**2
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example16(n):
if n <= 1:
return 1
for i in range(n):
for j in range(n):
example16(n-1)
example16(n-2)
𝑂 1 𝑛≤1
𝑇 𝑛 =ቐ
𝑛2 ∗ 𝑇 𝑛 − 1 + 𝑛2 ∗ 𝑇 𝑛 − 2 + 𝑂(1) 𝑛>1
Construct recurrence relations for given recursive algorithms.
def example17(n):
if n <= 1:
return 1
example17(n-1)
example17(n-2)
example17(n-3)
example17(n-4)
𝑂 1 𝑛≤1
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + 𝑇 𝑛 − 3 + 𝑇 𝑛 − 4 + 𝑂(1) 𝑛>1
Construct recurrence relations for given recursive algorithms.
def example18(n):
if n <= 0:
return 1
return example18(n/2) + example18(n/4)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛/2 + 𝑇 𝑛/4 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def example20(n):
if n <= 0:
return 1
example20(n-1)
example20(n-1)
for i in range(n):
example20(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
2𝑇 𝑛 − 1 + 𝑛 ∗ 𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def factorial(n):
if n == 0:
return 1
else:
return n * factorial(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
Construct recurrence relations for given recursive algorithms.
def fibonacci(n):
if n <= 1:
return n
else:
return fibonacci(n-1) + fibonacci(n-2)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + 𝑂(1) 𝑛>0
Methods of analyzing Recursive algorithms
➢ Recursion tree
➢ Substitution method
➢ Master Theorem
Recursion Tree
def test(n):
if n <= 0:
return 1
print(n)
test(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛
𝑇 𝑛−1 𝑂(1)
𝑇 𝑛−𝑛
𝑇 𝑛 = 𝑂 1 + 𝑂 1 + ⋯+ 𝑂 1 (𝑛 𝑡𝑖𝑚𝑒𝑠)
𝑂(1)
=𝑂 𝑛
def test(n):
if n <= 0:
return 1
i=0
while i<n: 𝑂 1 𝑛≤0
print(n) 𝑇 𝑛 =ቐ
i=i+1 𝑇 𝑛 − 1 + 𝑂(𝑛) 𝑛>0
test(n-1)
𝑇 𝑛
𝑇 𝑛−1 𝑛
𝑂 1 𝑛≤0
𝑇 𝑛−2 𝑛−1 𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(𝑛) 𝑛>0
𝑇 𝑛−3 𝑛−2
𝑇 𝑛−𝑛
1 𝑛(𝑛+1)
𝑇 𝑛 = 𝑛 + 𝑛 − 1 + 𝑛 − 2 + ⋯+ 1 = = 𝑂 𝑛2
2
def test(n):
if n <= 0:
return 1
print(n)
test(n-1)
test(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
2𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛
𝑇 𝑛−1
𝑇 𝑛−1 1 20
0 1
𝑇 𝑛 = 2 + 2 + 2 + ⋯+ 2 2 𝑛−1
=𝑂 2 𝑛
𝑇 𝑛−𝑛
2𝑛−1
Substitution Method
def test(n):
if n <= 0:
return 1
print(n)
test(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛 =𝑇 𝑛−1 +1
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛 = 𝑇 𝑛−2 +1 +1
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛 =𝑇 𝑛−2 +1+1
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛 =𝑇 𝑛−2 +2
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛 =𝑇 𝑛−2 +2
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛 =𝑇 𝑛−2 +2
𝑇 𝑛 = 𝑇 𝑛−3 +1 +1
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑇 𝑛 =𝑇 𝑛−1 +1
𝑇 𝑛 =𝑇 𝑛−2 +2
𝑇 𝑛 =𝑇 𝑛−3 +3
.
.
.
𝑇 𝑛 =𝑇 𝑛−𝑛 +𝑛
𝑇 𝑛 =1 +𝑛 𝑻 𝒏 = 𝑶(𝒏)
def test(n):
if n <= 0:
return 1
i=0
while i<n: 𝑂 1 𝑛≤0
print(n) 𝑇 𝑛 =ቐ
i=i+1 𝑇 𝑛 − 1 + 𝑂(𝑛) 𝑛>0
test(n-1)
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(𝑛) 𝑛>0
𝑇 𝑛 =𝑇 𝑛−1 +𝑛 𝑇 𝑛 − 1 = 𝑇 𝑛 − 2 + (𝑛 − 1)
𝑇 𝑛 = 𝑇 𝑛−2 + 𝑛−1 + 𝑛
𝑂 1 𝑛≤0
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(𝑛) 𝑛>0
𝑇 𝑛 =𝑇 𝑛−1 +𝑛 𝑇 𝑛 − 2 = 𝑇 𝑛 − 3 + (𝑛 − 2)
𝑇 𝑛 = 𝑇 𝑛−2 + 𝑛−1 + 𝑛
𝑇 𝑛 =𝑇 𝑛−1 +𝑛 𝑇 𝑛 − 2 = 𝑇 𝑛 − 3 + (𝑛 − 2)
𝑇 𝑛 = 𝑇 𝑛−2 + 𝑛−1 + 𝑛
.
.
.
𝑛(𝑛+1)
𝑇 𝑛 = 𝑇 𝑛 − 𝑛 + 1 + 2 + …+ 𝑛−2 + 𝑛−1 + 𝑛 = = 𝑂 𝑛2
2
Master Theorem
Master Theorem
The Master Theorem is a powerful tool in algorithm design and analysis,
particularly in the context of analyzing recursive algorithms. Master
theorem is used to determine the Big – O upper bound on functions
which possess recurrence. Master Theorem for Decreasing Functions can
be formulated as follow:
𝑐 𝑖𝑓 𝑛 ≤ 0
T 𝒏 =ቐ .
𝑎𝑇 𝒏 − 𝒃 + 𝑂(𝑛𝑘 ), 𝑛>0
Master Theorem
𝑐 𝑖𝑓 𝑛 ≤ 0
T 𝒏 =ቐ .
𝑎𝑇 𝒏 − 𝒃 + 𝑂(𝑛𝑘 ), 𝑛>0
𝒂 > 𝟎, 𝒃 > 𝟎, 𝒌 ≥ 𝟎
𝑘 𝑘=0
𝑪𝒂𝒔𝒆 𝟏: 𝐼𝑓 𝑎<1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛 ) 𝑎=1
𝑏=1
𝑪𝒂𝒔𝒆 𝟐: 𝐼𝑓 𝑎=1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛𝑘+1 )
𝑪𝒂𝒔𝒆 𝟑: 𝐼𝑓 𝑎>1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛𝑘 𝑎𝑛/𝑏 ) 𝑻 𝒏 = 𝑶(𝒏)
Master Theorem
𝑐 𝑖𝑓 𝑛 ≤ 0
T 𝒏 =ቐ .
𝑎𝑇 𝒏 − 𝒃 + 𝑂(𝑛𝑘 ), 𝑛>0
𝑂 1 𝑛≤0
𝒂 > 𝟎, 𝒃 > 𝟎, 𝒌 ≥ 𝟎
𝑇 𝑛 =ቐ
𝑇 𝑛 − 1 + 𝑂(𝑛) 𝑛>0
𝑘 𝑘=1
𝑪𝒂𝒔𝒆 𝟏: 𝐼𝑓 𝑎<1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛 ) 𝑎=1
𝑏=1
𝑪𝒂𝒔𝒆 𝟐: 𝐼𝑓 𝑎=1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛𝑘+1 )
𝑪𝒂𝒔𝒆 𝟑: 𝐼𝑓 𝑎>1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛𝑘 𝑎𝑛/𝑏 ) 𝑻 𝒏 = 𝑶(𝒏𝟐 )
Master Theorem
𝑐 𝑖𝑓 𝑛 ≤ 0
T 𝒏 =ቐ .
𝑎𝑇 𝒏 − 𝒃 + 𝑂(𝑛𝑘 ), 𝑛>0
𝑂 1 𝑛≤0
𝒂 > 𝟎, 𝒃 > 𝟎, 𝒌 ≥ 𝟎 𝑇 𝑛 =ቐ
2𝑇 𝑛 − 1 + 𝑂(1) 𝑛>0
𝑘 𝑘=0
𝑪𝒂𝒔𝒆 𝟏: 𝐼𝑓 𝑎<1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛 ) 𝑎=2
𝑏=1
𝑪𝒂𝒔𝒆 𝟐: 𝐼𝑓 𝑎=1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛𝑘+1 )
𝑪𝒂𝒔𝒆 𝟑: 𝐼𝑓 𝑎>1 𝑡ℎ𝑒𝑛 𝑇(𝑛) = 𝑂(𝑛𝑘 𝑎𝑛/𝑏 ) 𝑻 𝒏 = 𝑶(𝒏𝟎 𝟐𝒏/𝟏 )
𝑻 𝒏 = 𝑶(𝟐𝒏 )