0% found this document useful (0 votes)
10 views36 pages

4 Substitution Method and Master Theorem

The document discusses the Substitution Method and the Master Theorem for analyzing the complexity of algorithms. It provides examples of recurrence relations and their solutions, particularly focusing on the mergesort recurrence and other related cases. The Master Theorem is presented as a means to categorize and solve recurrences based on specific parameters.

Uploaded by

farjadkhalil917
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
10 views36 pages

4 Substitution Method and Master Theorem

The document discusses the Substitution Method and the Master Theorem for analyzing the complexity of algorithms. It provides examples of recurrence relations and their solutions, particularly focusing on the mergesort recurrence and other related cases. The Master Theorem is presented as a means to categorize and solve recurrences based on specific parameters.

Uploaded by

farjadkhalil917
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 36

CS 332 - Design and Analysis of Algorithms

Substitution Method
Master Theorem

2
How to determine complexity of code structures
If Statement: Take the complexity of the most expensive case :
char key;
int[][] A = new int[5][5];
int[][] B = new int[5][5];
int[][] C = new int[5][5];
........
if(key == '+') {
for(int i = 0; i < n; i++)
for(int j = 0; j < n; j++) O(n2)
C[i][j] = A[i][j] + B[i][j];
} // End of if block
Overall
complexity
else if(key == 'x')
O(n3)
C = matrixMult(A, B);
O(n3)
else
System.out.println("Error! Enter '+' or 'x'!");
O(1)
1 n=0
T(n)
Solving the Recurrence Relation = T(n) = T(n-1) + 1 n>0

T(n) = T(n-1) + 1 n>0


Substitution Method
Put n=n-1 both Sides
T(n) = T(n-1) + 1 n>0 T(n-1) = T(n-1-1) + 1

T(n-1) = T(n-2) + 1
T(n) = {T(n-2) + 1} + 1
Put n=n-1 both Sides
T(n) = T(n-2) + 2
T(n-1-1) = T(n-1-2) + 1
T(n) = [T(n-3) + 1 ] + 2
T(n-2) = T(n-3) + 1
T(n) = T(n-3) + 3
1 n=0
T(n) =
Solving the Recurrence Relation T(n) = T(n-1) + 1 n>0

Substitution Method

T(n) = T(n-1) + 1 n>0

T(n) = T(n-2) + 2

T(n) = T(n-3) + 3

T(n) = T(n-4) + 4

T(n) = T(n-5) + 5

T(n) = T(n-k) + k
Solving the Recurrence Relation
A time will come k=n
Substitution Method
So we can put k=n
T(n) = T(n-1) + 1 n>0
T(n) = T(n-n) + n
T(n) = T(n-2) + 2
T(n) = T(0) + n
T(n) = T(n-3) + 3

T(n) = T(n-4) + 4

T(n) = T(n-5) + 5

T(n) = 1 + n
T(n) = T(n-k) + k
T(n) = theta(n)
Example
Recurrence Relation
Solution – Recursion TreeT(n)
1 n=0 n T (n-1)
T(n) =
T(n) = T(n-1) + n n>0
n-1 T (n-2)

n-2 T (n-3)

n-k T (n-k-1)

1 T (0)
Recurrence Relation Solution – Recursion
Tree 1 n=0

T(n) n T(n) =
T(n) = T(n-1) + n n>0

n T (n-1) n-1

n-1 T (n-2) n-2

n-2 T (n-3) n-3

n-k T (n-k-1) n-k

1 T (0) 0
Recurrence Relation Solution – Recursion
Tree 1 n=0

T(n) n T(n) =
T(n) = T(n-1) + n n>0

n T (n-1) n-1

n-1 T (n-2) n-2

n-2 T (n-3) n-3

T(n)= 1+2+3+4 . . .+ n-1+ n


n-k T (n-k-1) n-k
T(n)= n(n+1)/2
1
T(n)= Theta (n ) 2

1 T (0) 0
The M a s te r Theorem

The M a s t e r Theorem is a theorem


for asymptotically bounding recurrences of the
type we've seen so far.
Intuitively, categorizes recurrences into one of
the three groups just mentioned, then
determines the runtime based on that
category.
The M a s te r Theorem
T h e o r e m : Let T(n) be defined as follows:
T(1) ≤ Θ (1)
T(n) ≤ aT(⌈n / b⌉) + O (n d )
Then

O (n d ) if log b a <
T(n) = O (n d log d if log b a
n)
O (n
log b a
) = d if log b
a > d
The M a s te r Theorem

• Let a>=1 and b > 1 be constants, let f (n) be a function,


and let T (n) be defined on the nonnegative integers by
the recurrence
T h e o r e m : Let T(n) be defi ned as
follows:
T(1) ≤ Θ(1)
T(n) ≤ aT(⌈n / b⌉) + O (n d )

12
Solving Existing Recurrences
● Consider the mergesort recurrence

T(0) = Θ (1)
T(1) = Θ (1)
T(n) = T(⌈n / 2⌉) + T(⌊n / 2⌋) +
Θ (n)
Solving Existing Recurrences
● Consider the mergesort recurrence

T(0) = Θ (1)
T(1) = Θ (1)
T(n) ≤ 2T(⌈n / 2⌉) + Θ (n)

● What are a, b , and d? a = 2 , b = 2 , d =


● 1 . What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n l o g
n).
Solving Existing Recurrences
● Consider the mergesort recurrence

T(0) = Θ (1)
T(1) = Θ (1)
T(n) ≤ 2T(⌈n / 2⌉) + Θ (n)

● What are a, b , and d? a = 2 , b = 2 , d =


● 1 . What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n l o g
n).
Solving Existing Recurrences
● Consider the mergesort recurrence

T(0) = Θ (1)
T(1) = Θ (1)
T(n) ≤ 2T(⌈n / 2⌉) + Θ (n)

● What are a, b , and d? a = 2 , b = 2 , d =


● 1 . What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n l o g
n).
Solving Existing Recurrences
● Consider the mergesort recurrence

T(0) = Θ (1)
T(1) = Θ (1)
T(n) ≤ 2T(⌈n / 2⌉) + Θ (n)

● What are a, b , and d? a = 2 , b = 2 , d =


● 1 . What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n l o g
n).
Solving Existing Recurrences
● Consider the mergesort recurrence

T(0) = Θ (1)
T(1) = Θ (1)
T(n) ≤ 2T(⌈n / 2⌉) + Θ (n)

● What are a, b , and d? a = 2 , b = 2 , d =


● 1 . What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n l o g
n).
Solving Existing Recurrences
● Consider the mergesort recurrence

T(0) = Θ (1)
T(1) = Θ (1)
T(n) ≤ 2T(⌈n / 2⌉) + Θ (n)

● What are a, b , and d? a = 2 , b = 2 , d =


● 1 . What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n l o g
n).
Solving Existing Recurrences

Consider the weakly unimodal maximum
recurrence:
T(1) ≤ c
T(n) ≤ 2T(n / 2) + c
● What are a, b, d? a = 2 , b = 2 , d = 0
● What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the weakly unimodal maximum
recurrence:
T(1) ≤ c
T(n) ≤ 2T(⌈n / 2⌉) + c
● What are a, b, d? a = 2 , b = 2 , d = 0
● What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the weakly unimodal maximum
recurrence:
T(1) ≤ c
T(n) ≤ 2T(⌈n / 2⌉) + c
● What are a, b, d? a = 2 , b = 2 , d = 0
● What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the weakly unimodal maximum
recurrence:
T(1) ≤ c
T(n) ≤ 2T(⌈n / 2⌉) + c
● What are a, b, d? a = 2 , b = 2 , d = 0
● What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the weakly unimodal maximum
recurrence:
T(1) ≤ c
T(n) ≤ 2T(⌈n / 2⌉) + c
● What are a, b, d? a = 2 , b = 2 , d = 0
● What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the weakly unimodal maximum
recurrence:
T(1) ≤ c
T(n) ≤ 2T(⌈n / 2⌉) + c
● What are a, b, d? a = 2 , b = 2 , d = 0
● What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the weakly unimodal maximum
recurrence:
T(1) ≤ c
T(n) ≤ 2T(⌈n / 2⌉) + c
● What are a, b, d? a = 2 , b = 2 , d = 0
● What is log b a? 1

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the recurrence for the code to
find the maximum value in an array:
T(1) ≤ c
T(n) ≤ T(⌈n / 2⌉) + c n
● What are a, b, d? a = 1 , b = 2 , d = 1
● What is log b a? 0

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the recurrence for the code to
find the maximum value in an array:
T(1) ≤ c
T(n) ≤ T(⌈n / 2⌉) + c n
● What are a, b, d? a = 1 , b = 2 , d = 1
● What is log b a? 0

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the recurrence for the code to
find the maximum value in an array:
T(1) ≤ c
T(n) ≤ T(⌈n / 2⌉) + c n
● What are a, b, d? a = 1 , b = 2 , d = 1
● What is log b a? 0

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the recurrence for the code to
find the maximum value in an array:
T(1) ≤ c
T(n) ≤ T(⌈n / 2⌉) + c n
● What are a, b, d? a = 1 , b = 2 , d = 1
● What is log b a? 0

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the recurrence for the code to
find the maximum value in an array:
T(1) ≤ c
T(n) ≤ T(⌈n / 2⌉) + c n
● What are a, b, d? a = 1 , b = 2 , d = 1
● What is log b a? 0

By the M a s te r Theorem, T(n) = O ( n )
Solving Existing Recurrences

Consider the recurrence for the code to
find the maximum value in an array:
T(1) ≤ c
T(n) ≤ T(⌈n / 2⌉) + c n
● What are a, b, d? a = 1 , b = 2 , d = 1
● What is log b a? 0

By the M a s te r Theorem, T(n) = O ( n )
Proving the M a s te r Theorem

We c a n prove the M a s te r Theorem by
writing out a generic proof using a
recursion tree.
● Draw out the tree.
● Determine the work per level.
● S u m across all levels.

The three cases of the M a s te r Theorem
correspond to whether the recurrence is
topheavy, balanced, or bottomheavy.
Simplifying the Recurrence

The recurrence given by the M a s te r
Theorem is shown here:
T(1) ≤ Θ (1)
T(n) ≤ aT(⌈n / b⌉) +
O (n d )
● We will apply our standard simplifications
to this recurrence:
● Assume inputs are powers of b.
● Replace Θ and O with constant multiples.
T(1) ≤ c
T(n) ≤ aT(n / b) + c n d
Simplifying the Recurrence

The recurrence given by the M a s te r
Theorem is shown here:
T(1) ≤ Θ (1)
T(n) ≤ aT(⌈n / b⌉) +
O (n d )

We will apply our standard simplifications
to this recurrence:
● Assume inputs are powers of b.
● Replace Θ and O with constant multiples.
T(1) ≤ c
T(n) ≤ aT(n / b) + c n d
T(1) ≤ c
T(n) ≤ aT(n / b) + c n d

cn d cn d

c (n / c (n / ... c (n / ac (n /
b)d b)d b)d b)d
c (n / c (n / c (n / a2 c (n /
b 2 )d b 2 )d b 2 )d b 2 )d
c (n / c (n / c (n / a3 c (n /
b 3 )d b 3 )d b 3 )d b 3 )d
...
log b n
c c ... c ca

You might also like