Recurrences and Methods For Solution

4.
Recurrences
Methods for Solving
Recurrences
Computer Theory Lab.
Recurrences --T ( n ) aT ( n / b ) f ( n )
Chapter 4
A recurrence is an equation or inequality that

describes a function in terms of value on
smaller inputs.
P.2
Recurrences and Running

Time
An equation or inequality that describes a

function in terms of its value on smaller inputs.
T(n) = T(n-1) + n
Recurrences arise when an algorithm contains

recursive calls to itself
What is the actual running time of the algorithm?
Need to solve the recurrence
Find an explicit formula of the expression
Bound the recurrence by an expression that involves n
Example Recurrences
T(n) = T(n-1) + n (n2)

Recursive algorithm that loops through the input
to eliminate one item
T(n) = T(n/2) + c (lgn)
Recursive algorithm that halves the input in one
step
T(n) = T(n/2) + n (n)
Recursive algorithm that halves the input but
must examine every item in the input
T(n) = 2T(n/2) + 1
(n)
Recursive algorithm that splits the input into 2
halves and does a constant amount of other work
Recurrences --T ( n ) aT ( n / b ) f ( n )
Chapter 4
Substitution method
Recursion-tree method
Master method
P.5
Technicalities
Chapter 4
We neglect certain technical details when we

state and solve recurrences.
A good example of a detail that is often
glossed over is the assumption of integer
arguments(even or odd) to functions. Omit
floors, ceilings.
Boundary conditions is ignored.
P.6
Methods for Solving

Recurrences
Iteration method
Substitution method
Recursion tree method
Master method
The Iteration Method
Convert the recurrence into a summation

and try to bound it using known series
Iterate the recurrence until the initial

condition is reached.
Use back-substitution to express the

recurrence in terms of n and the initial
(boundary) condition.
The Iteration Method

T(n) = c + T(n/2)
T(n) = c + T(n/2)
T(n/2) = c + T(n/4)
= c + c + T(n/4)
T(n/4) = c + T(n/8)
= c + c + c + T(n/8)
Assume n = 2k
T(n) = c + c + + c + T(1)
= clgn + T(1)
= (lgn)
Iteration Method
Example
T(n) = n + 2T(n/2)
Assume: n = 2k
T(n) = n + 2T(n/2)
T(n/2) = n/2 + 2T(n/4)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
= in + 2iT(n/2i)
= kn + 2kT(1)
10
= nlgn + nT(1) = (nlgn)
k = lgn & n = 2k
4.1 The substitution method

: Mathematical induction
The substitution method for

solving recurrence entails two
steps:
1. Guess the form of the solution.
2. Use mathematical induction to find
the constants and show that the
solution works.
Chapter 4
P.11
Substitution method
Guess a solution
T(n) = O(g(n))
Induction goal: apply the definition of the asymptotic
notation
T(n) c g(n), for some c > 0 and n n0
(strong induction)
Induction hypothesis: T(k) c g(k) for all k < n
Prove the induction goal
Use the induction hypothesis to find some values of the

constants c and n0 for which the induction goal holds
Example: Binary Search

T(n) = c + T(n/2)
Guess: T(n) = O(lgn)
Induction goal: T(n) d lgn, for some d and n n0
Induction hypothesis: T(n/2) d lg(n/2)
Proof of induction goal:

T(n) = T(n/2) + c d lg(n/2) + c
= d lgn d + c d lgn
if: d + c 0, d c
Base case?
Example: Binary Search
Mathematical induction requires our solution to holds for

boundary condition as base cases for inductive proof.
As we know that T(1) = 1

So if n = 1 then,
T(n) <= d log n
T(1) <=d log 1 as log2 1 = 0
T(1) <= 0
1<=0
Consequently, the base case of inductive proof fails to
hold.
Example 2
T(n) = T(n-1) + n
Guess: T(n) = O(n2)
Induction goal: T(n) c n2, for some c and n n0
Induction hypothesis: T(n-1) c(n-1)2 for all k < n

T(n) = T(n-1) + n c (n-1)2 + n
= cn2 (2cn c - n) cn2
if: 2cn c n 0 c n/(2n-1) c 1/(2 1/n)
For n 1 2 1/n 1 any c 1 will work
Example 3
T(n) = 2T(n/2) + n
Guess: T(n) = O(nlgn)
Induction goal: T(n) cn lgn, for some c and n n0
Induction hypothesis: T(n/2) cn/2 lg(n/2)

T(n) = 2T(n/2) + n 2c (n/2)lg(n/2) + n
= cn lgn cn + n cn lgn
if: - cn + n 0 c 1
Base Case ?
as cn log n/2 = cn logn cn log 2
Changing variables
T(n) = 2T( n ) + lgn
Rename: m = lgn n = 2m
T (2m) = 2T(2m/2) + m
Rename: S(m) = T(2m)
S(m) = 2S(m/2) + m S(m) = O(mlgm)

(demonstrated before)
Example
T ( n ) 2T ( n / 2 ) n
T (1 ) 1
(We may omit the initial condition

later.)
T (n) O(n log n)
Guess
Assume
Chapter 4
T ( n / 2 ) c n / 2 log n / 2
P.18
n
T ( n ) 2( c n / 2 log n / 2 ) n cn log n
2
cn log n cn log 2 n cn log n (if c 1.)
Initial condition1 T (1) cn log1 0()

However, 4 T ( 2 ) cn log 2 ( if c 4 )
Chapter 4
P.19
Making a good guess

T ( n ) 2T ( n / 2 17 ) n
We guessT ( n ) O( n log n )
Making guess provides loose upper
bound and lower bound. Then
improve the gap.
Chapter 4
P.20
Subtleties
T ( n ) T ( n / 2 ) T ( n / 2 ) 1
Guess T ( n ) O( n )
AssumeT ( n ) cn
T ( n ) c n / 2 c n / 2 1 cn 1 cn
T ( n ) cn b
However, assume
T ( n ) ( c n / 2 b ) ( c n / 2 b ) 1
cn 2 b 1 cn b ( Choose b 1 )
Chapter 4
P.21
Chapter 4
P.22
Chapter 4
P.23
Avoiding pitfalls
T ( n ) 2T ( n / 2 ) n
T (1 ) 1
AssumeT ( n ) O( n )
Hence T ( n ) cn
T (n) 2(c n / 2 ) n cn n O(n)
(Since c is a constant)
Chapter 4
(WRONG!) You cannot find such a

c.
P.24
The recursion-tree method

Convert the recurrence into a tree:
Each node represents the cost incurred at

various levels of recursion
Sum up the costs of all levels
Used to guess a solution for the recurrence

25
Example 1
W(n) = 2W(n/2) + n2
Subproblem size at level i is: n/2i

Subproblem size hits 1 when 1 = n/2i i = lgn
Cost of the problem at level i = (n/2i)2
No. of nodes at level i = 2i
i
i
lg n 1 2
lg n 1
Total cost:
n
1
1
1
lg n
2
2
2
W (n)
2
i 0
W(n) = O(n2)
2 W (1) n

i 0
nn

i 0
O(n) n
1 1
O ( n) 2n 2
2
26
4.2 the Recursion-tree

method
T (n) 3T ( n / 4 ) (n 2 )
Chapter 4
P.27
Chapter 4
P.28
The cost of the entire tree
Subproblem size at level i is: n/4i

Subproblem size hits 1 when 1 = n/4i i = log4n
Cost of a node at level i = c(n/4i)2
Total cost at level i = 3i c(n/4i)2 = (3/ 16)i cn2
Number of nodes at level i = 3i last level has 3log4n = nlog43 nodes
Total cost:
T (n) cn 2
log 4 n 1
i 0
3 2 3
3
2
cn
cn
...
16
16
16
i
3
log
2
cn
16
log 4 n 1
cn 2 (nlog 3 )
4
(3 / 16)log n 1 2
cn (n log 3 ).
(3 / 16) 1
4
By Equation A.5
T ( n)
log 4 n 1
i 0
i
3
log
2
cn n
16
3
log 3
2

cn
i 0 16
1
cn 2 n log
1 (3 / 16)
16
cn 2 ( n log 3 )
13
4
By Theorem
A.6
O(n 2 )
Chapter 4
P.30
substitution method
We want to Show that T(n) dn2 for some
constant d > 0. using the same constant c > 0
as before, we have
T ( n)
3T ( n / 4 ) cn 2
3d n / 4
cn 2
3d (n / 4) 2 cn 2
3 2
dn cn 2
16
dn 2 ,
Where the last step holds as long as d (16/13)c.

Chapter 4
P.31
T (n) T (n / 3) T (2n / 3) cn
Chapter 4
P.32
substitution method
T (n) T (n / 3) T (2n / 3) cn
d (n / 3) lg(n / 3) d (2n / 3) lg(2n / 3) cn
(d (n / 3) lg n d (n / 3) lg 3) (d (2n / 3) lg n d (2n / 3) lg(3 / 2)) cn
dn lg n d ((n / 3) lg 3 (2n / 3) lg(3 / 2)) cn
dn lg n d ((n / 3) lg 3 (2n / 3) lg 3 (2n / 3) lg 2 cn
dn lg n dn(lg 3 2 / 3) cn
dn lg n,
As long as d c/lg3 (2/3)).
Chapter 4
P.33
Masters method
Cookbook for solving recurrences of the form:
n
T (n) aT f (n)
b
where, a 1, b > 1, and f(n) > 0

This recurrence describes the running time of an algorithm that
divides a problem of size n into a subproblems, each of size n/b.
The function f(n) encompass the cost of dividing of the problem
and combining the result of the subproblems.
Idea: compare f(n) with nlogba, the larger of two function determines the
solution
f(n) is asymptotically smaller or larger than nlogba by a polynomial factor

n
34
Masters method
Cookbook for solving recurrences of the form:
n
T (n) aT f (n)
b
where, a 1, b > 1, and f(n) > 0
Case 1: if f(n) = O(nlogba -) for some > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some > 0, and if
af(n/b) cf(n) for some c < 1 and all sufficiently large n, then:
regularity conditionT(n) = (f(n))
35
Masters method
In case 1 f(n) must be smaller than nlogba. But also be polynomially

smaller. That is f(n) must be asymptotically smaller than nlogba by a
factor of n for some constant > 0.
In case 3 f(n) must be larger than nlogba. But also be polynomially
larger. That is f(n) must be asymptotically larger than nlogba and in
addition satisfy the regularity condition that af(n/b) cf(n).
When f(n) is not polynomally smaller or larger than nlogba. and f(n) is
polynomally smaller or larger but reularity condition fails to hold,
master theorem can not be used to solve the problem.
36
Chapter 4
P.39
Chapter 4
P.40
Examples
T(n) = 2T(n/2) + n
a = 2, b = 2, log22 = 1
Compare nlog22 with f(n) = n
As 1 < n case 3 but it is 1 hence case 2.

f(n) = (n) Case 2
T(n) = (nlgn)
41
Examples
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare n with f(n) = n2
f(n) = (n1+) Case 3 verify regularity cond.
a f(n/b) c f(n)
2 n2/4 c n2 c = is a solution (c<1)
T(n) = (n2)
42
Examples (cont.)
T(n) = 2T(n/2) +
a = 2, b = 2, log22 = 1
Compare n with f(n) = n1/2
f(n) = O(n1-)
Case 1
T(n) = (n)
43
Examples
T(n) = 3T(n/4) + nlgn
a = 3, b = 4, log43 = 0.793
Compare n0.793 with f(n) = nlgn
f(n) = (nlog43+) Case 3
Check regularity condition:
3(n/4)lg(n/4) (3/4)nlgn = c f(n), c=3/4
T(n) = (nlgn)
44
Examples
T(n) = 2T(n/2) + nlgn
a = 2, b = 2, log22 = 1
Compare n with f(n) = nlgn
seems like case 3 should apply
f(n) must be polynomially larger by a factor of n

In this case it is only larger by a factor of lgn
45
Chapter 4
The master method does not apply to

T (n) 2T (n / 2) n lg n,
the recurrence
even though it has the proper form: a
log b a
= 2, b=2, f(n)= n lgn,
n and n. It might
seem that case 3 should apply, since
log b a
f(n)= n lgn is asymptotically larger
n
n.
than
The problem is that it is not
polynomially larger.
P.46

Recurrences and Methods For Solution

Uploaded by

Copyright:

Available Formats

Recurrences and Methods For Solution

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Recurrences and Methods For Solution

Uploaded by

Copyright:

Available Formats

What are recurrences and how do they arise?

What are recurrences and how do they arise?

What are the main methods for solving recurrences?

What are the main methods for solving recurrences?

4.

Computer Theory Lab.

A recurrence is an equation or inequality that

Computer Theory Lab.

Recurrences and Running

An equation or inequality that describes a

Recurrences arise when an algorithm contains

What is the actual running time of the algorithm?

Need to solve the recurrence

Find an explicit formula of the expression

Bound the recurrence by an expression that involves n

Computer Theory Lab.

T(n) = T(n-1) + n (n2)

Computer Theory Lab.

Computer Theory Lab.

We neglect certain technical details when we

Computer Theory Lab.

Methods for Solving

Recursion tree method

Computer Theory Lab.

The Iteration Method

Convert the recurrence into a summation

Iterate the recurrence until the initial

Use back-substitution to express the

Computer Theory Lab.

The Iteration Method

Computer Theory Lab.

Computer Theory Lab.

4.1 The substitution method

The substitution method for

Computer Theory Lab.

T(n) c g(n), for some c > 0 and n n0

Induction hypothesis: T(k) c g(k) for all k < n

Prove the induction goal

Use the induction hypothesis to find some values of the

Computer Theory Lab.

Example: Binary Search

Guess: T(n) = O(lgn)

Induction goal: T(n) d lgn, for some d and n n0

Induction hypothesis: T(n/2) d lg(n/2)

Proof of induction goal:

Computer Theory Lab.

Example: Binary Search

Mathematical induction requires our solution to holds for

As we know that T(1) = 1

Computer Theory Lab.

Guess: T(n) = O(n2)

Induction goal: T(n) c n2, for some c and n n0

Induction hypothesis: T(n-1) c(n-1)2 for all k < n

Proof of induction goal:

For n 1 2 1/n 1 any c 1 will work

Computer Theory Lab.

Guess: T(n) = O(nlgn)

Induction goal: T(n) cn lgn, for some c and n n0