Recurrences

Recurrences
Recurrence
A recurrence is an equality or inequality on a function that
uses values of the function on smaller inputs.

Examples:
(
1 if n = 1
T (n) =
T (n − 1) + 1 if n > 1
(
1 if n = 1
T (n) =
2T (bn/2c) + n if n > 1
(
0 if n ≤ 2
T (n) = √
T (b nc) + 1 if n > 2
(
1 if n = 1
T (n) =
T (b2n/3c) + T (bn/3c) + 1 if n > 1

Recurrences
Methods for solving recurrences

The substitution method.

The iteration method.
The master method.

Recurrences
The substitution method

In the substitution method, we guess the answer to the

recurrence and then prove the correctness using induction.

Recurrences
Example 1: Guessing the exact solution
(
1 if n = 1
Recurrence T (n) =
2T (n/2) + n if n > 1
Guess T (n) = n log n + n.
Base For n = 1, n log n + n = 1 = T (1).
Induction step Assume T (m) = m log m + m for all m < n.
T (n) = 2T (n/2) + n
n n n
=2 log + + n by the induction hypothesis
2 2 2
n
= n log + n + n
2
= n log n − n log 2 + n + n
= n log n − n + n + n
= n log n + n

Recurrences
Example 2: Guessing upper and lower bounds
(
1 if n = 1
Recurrence T (n) =
2T (n/2) + n if n > 1
Guess T (n) ≤ cn log n for n ≥ 2.
Base T (2) = 2T (1)+2 = 4 ≤ c ·2 log 2 = 2c ⇒ c ≥ 2.
Induction step Assume T (m) ≤ cm log m for all 2 ≤ m < n.
T (n) = 2T (n/2) + n
 n n
≤ 2 c log +n
2 2
n
= cn log + n
2
= cn log n − cn + n
≤ cn log n if −cn + n ≤ 0 ⇒ c ≥ 1

⇒ T (n) ∈ O(n log n).

Recurrences
Lower bound

Guess T (n) ≥ dn log n for all n.

Base T (1) = 1 ≥ d · 1 log 1 = 0.
Induction step Assume T (m) ≥ dm log m for all 2 ≤ m < n.
T (n) = 2T (n/2) + n
 n n
≥ 2 d log +n
2 2
n
= dn log + n
2
= dn log n − dn + n
≥ dn log n if −dn + n ≤ 0 ⇒ d ≤ 1

⇒ T (n) ∈ Ω(n log n).

SinceT (n) ∈ O(n log n), we conclude that T (n) ∈ Θ(n log n).

Recurrences
Example 3: Revising the guess

Recurrence T (n) = 8T (n/2) + n2 .

Guess T (n) ≤ cn3 .
Induction step
T (n) = 8T (n/2) + n2
 n 3
≤ 8c + n2
2
n3
= 8c · + n2
8
= cn3 + n2
6≤ cn3

Recurrences
Example 3: Revising the guess
Sometimes the guess needs to be revised by subtracting a
lower-order term from the previous guess.
Guess T (n) ≤ cn3 − dn2 .
Induction step
T (n) = 8T (n/2) + n2
    n 2 
n 3
≤8 c −d + n2
2 2
n3 n2
= 8c · − 8d · + n2
8 4
= cn3 − 2dn2 + n2
= cn3 − dn2 − dn2 + n2
≤ cn3 − dn2 if −dn2 + n2 ≤ 0 ⇒ d ≥ 1

Recurrences
The iteration method

Expand the recurrence into a summation.

Evaluate the summation.

Recurrences
Example 1
(
0 if n = 0
Recurrence T (n) =
T (n − 1) + 1 if n > 0
Iteration

T (n) = T (n − 1) + 1


= T (n − 2) + 1 + 1
= T (n − 3) + 3
..
.
= T (0) + n
=n

Recurrences
Example 2
(
1 if n = 1
Recurrence T (n) =
T (n − 1) + n if n > 1
Iteration

T (n) = T (n − 1) + n
= T (n − 2) + (n − 1) + n
= T (n − 3) + (n − 2) + (n − 1) + n
..
.
= 1 + 2 + ··· + n
n(n + 1)
=
2

Recurrences
Changing free terms
If we are only interested in the asymptotic behavior, the free
terms in the recurrence can be simplified.
(
1 if n = 1
T (n) =
T (n − 1) + n if n > 1
(
3 if n = 1
T2 (n) =
T2 (n − 1) + 3n if n > 1
(
1 if n = 1
T3 (n) = √
T3 (n − 1) + n + 2 n if n > 1

T2 (n) = 3T (n) for all n (T2 (n) = 3 + 6 + · · · + 3n).

T (n) ≤ T3 (n) ≤ 3T (n) for all n.

Recurrences
Recurrence tree

T (n) = T (n/3) + T (2n/3) + n.

Recurrences
Recurrence tree

T (n) = T (n/3) + T (2n/3) + n.

Recurrences
Recurrence tree

T (n) = T (n/3) + T (2n/3) + n.

Recurrences
Recurrence tree

T (n) = T (n/3) + T (2n/3) + n.

1 1
· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n =
log 3 log(3/2)
· n log n
⇒ T (n) ∈ Θ(n log n).

Recurrences
The master method

A method for solving recurrences of the form

T (n) = aT (n/b) + f (n)

where a ≥ 1, b > 1, and f (n) is an asymptotic positive

function.

Recurrences
The master theorem
Let T (n) = aT (n/b) + f (n) where a ≥ 1, b > 1 and f (n) is
asymptotically positive.
1 If f (n) ∈ O(nlogb a− ) for some constant  > 0 then
T (n) ∈ Θ(nlogb a ).
2 If f (n) ∈ Θ(nlogb a ) then T (n) ∈ Θ(f (n) · log n).
3 f (n) ∈ Ω(nlogb a+ ) for some constant  > 0 and if
af (n/b) ≤ cf (n) for some constant c < 1 and all
sufficiently large n, then T (n) ∈ Θ(f (n)).

T (n/b) means either bn/bc or dn/be.

Case 2 can be generalized: If f (n) ∈ Θ(nlogb a logk n) for
some k ≥ 0 then T (n) ∈ Θ(f (n) · logk+1 n).

Recurrences
Example 1 (case 1)

T (n) = 9T (n/3) + n.
We have a = 9, b = 3, f (n) = n.
logb a = log3 9 = 2.
All the conditions of the first case are satisfied:
a = 9 ≥ 1, b = 3 > 1, f (n) = n ≥ 0.
For  = 1/2, f (n) ∈ O(nlogb a− ) = O(n3/2 ).
Hence, T (n) ∈ Θ(nlogb a ) = Θ(n2 ).

Recurrences
Example 2 (case 1)

T (n) = 5T (n/2) + n2 .
We have a = 5, b = 2, f (n) = n2 .
logb a = log2 5 ≈ 2.322.
All the conditions of the first case are satisfied:
a = 5 ≥ 1, b = 2 > 1, f (n) = n2 ≥ 0.
For  = 0.3, f (n) ∈ O(nlogb a− ) = O(n2.022 ).
Hence, T (n) ∈ Θ(nlogb a ) = Θ(nlog2 5 ).

Recurrences
Example 3 (case 2)

T (n) = T (2n/3) + 1.
We have a = 1, b = 3/2, f (n) = 1.
logb a = log3/2 1 = 0.
All the conditions of the second case are satisfied:
a = 1 ≥ 1, b = 3/2 > 1, f (n) = 1 ≥ 0.
f (n) ∈ Θ(nlogb a ) = Θ(1).
Hence, T (n) ∈ Θ(f (n) · log n) = Θ(log n).

Recurrences
Example 4 (case 3)

T (n) = 3T (n/4) + n log n.

We have a = 3, b = 4, f (n) = n log n.
logb a = log4 3 ≈ 0.792.
All the conditions of the third case are satisfied:
a = 3 ≥ 1, b = 4 > 1, f (n) = n log n ≥ 0.
For  = 0.1, f (n) ∈ Ω(nlogb a+ ) = Ω(n0.892 ).
For all n, af (n/b) = 3 n4 log n4 ≤ 43 n log n = 43 f (n).
Hence, T (n) ∈ Θ(f (n)) = Θ(n log n).

Recurrences
Example 5

T (n) = 2T (n/2) + n/ log n.

We have a = 2, b = 2, f (n) = n/ log n.
logb a = log2 2 = 1.
/ O(nlogb a− ) for every  > 0.
n/ log n < n, but n/ log n ∈

Recurrences