Appendix B

Solving Recurrence Equations

1
• The analysis of recursive algorithms is not as
straightforward as it is for iterative algorithms.

• it is not difficult to represent the time complexity of a

recursive algorithm by a recurrence equation.

• The recurrence equation must then be solved to

determine the time complexity.

• We discuss techniques for solving such equations and for

using the solutions in the analysis of recursive algorithms.

2
B.1 Solving Recurrences Using Induction
Algorithm B.1 Factorial
• Problem: Determine n! when n ≥ 1., 0! = 1
• Inputs: a nonnegative integer n.
• Outputs: n!.

int fact (int n )

{
if ( n == 0 )
return 1;
else
return n * fact ( n – 1 );
}
3
• tn : the number of multiplications for a given value of n
tn= tn-1+1 , t0= 0 (no multiplication are done when n=0)
• Such equation is called recurrence equation because the
value of the function at n is given in terms of the value of
the function at a smaller value of n.
t1 =1, t2 =t1 +1 =2, t3 =3,
• An inspection of the values just computed indicates that

tn = n

Proof by induction
4
 Example B.1
• Consider the recurrence
tn= tn/2+1 for n>1, n is a power of 2, t1=1
• We use induction to prove tn=log n + 1

Induction base: n=1, t1=log 1 + 1 = 0 +1 =1

Induction step: Assume, for an arbitrary n > 0 and n a power

of 2, that tn=log n + 1
Because the recurrence is only for powers of 2, the next value
to consider after n is 2n.
We need show that t2n= log 2n +1
t2n= tn+1 = log n + 1 +1= log n + log 2 +1= log 2n +1
5
Example B.2
• Consider the recurrence
tn=7 tn/2 for n>1, n is a power of 2
t1=1
t1 = 7 0 ,
t2 =7 t2/2=7 t1=7, t4 =7 t4/2=7 t2= 72,
t8= =7 t8/2=7 t4= 73,
tn= ?
tn=7 log n
Use induction to prove that this is correct.

6
 Induction base: For n = 1,
t1=1, 7log 1 =7 0=1
Induction hypothesis: Assume, for an arbitrary n > 0 and
n a power of 2, that tn=7log n
We need to show that t2n=7log 2n?
t2n=7 tn
=7  7log n = 71  7log n
= 7log 2  7log n
= 7log 2 + log n= 7 log 2n

7
Example:

Consider the recurrence

tn= 2tn/2+n+1 for n>1, n is a power of 2, t1=0

There is no obvious candidate solution suggested by these values. As mentioned

earlier, induction can only verify that a solution is correct. Because we have no
candidate solution, we cannot use induction.

8
B.2 Solving Recurrences Using the Characteristic
Equation
B.2.1 Homogeneous Linear Recurrences
Definition
• A recurrence of the form
a0 tn + a1 tn-1 +…+ ak tn-k =0
where k and the ai terms are constants, is called a homogeneous linear
recurrence equation with constant coefficients:
Linear : every term ti appears only to the first power and : there be no
terms ( ) , where c is a positive constant other than 1. For example,
there may not be terms such as / , ( ) , etc.
Homogeneous: because the linear combination of the terms =0.

9
Homogeneous Linear
Recurrences

10
Example B.4
The following is homogeneous linear recurrence
equations with constant coefficients:
6 tn - 5 tn-1 + 8 tn-2 =0

Example B.5
The Fibonacci sequence is defined as follows:
tn = tn-1 + tn-2 t0 =0, t1 =1
 tn - tn-1 - tn-2 =0
which shows that the Fibonacci sequence is defined by a
homogeneous linear recurrence.

11
Example B.6
• Suppose we have the recurrence
tn - 5tn-1 + 6 tn-2 =0 for n>1, t0 =0, t1 =1

• Notice that if we set tn = r n

r n – 5 r n -1 + 6 r n -2 =0
r n-2 (r 2 – 5 r 1+ 6 )=0
r n-2 ( r-2 )( r-3 )=0
• Solution r =0 or r =2 or r =3
tn =0 or tn = 2 n or tn = 3 n
• We verify that these are solutions

12
• We verify this for 3 n by substituting it into the left
side of the recurrence, as follows:
• 3 n – 5 (3 n -1)+ 6 (3 n -2 )
= 3 n – 5 (3 n -1)+ 2 ( 3 n -1 )
= 3 n – 3 ( 3 n-1 ) =0
• The solution is
tn = c1 3 n + c2 2 n
To find the constants
t0 = 0= c1 + c2
t1 = 1= 3c1 + 2c2
c1 =1 , c2 =-1
tn = 3 n - 2 n
13
Definition
• The characteristic equation for the homogeneous
linear recurrence equation with constant
coefficients
a0 tn + a1 tn-1 +…+ ak tn-k =0

is defined as

a0 r k + a1 r k-1 +…+ ak r 0 =0

14
Example B.7
• The characteristic equation for the recurrence
5tn - 7tn-1 + 6 tn-2 =0
is
5 r 2 – 7 r 1 + 6 =0

15
Theorem B.1
• Consider the homogeneous linear recurrence equation with
constant coefficients
a0 tn + a1 tn-1 +…+ ak tn-k =0
If its characteristic equation
a0 r k + a1 r k-1 +…+ ak r 0 =0
has k distinct solutions r1 r2 … rk , then the only solutions to
the recurrence are

t n  c1r1n  c2 r2n  ...  ck rkn

where the ci terms are constants.
• The constants are determined by the initial conditions.

16
Example B.8
• We solve the recurrence
tn – 3 tn-1 - 4 tn-2 =0 , t0 =0, t1 =1
The characteristic equation
r 2 – 3 r - 4 =0
(r-4)(r+1)=0
• The roots are r = 4 and r = −1.
• Apply Theorem B.1 to get the general solution to the recurrence:
tn = c1 4 n + c2 (-1) n
• Determine the values of the constants ?

c1 =1/5 , c2 = -1/5

17
• See Example B.9, the Fibonacci sequence:

18
• Theorem B.1 requires that all k roots of the characteristic
equation be distinct. The theorem does not allow a
characteristic equation of the following form:
( r-1 )( r-2)3 =0

multiplicity 3
• 2 is called a root of multiplicity 3 of the equation.

19
Theorem B.2
• Let r be a root of multiplicity m of the characteristic
equation for a homogeneous linear recurrence with
constant coefficients. Then
tn = r n , tn = n r n , tn = n2 r n , . . ., tn = nm-1 r n
are all solutions to the recurrence.

20
Example B.10
• We solve the recurrence
tn – 7 tn-1 + 15 tn-2 - 9 tn-3 =0 , for n>2, t0 =0, t1 =1, t2 =2
r3 – 7 r 2 + 15 r -9 =0
( r-1 )( r-3)2 =0
• The roots are r = 1 and r = 3, of multiplicity 2.
• Apply Theorem B.2 to get the solution to the recurrence:
tn = c1 1n + c2 3n + c3 n 3n
• Solving this system yields c1 = −1, c2 = 1, c3=1/3
• See Example B.11

21
B.2.2 Nonhomogeneous Linear Recurrences

22
Definition
• A recurrence of the form
a0 tn + a1 tn-1 +…+ ak tn-k = f (n)

where k and the ai terms are constants and f (n) is a

function other than the zero function, is called a
nonhomogeneous linear recurrence equation with
constant coefficients.

23
• We develop a method for solving the common special case
a0 tn + a1 tn-1 +…+ ak tn-k =bn p(n)
b : constant
p (n) : polynomial in n.

24
Example B.12
• The recurrence
tn – 3 tn-1 = 4n
k = 1, b = 4, and p (n) = 1.

Example B.13
• The recurrence
tn – 3 tn-1 =4n (8n+7 )
k = 1, b = 4, and p (n) = 8n + 7.

25
Example B.14
• We solve the recurrence
tn – 3 tn-1 =4n , t0 =0 , t1 =4
Replace n with n − 1
tn-1 – 3 tn-2 =4n-1 ……. 1

Divide the original recurrence by 4

tn /4 – 3 tn-1 /4=4n-1 …… 2

Subtracting 1 from 2
tn /4 – 7 tn-1 /4 + 3 tn-2 =0

26
• We can multiply by 4
tn – 7 tn-1 + 12 tn-2 =0
• This is a homogeneous linear recurrence equation, which
means that it can be solved by applying Theorem B.1. That is,
we solve the characteristic equation

27
28
Theorem B.3
• A nonhomogeneous linear recurrence of the form
a0 tn + a1 tn-1 +…+ ak tn-k = b n p (n )
can be transformed into a homogeneous linear recurrence that has the
characteristic equation
(a0 r k + a1 r k-1 +…+ ak )(r-b) d+1 =0
where d is the degree of p (n).
• If there is more than one term like b n p (n ) on the right side, each one
contributes a term to the characteristic equation.

29
Example B.15
Solve the recurrence
• tn - 3 tn-1 = 4n (2n+1 ) for n>1
• t0 =0 , t1 =12
1. Obtain the charac. equation for the corresponding homo. equation
r-3=0
2. Obtain a term from the nonhomo. part of the recurrence:
(r-b) d+1 = (r-4) 1+1 = (r-4)2
3. From Theorem B.3: the characteristic equation (r-3) (r-4)2 =0
The roots are r = 3 and r = 4, and the root r = 4 has multiplicity 2.
tn =c1 3 n + c2 4 n + c3 n 4 n

30
 tn =c1 3 n + c2 4 n + c3 n 4 n
• Substituting n=2 (the orginal recurrence equation):
t2 - 3 t1 = 42 (22 +1)
t2 = 3 t1 +80= 116 = c1 3 2 + c2 4 2 + c3 2 4 2
Exercise
you are asked to determine the values c1 ,c2 , and c3
tn =20 (3 n )- 20 ( 4 n )+ 8 n 4 n

31
Example B.16
Solve the recurrence
tn - tn-1 = (n-1 ) for n>0
t0 =0 ??

32
Example B.17
• We solve the recurrence
tn - 2 tn-1 = n+ 2n for n>1
t1 =0
Determine the characteristic equation for the corresponding
homogeneous recurrence
tn - 2 tn-1 =0  r-2=0
This is a case in which there are two terms on the right
n= 1n ( n), 2n = 2n (n0)
(r-1) 1+1 =(r-1) 2 , (r-2) 0+1 =r-2
Apply Theorem B.3 to obtain the characteristic equation from all
the terms:
(r-2) (r-1) 2 (r-2) = (r-2) 2 (r-1) 2

33
B.2.3 Change of Variables (Domain Transformations)
Example B.18
• We solve the recurrenceT(n) = T(n/2) + 1 for n>1 and n power of 2
T(1) = 1
• We can transform it into a recurrence that is in that form as follows.
First, set n=2k
• T(2k) = T(2k-1) + 1
• Next, set tk = T(2k)
tk = tk-1 +1
tk - tk-1 =1 , p(n)=1, b=1, p(n) is of degree 0
(r-1) (r-1)0+1 =0
(r-1)2 =0

34
• Therefore, applying theorem B.3, we can determine its
general solution to be
tk =c1 + c2 k
• Substitute T(2k) for tk in the general solution
T(2k) =c1 + c2 k
T(n) =c1 + c2 log n
T(2) = T(1) + 1=1+1= c1 + c2 log 2= c1 + c2  2 = c1 + c2
T(1) =1= c1 + c2 log 1= c1
c1 = 1, c2=1
T(n) =1 + log n

35
Example B.19
T(n) =2 T(n/2) + n-1 for n>1 and n power of 2
T(1) =0

36
 T(n) =2 T(n/2) + n-1 for n>1 and n power of 2
T(1) =0
• First, set n=2k
• T(2k) = 2T(2k-1) +2k -1
• Next, set tk = T(2k)
tk = 2tk-1 +2k -1
tk - 2tk-1 =2k -1,

37
tk - 2tk-1 =2k -1,
2k : b=2, p(n) =1 is of degree 0
1 : b=1, p(n) =-1 is of degree 0
(r-2)(r-2)0+1 (r-1)0+1 =0
tk = c1 + c2 2k + c3 k 2k
T(2k) = c1 + c2 2k + c3 k 2k
T(n) = c1 + c2 n + c3 n log n

38
Excercise
• solve the recurrence eq
T(n) =7 T(n/2) + 18 (n/2)2 for n>1 and n power of 2, T(1) =0

39
Example B.20
• We solve the recurrence
T(n) =7 T(n/2) + 18 (n/2)2 for n>1 and n power of 2, T(1) =0
• First, set n=2k
• T(2k) = 7T(2k-1) +18( 2k-1)2
• Next, set tk = T(2k)
tk = 7tk-1 +18 22k-2
tk - 7tk-1 =18 22k-2  tk - 7tk-1 =9/2 22k
tk - 7tk-1 =9/2 4k
4k : b=4, p(n) =9/2 is of degree 0
(r-7)(r-4)0+1 =0
T(2k) = c1 7k + c2 4k
T(n) = c1 7log n + c2 4log n
T(n) = c1 nlog 7 + c2 n2

40
B.3 Solving Recurrences by Substitution
• The method of substitution if you cannot obtain a solution
using the methods in the last two sections.
Example B.21
• We solve the recurrence
tn = tn-1 +n, for n>1 , t1 =1
• In a sense, substitution is the opposite of induction. That is, we
start at n and work backward:
tn = tn-1 +n tn = tn-1 +n
tn-1 = tn-2 +n-1 substitute each equality tn = tn-2 + (n-1) +n
tn-2 = tn-3 +n-2 into the previous one, = tn-3 +(n-2)+(n-1)+n
……. as follows: …….
t2 = t1 +2 = t1 +2+ … +n
t1 =1 =1 +2+ … +n
= n(n+1)/2
41
Example B.22
• We solve the recurrence
tn = tn-1 +2/n, for n>1 , t1 =0

For large n, see appendix A

42
B.4 Extending Results to n in General
• Recursive algorithms: we can determine the exact time
complexity only when n is a power of some base b, where b is
a positive constant. Often the base b is 2.
• It seems that a result that holds for n a power of b should
approximately hold for n in general.
• For example, if for some algorithm we establish that
T(n)=2 n log n , for n a power of 2,
it seems that for n in general we should be able to conclude
that T(n) (n log n)

43
Definition
• A complexity function f (n) is called strictly increasing if
n1 > n2  f(n1)> f (n2)

Example
• lg n, n, nlg n, n2, and 2n are all strictly increasing as long as n
is nonnegative.

Definition
• A complexity function f (n) is called nondecreasing if
n1 > n2  f(n1)>= f (n2)

44
45
6

3 Series1

2 Series2

0
0 5 10 15 20 25 30
-1

Log n, floor(log n))

46
• The time (or memory) complexities of most algorithms are
nondecreasing.
Definition
• A complexity function f (n) is called eventually nondecreasing
if there exists an N such that
n1> n2 > N  then f(n1)>= f (n2)

• Any nondecreasing function is eventually nondecreasing

Definition
• A complexity function f(n) is called smooth if
– f (n) eventually nondecreasing and
– and if f(2 n) (f(n))

47
Example B.23
• The functions lg n, n, n lg n, and nk, where
k ≥ 0, are all smooth.

Example
• lg n is eventually nondecreasing.
• As to the second condition, we have
log(2 n)=log 2 + log n (log n )

48
Example B.24
• The function 2n is not smooth,
• By property 4: 2n o(4n )
therefore, 22n = 4n (2n )

49
Theorem B.4
• Let b ≥ 2 be an integer, let f(n) be a smooth complexity
function, and let T (n) be an eventually nondecreasing
complexity function.
If T(n)(f(n) ) for n a power of b
Then T(n)(f(n) ) for any n
Furthermore, the same implication holds if Θ is replaced by
“big O,” Ω, or “small o.”

50
Example B.25
• Suppose for some complexity function we establish that
T(n)=T(n/2) +1, n>1, T(1)=1
• When n is a power of 2, we have
T(n)=1+ log n (log n )
• Because lg n is smooth, we need only show that T (n) is
eventually nondecreasing in order to apply Theorem B.4 to
conclude that T(n) (log n )

• W(n)=W(n/2) + W(n/2) + n-1

51
• T (n) is eventually nondecreasing
• We use induction to establish for n ≥ 2 that

if 1 ≤ k < n, then that T (k )  T (n)

• Induction base: For n = 2, we must show that T (1 )  T (2)

T (1)=1
T (2)= T(2/2) +1=T (1)+1=2
Therefore, T (1 )  T (2)

52
• Induction hypothesis: Let n >=2 .
• Assume for all k ≤ n that T (k)  T (n)
• Induction step: we need only show that T (n)  T (n+1)
• It is not hard to see that if n ≥ 1, then n/2  (n+1)/2
Therefore, by the induction hypothesis,
T ( n/2 )  T ( (n+1)/2 )
Using the recurrence, we have
T (n)= T ( n/2 ) +1  T ( (n+1)/2 ) +1= T (n+1)

53
Theorem B.5
Suppose a complexity function T (n) is eventually nondecreasing
and satisfies

then Result B.5 holds with “big O” or Ω, respectively, replacing Θ. 54

Example B.26
• Suppose that T (n) is eventually nondecreasing and satisfies
T (n)= 8 T ( n/4 ) +5 n2 for n>1 a power of 4
T (1)=3
By Theorem B.5, a=8, b=4,k=2,because 8 < 42, T (n) (n2 )

55
Example B.27
• Suppose that T (n) is eventually nondecreasing and satisfies
T (n)= 9 T ( n/3 ) +5 n1 for n>1 a power of 3
T (1)=7
By Theorem B.5, because 9 > 31, T (n) (nlog3 9 ) = (n2 )

56
Theorem B.6
• Suppose that a complexity function T (n) is eventually
nondecreasing and satisfies

• where s is a constant that is a power of b, b ≥ 2 and k ≥ 0 are

constant integers, and a, c, and d are constants such that a > 0, c >
0, and d ≥ 0. Then the results in Theorem B.5 still hold.

57
Example B.28
• Suppose that T (n) is eventually nondecreasing and satisfies
T (n)= 8 T ( n/2 ) +5 n3 for n>64 a power of 2
T (64)=200
By Theorem B.6, because b=2, a=8 = 23,k=3, bk =23,
T (n) (n3 log n )

58