Recall: Recursively Defined Sequences

CS311H: Discrete Mathematics

I In previous lectures, we looked at recursively-defined sequences

Recurrence Relations I Example: What sequence is this?

a0 = 1
Instructor: Işıl Dillig
a1 = 1
an = an − 1 + an−2

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 1/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 2/23

Recurrence Relations Closed Form Solutions

I Often, we need to find a closed form solution for a given

I Recurively defined sequences are often referred to as
recurrence
recurrence relations
I Recall: Closed form solution defines n’th number in the
I The base cases in the recursive definition are called initial
sequence as a function of n
values of the recurrence relation
I What is closed form solution to the following recurrence?
I Example: Write recurrence relation representing number of
bacteria in n’th hour if colony starts with 5 bacteria and a0 = 0
doubles every hour? an = an−1 + n

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 3/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 4/23

Closed Form Solutions of Recurrence Relations Examples and Non-Examples

I Given an arbitrary recurrence relation, is there a mechanical I Which of these are linear homogenous recurrence relations
way to obtain the closed form solution? with constant coefficients?
I Not for arbitrary, but for a subclass of recurrence relations I an = an−1 + 2an−5

I A linear homogeneous recurrence relation with constant I an = 2an−2 + 5

coefficients is a recurrence relation of the form:
I an = an−1 + n
an = c1 an−1 + c2 an−2 + . . . + ck an−k
I an = an−1 · an−2
where each ci is a constant and ck is non-zero
I an = n · an−1
I The value of k is called the degree of the recurrence relation

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 5/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 6/23

1
Characteristic Polynomial Characteristic Equation Examples

I Cook-book recipe for solving linear homogenous recurrence

I What are the characteristic equations for the following
relations with constant coefficients recurrence relations?
I Definition: The characteristic equation of a recurrence relation
I fn = fn−1 + fn−2
of the form an = c1 an−1 + c2 an−2 + . . . ck an−k is
I an = 2an−1
r k = c1 r k −1 + c2 r k −2 + . . . + ck
I an = 2an−1 + 5an−3
I i.e., replace ai with r i−(n−k )

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 7/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 8/23

Characteristic Roots Theorem I for Solving Linear Homogenous Recurrence

Relations

Let an = c1 an−1 + c2 an−2 + . . . + ck an−k be a recurrence relation

I The characteristic roots of a linear homogeneous recurrence with k distinct characteristic roots r1 , . . . , rk .
relation are the roots of its characteristic equation.

I What are the characteristic roots of the following recurrence I Then the closed form solution for an is of the form:
relations?
α1 r1n + α2 r2n + . . . + αk rkn
I an = 2an−1 + 3an−2
I Furthermore, given k initial conditions, the constants
I fn = fn−1 + fn−2 α1 , . . . , αk are uniquely determined

I Note: Won’t do the proof because requires a good amount of

linear algebra

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 9/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 10/23

Example Generalized Theorem

I So far, we assume all characteristic roots are distinct – what

Find a closed form solution for the recurrence an = an−1 + 2an−2
happens if this is not the case?
with initial conditions a0 = 2 and a1 = 7

I Characteristic equation: I Theorem: Let an = c1 an−1 + c2 an−2 + . . . + ck an−k be a

recurrence relation with t distinct characteristic roots
I Characteristic roots: r1 , . . . , rk with multiplicities m1 , . . . , mk . Then solutions are
of the form:
I Coefficients: t
X
an = (αi,0 + αi,1 · n + . . . + αi,mi−1 · n mi −1 )rin
I Closed-form solution: i=0

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 11/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 12/23

2
An Example Solving Linear Non-Homogeneous Recurrence Relations

I How do we solve linear, but non-homogeneous recurrence

I Find closed form of recurrence an = 3an−1 − 3an−2 + an−3 relations, such as an = 2an−1 + 1?
with initial conditions a0 = 1, a1 = 3, a2 = 7
I A linear non-homogeneous recurrence relation with constant
I Characteristic equation: coefficients is of the form:

I Characteristic roots: an = c1 an−1 + a2 an−2 + . . . + ck an−k + F (n)

I Solution form: I The recurrence obtained by dropping F (n) is called the

associated homogeneous recurrence relation
I Coefficients:
I To solve these recurrences, we will combine the solution for
the homogenous recurrence with particular solution

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 13/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 14/23

Particular Solution Theorem about Linear Non-homogeneous Recurrences

I A particular solution for a recurrence relation is one that

satisfies the recurrence but not necessarily the initial
conditions Suppose an = c1 an−1 + . . . + ck an−k + F (n) has particular
solution anp , and anh is solution for associated homogeneous
I Example: Consider the recurrence an = an−1 + 1 with initial recurrence. Then every solution is of the form anp + anh .
condition a0 = 5

I A particular solution for this recurrence is an = n, but it does

not satisfy the initial condition

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 15/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 16/23

Why is this theorem useful? How do we find a particular solution?

Theorem: Consider an = c1 an−1 + . . . + ck an−k + F (n) where:
I If we can find a particular solution, then we can also
mechanically find a solution that satisfies initial conditions. F (n) = (bt n t + bt−1 n t−1 + . . . + b1 n + b0 )s n

I Example: Solve the recurrence relation an = 3an−1 + 2n with

initial condition a1 = 3 I Case 1: If s is not a root of the associated characteristic
equation, then there exists a particular solution of the form:
I A particular solution: −n − 3
(Why?)
2
(pt n t + pt−1 n t−1 + . . . + p1 n + p0 )s n
I Solutions for homogeneous recurrence:
I Case 2: If s is a root with multiplicity m of the characteristic
I Solutions for recurrence: equation, then there exists a solution of the form:

n m (pt n t + pt−1 n t−1 + . . . + p1 n + p0 )s n

I Solve for α:

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 17/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 18/23

3
Example I Example II
I Consider again the recurrence an = 3an−1 + 2n

I Here, s = 1 and characteristic root is 3 I Find a particular solution for an = 6an−1 − 9an−2 + 2n

I Hence, there exists a particular solution of the form p1 n + p0 I Characteristic root:

I Now, solve for p0 , p1 : I Particular solution of the form:

p1 n + p0 = 3(p1 (n − 1) + p0 ) + 2n I Find p0 such that p0 · 2n = 6(p0 · 2n−1 ) − 9(p0 · 2n−2 ) + 2n

I Rearrange: 2n(p1 + 1) + (2p0 − 3p1 ) = 0 I Solve for p0 :

I A solution p1 = −1, p0 = − 23 I Particular solution:

I A particular solution: −n − 3
2

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 19/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 20/23

Towers of Hanoi A Recursive Solution

I Solve recursively – Tn is number of steps to move n disks

I Base case: n = 1, move disk from first peg to second: T1 = 1

I Induction: Suppose we can move n − 1 disks in Tn−1 steps;

I Given 3 pegs where first peg contains n disks how many steps does it take to move Tn disks?

I Goal: Move all the disks to a different peg (e.g., second one) I Idea: First move the topmost n − 1 disks to peg 3; can be
done in Tn−1 steps
I Rule 1: Larger disks cannot rest on top of smaller disks
I Now, move bottom-most disk to peg 2 – takes just 1 step
I Rule 2: Can only move the top-most disk at a time
I Finally, recursively move n − 1 disks in peg 3 to peg 2 – can
I Question: How many steps does it take to move all n disks? be done in Tn−1 steps

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 21/23 Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 22/23

Towers of Hanoi, cont.

I Recurrence relation:

I Initial condition:

I Now find closed form for Tn

I What is a particular solution?

I Solution for homogeneous recurrence:

I Solve for α:

I Solution for recurrence:

Instructor: Işıl Dillig, CS311H: Discrete Mathematics Recurrence Relations 23/23