Mathematical Induction

Merge sort Algorithm:

A merge sort proceeds by iteratively splitting
lists into two sublists of equal length (or where
one sublist has one more element than the
other) until each sublist contains one element.
This succession of sublists can be represented
by a balanced binary tree. The procedure
continues by successively merging pairs of
lists, where both lists are in increasing order,
into a larger list with elements in increasing
order, until the original list is put into
increasing order.
Example: Use the merge sort to put the terms
of the list 8, 2, 4, 6, 9, 7, 10, 1, 5, 3 in
increasing order
Example: Merge the two lists 2, 3, 5, 6 and 1, 4.

Solution:
First, compare the smallest elements in the two lists, 2 and 1,
respectively. Because 1 is the smaller, put it at the beginning
of the merged list and remove it from the second list. At this
stage, the first list is 2, 3, 5, 6, the second is 4, and the
combined list is 1.

Next, compare 2 and 4, the smallest elements of the two lists.

Because 2 is the smaller, add
it to the combined list and remove it from the first list. At this
stage the first list is 3, 5, 6, the
second is 4, and the combined list is 1, 2.

Continue by comparing 3 and 4, the smallest elements of

their respective lists. Because 3
is the smaller of these two elements, add it to the combined
list and remove it from the first list.
At this stage the first list is 5, 6, and the second is 4. The combined list
is 1, 2, 3. Then compare 5 and 4, the smallest elements in the two
lists. Because 4 is the smaller of these two elements, add it to the
combined list and remove it from the second list. At this stage the first
list is 5, 6, the second list is empty, and the combined list is 1, 2, 3, 4.

Finally, because the second list is empty, all elements of the first list
can be appended to
the end of the combined list in the order they occur in the first list.
This produces the ordered
list 1, 2, 3, 4, 5, 6.
Recurrence Relation:
A recurrence relation for the sequence {an} is
an equation that expresses an is terms of one
or more of the previous terms of the
sequence, namely,a0,a1,…,an-1, for all integers
n with n  n0, where n0 is a nonnegative
integer.

Solution:
A sequence is called a solution of a
recurrence relation if its terms satisfy the
recurrence relation.
In other words, a recurrence relation is like
a recursively defined sequence, but without
specifying any initial values (initial
conditions).

Therefore, the same recurrence relation can

have (and usually has) multiple solutions.

If both the initial conditions and the

recurrence relation are specified, then the
sequence is uniquely determined.
Example:
Consider the recurrence relation
an = 2an-1 – an-2 for n = 2, 3, 4, …

Is the sequence {an} with an=3n a solution

of this recurrence relation?
For n  2 we see that
2an-1 – an-2 = 2(3(n – 1)) – 3(n – 2) = 3n = an.
Therefore, {an} with an=3n is a solution of
the recurrence relation.
Is the sequence {an} with an=5 a solution of
the same recurrence relation?
For n2 we see that 2an-1 – an-2 = 25 - 5 = 5
= an.
Therefore, {an} with an=5 is also a solution
of the recurrence relation.
Modeling with Recurrence Relations:
Example: Someone deposits $10,000 in a
savings account at a bank yielding 5% per
year with interest compounded annually. How
much money will be in the account after 30
years?
Solution:
Let Pn denote the amount in the account after
n years.
How can we determine Pn on the basis of Pn-
1?
We can derive the following recurrence
relation:
Pn = Pn-1 + 0.05Pn-1 = 1.05Pn-1.
The initial condition is P0 = 10,000.
Then we have:
P1 = 1.05P0
P2 = 1.05P1 = (1.05)2P0
P3 = 1.05P2 = (1.05)3P0
…
Pn = 1.05Pn-1 = (1.05)nP0

We now have a formula to calculate Pn for any

Let us use this formula to find P30 under the
Initial condition P0 = 10,000:

P30 = (1.05)3010,000 = 43,219.42

After 30 years, the account contains

$43,219.42.
Example:
Let an denote the number of bit strings of
length n that do not have two consecutive 0’s
(“valid strings”). Find a recurrence relation
and give initial conditions for the sequence
{an}.

Solution:
Idea: The number of valid strings equals the
number of valid strings ending with a 0 plus
the number of valid strings ending with a 1.
Let us assume that n  3, so that the string
contains at least 3 bits.
Let us further assume that we know the
number an-1 of valid strings of length (n – 1)
and the number an-2 of valid strings of length
(n – 2).
Then how many valid strings of length n are
there, if the string ends with a 1?
There are an-1 such strings, namely the set of
valid strings of length (n – 1) with a 1
appended to them.
Note: Whenever we append a 1 to a valid
Now we need to know: How many valid
strings of length n are there, if the string
ends with a 0?
Valid strings of length n ending with a 0 must
have a 1 as their (n – 1)st bit (otherwise they
would end with 00 and would not be valid).
And what is the number of valid strings of
length (n – 1) that end with a 1?
We already know that there are an-1 strings of
length n that end with a 1.
Therefore, there are an-2 strings of length (n –
1) that end with a 1.
So there are an-2 valid strings of length n that
end with a 0 (all valid strings of length (n – 2)
with 10 appended to them).

As we said before, the number of valid strings

is the number of valid strings ending with a 0
plus the number of valid strings ending with a
1.

That gives us the following recurrence

relation:
an = an-1 + an-2
What are the initial conditions?
a1 = 2 (0 and 1)
a2 = 3 (01, 10, and 11)
a3 = a2 + a1 = 3 + 2 = 5
a4 = a3 + a2 = 5 + 3 = 8
a5 = a4 + a3 = 8 + 5 = 13
…
This sequence satisfies the same recurrence
relation as the Fibonacci sequence.
Since a1 = f3 and a2 = f4, we have an = fn+2.
In general, we would prefer to have an explicit
formula to compute the value of an rather than
conducting n iterations.

Solving Recurrence Relations:

A linear homogeneous recurrence relation of degree
k with constant coefficients is a recurrence relation
of the form:
an = c1an-1 + c2an-2 + … + ckan-k,
Where c1, c2, …, ck are real numbers, and ck  0.

A sequence satisfying such a recurrence relation is

uniquely determined by the recurrence relation and
the k initial conditions
a0 = C0, a1 = C1, a2 = C2, …, ak-1 = Ck-1.
Examples:
The recurrence relation Pn = (1.05)Pn-1 is a
linear homogeneous recurrence relation of
degree one.
The recurrence relation fn=fn-1+fn-2 is a
linear homogeneous recurrence relation of
degree two.
The recurrence relation an=an-5 is a linear
homogeneous recurrence relation of degree
five.
When solving such recurrence relations, we try
to find solutions of the form an = rn, where r is a
constant.

an = rn is a solution of the recurrence relation

an = c1an-1 + c2an-2 + … + ckan-k if and only if
rn = c1rn-1 + c2rn-2 + … + ckrn-k.

Divide this equation by rn-k and subtract the

right-hand side from the left:
rk - c1rk-1 - c2rk-2 - … - ck-1r - ck = 0
This is called the characteristic equation of
the recurrence relation.
The solutions of this equation are called the
characteristic roots of the recurrence relation.
Let us consider linear homogeneous
recurrence relations of degree two.
Theorem:
Let c1 and c2 be real numbers. Suppose that
r2–c1r–c2= 0 has two distinct roots r1 and r2.
Then the sequence {an} is a solution of the
recurrence relation an = c1an-1 + c2an-2 if and
only if an = 1r1n + 2r2n for n = 0, 1, 2, …,
where 1 and 2 are constants.
Example: What is the solution of the
recurrence relation an = an-1 + 2an-2 with a0 = 2
and a1 = 7 ?

Solution:
The characteristic equation of the recurrence
relation is r2 – r – 2 = 0.
Its roots are r = 2 and r = -1.
Hence, the sequence {an} is a solution to the
recurrence relation if and only if:
an = 12n + 2(-1)n for some constants 1 and
2.
Given the equation an = 12n + 2(-1)n and the
initial conditions a0 = 2 and a1 = 7, it follows
that
a0 = 2 = 1 + 2
a1 = 7 = 12 + 2 (-1)

Solving these two equations gives us

1 = 3 and 2 = -1.

Therefore, the solution to the recurrence

relation and initial conditions is the sequence
{an} with
n n
Example:
Give an explicit formula for the Fibonacci numbers.
Solution:
The Fibonacci numbers satisfy the recurrence
relation
fn = fn-1 + fn-2 with initial conditions f0 = 0 and f1 = 1.
The characteristic equation is r2 – r – 1 = 0.Its roots
1 5 1 5
are r1  , r2 
2 2

Therefore, the Fibonacci

n numbers
n are given by
 1 5   1 5 

f n 1     2  
 
 2   2 

For some constants α1 and α2

We can determine values for these
constants so that the sequence meets the
conditions f0=0 and
f 0 1   2 0
f1 = 1:    
1 5 1 5
f1 1   2
 
 1

 2   2 

The unique solution to this system of two

1 1
equations
  1
5
,  and5 two variables is
 2

So finally we obtaine an explicit formula for

thef Fibonacci
1  1  5  numbers:
   
n
1  1 5 
 
n

5   5  
n
2  2 
But what happens if the characteristic
equation has only one root?
How can we then match our equation with the
initial conditions a0 and a1 ?
Theorem:
Let c1 and c2 be real numbers with c2 0.
Suppose that r2 – c1r – c2 = 0 has only one root
r0.
A sequence {an} is a solution of the recurrence
relation an = c1an-1 + c2an-2 if and only if
an = 1r0n + 2nr0n, for n = 0, 1, 2, …, where 1
and 2 are constants.
Example: What is the solution of the recurrence
relation an = 6an-1 – 9an-2 with a0 = 1 and a1 = 6?
Solution:
The only root of r2 – 6r + 9 = 0 is r0 = 3.
Hence, the solution to the recurrence relation is
an = 13n + 2n3n for some constants 1 and 2.
To match the initial condition, we need
a0 = 1 = 1
a1 = 6 = 13 + 23
Solving these equations yields 1 = 1 and 2 = 1.
Consequently, the overall solution is given by
an = 3n + n3n.
Divide-and-Conquer Relations
Some algorithms take a problem and
successively divide it into one or more smaller
problems until there is a trivial solution to them.
For example, the binary search algorithm
recursively divides the input into two halves and
eliminates the irrelevant half until only one
relevant element remained.
This technique is called “divide and conquer”.
We can use recurrence relations to analyze the
complexity of such algorithms.
Suppose that an algorithm divides a problem
(input) of size n into a subproblems, where
each subproblem is of size n/b. Assume that
g(n) operations are performed for such a
division of a problem.
Then, if f(n) represents the number of
operations required to solve the problem, it
follows that f satisfies the recurrence relation
f(n) = af(n/b) + g(n).
This is called a divide-and-conquer
recurrence relation.
Example: The binary search algorithm reduces
the search for an element in a search sequence
of size n to the binary search for this element
in a search sequence of size n/2 (if n is even).
Two comparisons are needed to perform this
reduction.
Hence, if f(n) is the number of comparisons
required to search for an element in a search
sequence of size n, then
f(n) = f(n/2) + 2 if n is even.
Usually, we do not try to solve such divide-and
conquer relations, but we use them to derive a
big-O estimate for the complexity of an algorithm.
Theorem:
Let f be an increasing function that satisfies the
recurrence relation
f(n) = af(n/b) + cnd
whenever n = bk, where k is a positive integer, a,
c, and d are real numbers with a  1, and b is an
integer greater than 1. Then f(n) is
O(nd), if a < bd,
O(nd log n) if a = bd,
O(nlogba) if a > bd
Example:
For binary search, we have
f(n) = f(n/2) + 2, so a = 1, b = 2, and d = 0
(d = 0 because here, g(n) does not depend on
n).
Consequently, a = bd, and therefore,
f(n) is O(nd log n) = O(log n).
The binary search algorithm has logarithmic
time complexity.