Advanced Counting Techniques
Advanced Counting Techniques
Techniques
Chapter 8
Chapter Summary
● Applications of Recurrence Relations
● Solving Linear Recurrence Relations
● Homogeneous Recurrence Relations
● Nonhomogeneous Recurrence Relations
First, we use 1 move to transfer the largest disk to the second peg. Then we transfer the n −1
disks from peg 3 to peg 2 using Hn−1 additional moves. This can not be done in fewer steps.
Hence,
Hn = 2Hn−1 + 1.
The initial condition is H1= 1 since a single disk can be transferred from peg 1 to peg 2 in one move.
The Tower of Hanoi (continued)
● We can use an iterative approach to solve this recurrence relation by repeatedly expressing Hn in terms of the
previous terms of the sequence.
Hn = 2Hn−1 + 1
= 2(2Hn−2 + 1) + 1 = 22 Hn−2 +2 + 1
= 22(2Hn−3 + 1) + 2 + 1 = 23 Hn−3 +22 + 2 + 1
⋮
= 2n-1H1 + 2n−2 + 2n−3 + …. + 2 + 1
= 2n−1 + 2n−2 + 2n−3 + …. + 2 + 1 because H1= 1
= 2n − 1 using the formula for the sum of the terms of a geometric series
● There was a myth created with the puzzle. Monks in a tower in Hanoi are transferring 64 gold disks from
one peg to another following the rules of the puzzle. They move one disk each day. When the puzzle is
finished, the world will end.
● Using this formula for the 64 gold disks of the myth,
264 −1 = 18,446, 744,073, 709,551,615
days are needed to solve the puzzle, which is more than 500 billion years.
● Reve’s puzzle (proposed in 1907 by Henry Dudeney) is similar but has 4 pegs. There is a well-known
unsettled conjecture for the the minimum number of moves needed to solve this puzzle. (see Exercises 38-45)
Counting Bit Strings
Example 3: Find a recurrence relation and give initial conditions for the number of bit strings of length n
without two consecutive 0s. How many such bit strings are there of length five?
Solution: Let an denote the number of bit strings of length n without two consecutive 0s. To obtain a
recurrence relation for {an } note that the number of bit strings of length n that do not have two consecutive 0s
is the number of bit strings ending with a 0 plus the number of such bit strings ending with a 1.
● a3 = a2 + a1 = 3 + 2 = 5
● a4 = a3 + a2 = 5+ 3 = 8
● a5 = a4 + a3 = 8+ 5 = 13
Note that {an } satisfies the same recurrence relation as the Fibonacci
sequence. Since a1 = f3 and a2 = f4 , we conclude that an = fn+2 .
Counting the Ways to Parenthesize a
Product
Example: Find a recurrence relation for C , the number of ways to parenthesize the product of
n
n + 1 numbers, x · x · x · …⋯ · x , to specify the order of multiplication.
0 1 2 n
For example, C = 5, since all the possible ways to parenthesize 4 numbers are
3
((x · x )· x )· x , (x · (x · x ))· x , (x · x )· (x · x ), x · (( x · x ) · x ), x · ( x · ( x · x ))
0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3
Solution: Note that however parentheses are inserted in x · x · x · …⋯ · x , one “·” operator remains
0 1 2 n
outside all parentheses. This final operator appears between two of the n + 1 numbers, say x and x . Since there are
k k+1
C ways to insert parentheses in the product x · x · x · …⋯ · x and C ways to insert parentheses in
· …⋯ · x , we have
k 0 1 2 k n−k−1
the product x · x
k+1 k+2 n
a =a + 2a with a = 2 and a = 7?
n n−1 n−2 0 1
2
Solution: The characteristic equation is r − r − 2 = 0.
Its roots are r = n2 and r =n−1 . Therefore, {a } is a solution to the recurrence relation if and
n
only if a = α 2 + α (−1) , for some constants α and α .
n 1 2 1 2
a = 2 = α + α and a = 7 = α 2 + α (−1).
0 1 2 1 1 2
Solving, we obtain , .
Hence,
The Solution when there is a Repeated
Root
Theorem 2: Let c1 and c2 be real numbers with c2 ≠ 0.
Suppose that r2 – c1r – c2 = 0 has one repeated root r0.
Then the sequence {an} is a solution to the recurrence
relation an = c1an−1 + c2an−2 if and only if
a0 = 1 = α1 and a1 = 6 = α1 · 3 + α2 ·3.
Because F(n)= 2n is a polynomial in n of degree one, to find a particular solution we might try a linear function in n, say pn =
cn + d, where c and d are constants. Suppose that pn = cn + d is such a solution.
Then an = 3an−1 + 2n becomes cn + d = 3(c(n− 1) + d)+ 2n.
Simplifying yields (2 + 2c)n + (2d − 3c) = 0. It follows that cn + d is a solution if and only if
2 + 2c = 0 and 2d − 3c = 0. Therefore, cn + d is a solution if and only if c = − 1 and d = − 3/2.
Consequently, an(p) = −n − 3/2 is a particular solution.
By Theorem 5, all solutions are of the form an = an(p) + an(h) = −n − 3/2 + α3n, where α is a constant.
To find the solution with a1 = 3, let n = 1 in the above formula for the general solution.
Then 3 = −1 − 3/2 + 3 α, and α = 11/6. Hence, the solution is an = −n − 3/2 + (11/6)3n.
Divide-and-Conquer Algorithms
and Recurrence Relations
Section 8.3
Section Summary
● Divide-and-Conquer Algorithms and Recurrence
Relations
● Examples
● Binary Search
● Merge Sort
● Fast Multiplication of Integers
● Master Theorem
● Closest Pair of Points (not covered yet in these slides)
Divide-and-Conquer Algorithmic
Paradigm
Definition: A divide-and-conquer algorithm works by first
dividing a problem into one or more instances of the same
problem of smaller size and then conquering the problem
using the solutions of the smaller problems to find a
solution of the original problem.
Examples:
● Binary search, covered in Chapters 3 and 5: It works by comparing the
element to be located to the middle element. The original list is then split
into two lists and the search continues recursively in the appropriate
sublist.
● Merge sort, covered in Chapter 5: A list is split into two approximately
equal sized sublists, each recursively sorted by merge sort. Sorting is
done by successively merging pairs of lists.
Divide-and-Conquer Recurrence
Relations
when n is even.
Example: Merge Sort
● The merge sort algorithm splits a list of n (assuming n is
even) items to be sorted into two lists with n/2 items. It
uses fewer than n comparisons to merge the two sorted
lists.
● Hence, the number of comparisons required to sort a
sequence of size n, is no more than than M(n) where
M(n) = 2M(n/2) + n.
Example: Fast Multiplication of
Integers
● An algorithm for the fast multiplication of two 2n-bit integers (assuming n is even) first splits each of the 2n-bit integers
into two blocks, each of n bits.
● Suppose that a and b are integers with binary expansions of length 2n. Let
a = (a2n−1a2n−2 … a1a0)2 and b = (b2n−1b2n−2 … b1b0)2 .
● Let a = 2nA1 + A0, b = 2nB1 + B0 , where
A1 = (a2n−1 … an+1an)2 , A0 = (an−1 … a1a0)2 ,
B1 = (b2n−1 … bn+1bn)2 , B0 = (bn−1 … b1b0)2.
● The algorithm is based on the fact that ab can be rewritten as:
ab = (22n + 2n)A1B1 +2n (A1−A0)(B0 − B1) +(2n + 1)A0B0.
● This identity shows that the multiplication of two 2n-bit integers can be carried out using three multiplications of n-bit
integers, together with additions, subtractions, and shifts.
● Hence, if f(n) is the total number of operations needed to multiply two n-bit integers, then
f(2n) = 3f(n) + Cn
where Cn represents the total number of bit operations; the additions, subtractions and shifts that are a constant multiple
of n-bit operations.
Estimating the Size of Divide-and-Conquer
Functions
Theorem 1: Let f be an increasing function that satisfies
the recurrence relation
f(n) = af(n/b) + cnd
whenever n is divisible by b, where a≥ 1, b is an integer
greater than 1, and c is a positive real number. Then
Examples:
● The sequence {ak} with ak = 3 has the generating function
● The sequence {ak} with ak = k + 1 has the generating
function has the generating function
● The sequence {ak} with ak = 2k has the generating function
Generating Functions for Finite
Sequences
● Generating functions for finite sequences of real
numbers can be defined by extending a finite sequence
a0,a1, … , an into an infinite sequence by setting
an+1 = 0, an+2 = 0, and so on.
● The generating function G(x) of this infinite sequence
{an} is a polynomial of degree n because no terms of the
form ajxj with j > n occur, that is,
There are three solutions since the coefficient of x17 in the product is 3.
Counting Problems and Generating
Functions (continued)
Example: Use generating functions to find the number of k-combinations of a
set with n elements, i.e., C(n,k).
Solution: Each of the n elements in the set contributes the term (1 + x) to the
generating function
Hence f(x) = (1 + x)n where f(x) is the generating function for {ak}, where ak
represents the number of k-combinations of a set with n elements.
By the binomial theorem, we have
where
Hence,
Inclusion-Exclusion
Section 8.5
Principle of Inclusion-Exclusion
● In Section 2.2, we developed the following formula
for the number of elements in the union of two finite
sets:
≥
The Principle of Inclusion-Exclusion
(continued)
Proof: An element in the union is counted exactly once
in the right-hand side of the equation. Consider an
element a that is a member of r of the sets A1,…., An
where 1≤ r ≤ n.
● It is counted C(r,1) times by Σ|A |
i
By the inclusion-exclusion principle the number of onto functions from a set with six elements to a
set with three elements is
● Here the total number of functions from a set with six elements to one with three elements is N = 36.
● The number of functions that do not have in the range is N(P1) = 26. Similarly, N(P2) = N(31) = 26 .
● Note that N(P1P2) = N(P1P3) = N(P2P3) = 1 and N(P1P2P3)= 0.
Hence, the number of onto functions from a set with six elements to a set with three elements is: