Problems On Algorithms
Problems On Algorithms
Second Edition
Consisting of
http://hercule.csci.unt.edu/ian/books/free/license.html.
Faculty, if you wish to use this work in your classroom, you are requested to:
• encourage your students to make individual donations, or
• make a lump-sum donation on behalf of your class.
If you have a credit card, you may place your donation online at:
https://www.nationalmssociety.org/donate/donate.asp.
If you restrict your donation to the National MS Society's targeted research campaign, 100% of
your money will be directed to fund the latest research to find a cure for MS. For the story of Ian
Parberry's experience with Multiple Sclerosis, see http:// multiplesclerosissucks.com.
Problems on Algorithms
by
Ian Parberry
([email protected])
Dept. of Computer Science
University of North Texas
Denton, TX 76203
August, 1994
Contents
Preface ix
1 Introduction 1
1.1 How to Use This Book 1
1.2 Overview of Chapters 1
1.3 Pseudocode 3
1.4 Useful Texts 4
2 Mathematical Induction 8
2.1 Summations 8
2.2 Inequalities 10
2.3 Floors and Ceilings 10
2.4 Divisibility 11
2.5 Postage Stamps 11
2.6 Chessboard Problems 12
2.7 Fibonacci Numbers 14
2.8 Binomial Coefficients 14
2.9 What is Wrong? 15
2.10 Graphs 16
2.11 Trees 17
2.12 Geometry 18
2.13 Miscellaneous 19
2.14 Hints 20
2.15 Solutions 21
2.16 Comments 23
iii
iv Contents
4 Recurrence Relations 37
4.1 Simple Recurrences 37
4.2 More Difficult Recurrences 38
4.3 General Formulae 39
4.4 Recurrences with Full History 39
4.5 Recurrences with Floors and Ceilings 40
4.6 Hints 41
4.7 Solutions 41
4.8 Comments 44
5 Correctness Proofs 46
5.1 Iterative Algorithms 46
5.1.1 Straight-Line Programs 47
5.1.2 Arithmetic 47
5.1.3 Arrays 49
5.1.4 Fibonacci Numbers 51
5.2 Recursive Algorithms 51
5.2.1 Arithmetic 52
5.2.2 Arrays 53
5.2.3 Fibonacci Numbers 54
5.3 Combined Iteration and Recursion 54
5.4 Hints 55
5.5 Solutions 56
5.6 Comments 58
6 Algorithm Analysis 59
6.1 Iterative Algorithms 59
Contents v
7 Divide-and-Conquer 67
7.1 Maximum and Minimum 67
7.2 Integer Multiplication 68
7.3 Strassen’s Algorithm 73
7.4 Binary Search 74
7.5 Quicksort 75
7.6 Towers of Hanoi 77
7.7 Depth-First Search 78
7.8 Applications 79
7.9 Hints 81
7.10 Solutions 83
7.11 Comments 85
8 Dynamic Programming 87
8.1 Iterated Matrix Product 87
8.2 The Knapsack Problem 89
8.3 Optimal Binary Search Trees 90
8.4 Floyd’s Algorithm 91
8.5 Applications 92
8.6 Finding the Solutions 97
8.7 Hints 99
8.8 Solutions 99
8.9 Comments 100
vi Contents
12 N P-completeness 147
12.1 General 147
12.2 Cook Reductions 148
12.3 What is Wrong? 150
12.4 Circuits 152
12.5 One-in-Three 3SAT 153
12.6 Factorization 154
Contents vii
13 Miscellaneous 156
13.1 Sorting and Order Statistics 156
13.2 Lower Bounds 157
13.3 Graph Algorithms 158
13.4 Maximum Flow 160
13.5 Matrix Reductions 161
13.6 General 162
13.7 Hints 164
13.8 Solutions 166
13.9 Comments 166
Bibliography 168
Index 174
viii Contents
Preface
The ability to devise effective and efficient algorithms in new situations is a skill
that separates the master programmer from the merely adequate coder. The best
way to develop that skill is to solve problems. To be effective problem solvers,
master-programmers-in-training must do more than memorize a collection of stan-
dard techniques and applications — they must in addition be able to internalize and
integrate what they have learned and apply it in new circumstances. This book is a
collection of problems on the design, analysis, and verification of algorithms for use
by practicing programmers who wish to hone and expand their skills, as a supple-
mentary text for students enrolled in an undergraduate or beginning graduate class
on algorithms, and as a self-study text for graduate students who are preparing for
the qualifying (often called “breadth” or “comprehensive”) examination on algo-
rithms for a Ph.D. program in Computer Science or Computer Engineering. It is
intended to augment the problem sets found in any standard algorithms textbook.
Recognizing that a supplementary text must be cost-effective if it is to be useful,
I have made two important and perhaps controversial decisions in order to keep its
length within reasonable bounds. The first is to cover only what I consider to be
the most important areas of algorithm design and analysis. Although most instruc-
tors throw in a “fun” advanced topic such as amortized analysis, computational
geometry, approximation algorithms, number-theoretic algorithms, randomized al-
gorithms, or parallel algorithms, I have chosen not to cover these areas. The second
decision is not to search for the origin of the problems that I have used. A lengthy
discussion of the provenance of each problem would help make this book more schol-
arly, but would not make it more attractive for its intended audience — students
and practicing programmers.
To make this book suitable for self-instruction, I have provided at the end of
each chapter a small collection of hints, solutions, and comments. The solutions are
necessarily few for reasons of brevity, and also to avoid hindering instructors in their
selection of homework problems. I have included various preambles that summarize
the background knowledge needed to solve the problems so that students who are
familiar with the notation and style of their textbook and instructor can become
more familiar with mine.
ix
x Preface
The organization of this book is a little unusual and requires a few words of
explanation. After a chapter of introduction, it begins with five chapters on back-
ground material that most algorithms instructors would like their students to have
mastered before setting foot in an algorithms class. This will be the case at some
universities, but even then most students would profit from brushing up on this
material by attempting a few of the problems. The introductory chapters include
mathematical induction, big-O notation, recurrence relations, correctness proofs,
and basic algorithm analysis methods. The correctness proof chapter goes beyond
what is normally taught, but I believe that students profit from it once they over-
come their initial aversion to mathematical formalism.
The next four chapters are organized by algorithm design technique: divide-and-
conquer, dynamic programming, greedy algorithms, and exhaustive search. This
organization is somewhat nonstandard, but I believe that the pedagogical benefits
outweigh the drawbacks (the most significant of which seems to be the scattering of
problems on graph algorithms over several chapters, mainly in Sections 2.10, 2.11,
7.7, 8.4, 9.2, 9.3, 9.4, 13.3, and 13.4). Standard textbooks usually devote little time
to exhaustive search because it usually requires exponential time, which is a pity
when it contains many rich and interesting opportunities to practice the application
of algorithm design, analysis, and verification methods. The manuscript is rounded
out with chapters on advanced data structures and N P-completeness. The final
chapter contains miscellaneous problems that do not necessarily fit into the earlier
chapters, and those for which part of the problem is to determine the algorithmic
technique or techniques to be used.
The algorithms in this book are expressed in a Pascal-like pseudocode. Two
problems (Problems 326 and 327) require the reader to examine some code written
in Pascal, but the details of the programming language are not a major factor in
the solutions. In any case, I believe that students of Computer Science should be
literate in many different programming languages.
For those who are interested in technical details, this manuscript was produced
from camera-ready copy provided by the author using LATEX version 2.09 (which
used TEX C Version 3.14t3), using the Prentice Hall macros phstyle.sty writ-
ten by J. Kenneth Shultis, and the twoside, epsf, and makeidx style files. The
Bibliography was created by BibTEX. The index was produced using the program
makeindex. The figures were prepared in encapsulated Postscript form using idraw,
and the graphs using xgraph. The dvi file produced by LATEX was translated into
Postscript using dvips.
A list of errata for this book is available by anonymous ftp from ftp.unt.edu
(IP address 129.120.1.1), in the directory ian/poa. I will be grateful to receive
feedback, suggestions, errata, and in particular any new and interesting problems
(including those from areas not presently covered), which can be sent by electronic
mail to [email protected].
Chapter 1
Introduction
1
2 Chap. 1. Introduction
Symbol Meaning
easy
medium
difficult
hint
solution
comment
that you review this material and attempt some of the problems from Chapters 2–4
before attending the first class. Some kind instructors may spend a few lectures
“reminding” you of this material, but not all of them can afford the time to do so.
In any case, you will be ahead of the game if you devote some time to look over
the material again. If you ignored the material on mathematical induction in your
discrete mathematics class, thinking that it is useless to a programmer, then think
again. If you can master induction, you have mastered recursion — they are two
sides of the same coin.
You may also meet the material from Chapters 5 and 6 in other classes. Chap-
ter 5 contains problems on correctness proofs. Not all instructors will emphasize
correctness proofs in their algorithms class, but mastering them will teach you im-
portant skills in formalization, expression, and design. The approach to correctness
proofs taken in Chapter 5 is closer to that taken by working researchers than the
formal-logic approach expounded by others such as Dijkstra [21]. Chapter 6 con-
tains some problems involving the analysis of some naive algorithms. You should
have met this material in earlier classes, but you probably don’t have the depth of
knowledge and facility that you can get from working the problems in this chapter.
Chapters 7–10 consist of problems on various algorithm design techniques: re-
spectively, divide-and-conquer, dynamic programming, greedy algorithms, and ex-
haustive search. The problems address some of the standard applications of these
techniques, and ask you to demonstrate your knowledge by applying the techniques
to new problems. Divide-and-conquer has already been foreshadowed in earlier
chapters, but the problems in Chapter 7 will test your knowledge further. The
treatment of exhaustive search in Chapter 10 goes beyond what is normally covered
in algorithms courses. The excuse for skimming over exhaustive search is usually a
natural distate for exponential time algorithms. However, extensive problems are
provided here because exhaustive search contains many rich and interesting oppor-
tunities to practice the application of algorithm design, analysis, and verification
methods.
Chapter 11 contains problems on advanced data structures, and Chapter 12
Sec. 1.3. Pseudocode 3
1.3 PSEUDOCODE
The algorithms in this text are described in a Pascal-like pseudocode. Here is a
quick overview of the conventions used:
Data Types: Most variables are either integers or one- or two-dimensional arrays
of integers. The notation P [i..j] is shorthand for an array P , whose elements
are P [i], P [i + 1], . . . , P [j]. Occasionally, variables will be other mathematical
objects such as sets or lists.
Selection: The selection command uses Pascal’s if-then-else syntax, which has
the form
if condition
then S1
else S2
while condition do
S
repeat
S
until condition
4 Chap. 1. Introduction
and is equivalent to
S
while not condition do
S
i := s i := s
while i ≤ f do while i ≥ f do
S S
i := i + 1 i := i − 1
Aho, Hopcroft, and Ullman [1] This was the standard graduate text on algo-
rithms and data structures for many years. It is written quite tersely, with
some sentences requiring a paragraph of explanation for some students. If you
want the plain facts with no singing or dancing, this is the book for you. It is
a little dated in presentation and material, but not particularly so.
Aho, Hopcroft, and Ullman [2] This is the undergraduate version of the pre-
vious book, with more emphasis on programming. It is full of minor errors.
A large number of the Pascal program fragments don’t work, even when you
correctly implement their C-like return statement.
Aho and Ullman [3] This textbook is redefining the undergraduate computer sci-
ence curriculum at many leading institutions. It is a good place to go to brush
up on your discrete mathematics, data structures, and problem solving skills.
Bentley [9, 10] This delightful pair of books are a collection of Jon Bentley’s Pro-
gramming Pearls column from Communications of the ACM. They should be
recommended reading for all undergraduate computer science majors. Bent-
ley explores the problems and pitfalls of algorithm design and analysis, and
pays careful attention to implementation and experimentation. The subjects
chosen are too idiosyncratic and anecdotal to be a textbook, but nonetheless
a useful pair of books.
Brassard and Bratley [14] This is another good algorithms text with a strong
emphasis on design techniques. The title comes from the French word algo-
rithmique, which is what they (perhaps more aptly) call Computer Science.
Cormen, Leiserson, and Rivest [19] This is an excellent text for those who can
handle it. In the spirit of Aho, Hopcroft, and Ullman [1], it does not mess
around. A massive tome, it contains enough material for both a graduate and
undergraduate course in algorithms. It is the definitive text for those who
want to get right down to business, but it is not for the faint of heart.
Even [26] This is the canonical book on graph algorithms covering most of the
graph algorithm material taught in the typical algorithms course, plus more.
It is a little dated, but still very useful.
Gibbons [29] A more modern and carefully presented text on graph algorithms,
superior in many ways to Even [26].
Greene and Knuth [33] A highly mathematical text for an advanced course on
algorithm analysis at Stanford. Recommended for the serious graduate stu-
dent.
Harel [34] This is another good text along the lines of Brassard and Bratley. It
contains excellent high-level descriptions of subjects that tend to confuse the
beginning student if they are presented in all their detailed glory. It is noted
for its biblical quotes.
Horowitz and Sahni [37] This is a reasonable text for many of the topics found
in the typical algorithms class, but it is weak on analysis and but its approach
to backtracking is somewhat idiosyncratic and hard to follow.
Knuth [45, 46, 47] The Knuth three-volume series is the standard reference text
for researchers and students. The material in it has aged well, except for
Knuth’s penchant for an obscure fictitious assembly-level code instead of the
more popular and easy to understand pseudocode.
Kozen [50] A text for graduate students containing 40 lectures on algorithms. The
terse presentation is not for everybody. It contains enough material for a single
graduate course, as opposed to the normal text that allows the instructor
to pick and choose the material to be covered. The choice of material is
strong but not to everybody’s taste. The homework problems (all of which
have solutions) and miscellaneous exercises (some of which have solutions) are
particularly useful.
Lewis and Denenberg [52] A strong data structures text that also covers some
of the material in the typical algorithms course.
Manber [56] Some students find the approach taken in this book, which is algo-
rithm design by successive refinement of intuitive but flawed version of the
algorithm, very helpful. Others, who do not think this way, hate it. It is
certainly more realistic in its approach to algorithm design than more formal
Sec. 1.4. Useful Texts 7
texts, but perhaps it encourages the wrong habits. Some important topics are
not covered in very great detail.
Moret and Shapiro [58] This is planned as the first volume of a two-volume set.
This volume covers tractable problems, and the second volume (unsighted
as yet) will cover intractable problems. It has a definite combinatorial opti-
mization flavor. Dynamic programming is a notable omission from the first
volume.
Papadimitriou and Steiglitz [59] A reasonable text for an algorithms course,
but one which is very heavy with the jargon and mind-set of the combinatorial
optimization and numerical analysis community, which makes it a difficult
book to just dip into at random for the parts of it that are useful.
Purdom and Brown [64] A mathematically rigorous text that focuses on anal-
ysis almost to the exclusion of design principles.
Rawlins [67] An entertaining text that nonetheless is technically rigorous and
detailed.
Sedgewick [72] This book studies a large array of standard algorithms, but it is
largely descriptive in nature and does not spend much time on verification,
analysis, or design techniques. It is excellent for the elementary undergraduate
algorithms curriculum, but not sufficiently rigorous for upper-division courses.
Smith [74] This is another strong algorithms text with an awesome collection of
problems.
Solow [75] This is a good text for the student who has trouble with mathematical
formalism. The chapter on induction may be particularly useful for those who
have trouble getting started.
Weiss [82] A strong data structures text that also covers quite a bit of the material
in the typical algorithms course.
Wilf [83] A book that does a good job on analysis, but does not cover algorithm
design techniques. The selection of topics is limited to chapters on mathe-
matical preliminaries, recursion, network flow, number-theoretic algorithms,
and N P-completeness.
Chapter 2
Mathematical Induction
At first sight, this chapter may seem out of place because it doesn’t seem to be
about algorithms at all. Although the problems in this chapter are mainly mathe-
matical in nature and have very little to do with algorithms, they are good practice
in a technique that is fundamental to algorithm design, verification, and analysis.
Unfortunately, many students come to my algorithms class poorly versed in the use
of induction. Either it was badly taught or they discarded it as useless information
(or both). If you are one of these students, you should start by attempting at least
some of the following problems. In addition to mastering the technique of mathe-
matical induction, you should pay particular attention to the results in Sections 2.1,
2.2, 2.3, 2.7, 2.8, 2.10, and 2.11. Some of them are quite useful in algorithm design
and analysis, as we shall see later.
2.1 SUMMATIONS
1. Prove by induction on n ≥ 0 that ni=1 i = n(n + 1)/2.
n
2. Prove by induction on n ≥ 0 that i=1 i2 = n(n + 1)(2n + 1)/6.
n
3. Prove by induction on n ≥ 0 that i=1 i3 = n2 (n + 1)2 /4.
n
5. Prove by induction on n ≥ 0 that i=1 i(i + 1) = n(n + 1)(n + 2)/3.
8
Sec. 2.1. Summations 9
n
7. Prove by induction on n ≥ 0 that i · i! = (n + 1)! − 1.
i=1
n
8. Prove by induction on n ≥ 1 that i=1 1/2i = 1 − 1/2n .
n
11. Prove by induction on n ≥ 0 that i=0 2i = 2n+1 − 1.
n
13. Prove by induction on n ≥ 0 that i=1 i2i = (n − 1)2n+1 + 2.
2.2 INEQUALITIES
18. Prove by induction on n ≥ 1 that if x > −1, then (1 + x)n ≥ 1 + nx.
n
1 1
2
<2− .
i=1
i n
n n+m−1
= .
m m
Sec. 2.4. Divisibility 11
2.4 DIVISIBILITY
27. Prove by induction on n ≥ 0 that n3 + 2n is divisible by 3.
28. Prove by induction on n ≥ 0 that n5 − n is divisible by 5.
29. Prove by induction on n ≥ 0 that 5n+1 + 2 · 3n + 1 is divisible by 8.
30. Prove by induction on n ≥ 0 that 8n+2 + 92n+1 is divisible by 73.
31. Prove by induction that for all n ≥ 0, 11n+2 + 122n+1 is divisible by 133.
32. Define S ⊆ IN × IN as follows. (0, 0) ∈ S. If (m, n) ∈ S, then
(m + 2, n + 3) ∈ S. Prove by induction on n ≥ 0 that for all (m, n) ∈ S, m + n
is divisible by 5.
35. Prove by induction that the sum of the cubes of three successive natural
numbers is divisible by 9.
36. Let Sn = {1, 2, . . . , 2n} be the set of integers from 1 to 2n. Let
T ⊂ Sn be any subset containing exactly n + 1 elements of Sn . Prove by
induction on n that there exists x, y ∈ T , x = y, such that x divides evenly
into y with no remainder.
38. Show that any integer postage greater than 34 cents can be formed by
using only 5-cent and 9-cent stamps.
39. Show that any integer postage greater than 5 cents can be formed by
using only 2-cent and 7-cent stamps.
40. Show that any integer postage greater than 59 cents can be formed by
using only 7-cent and 11-cent stamps.
41. What is the smallest value of k such that any integer postage greater
than k cents can be formed by using only 4-cent and 9-cent stamps? Prove
your answer correct.
12 Chap. 2. Mathematical Induction
42. What is the smallest value of k such that any integer postage greater
than k cents can be formed by using only 6-cent and 11-cent stamps? Prove
your answer correct.
43. Show that for all n ≥ 1, any positive integer amount of postage that
is at least n(n − 1) cents can be formed by using only n-cent and (n + 1)-cent
stamps.
44. Show that for all m, n ≥ 1 such that gcd(m, n) = 1, there exists k ∈ IN
such that any positive integer amount of postage that is at least k cents can
be formed by using only m-cent and n-cent stamps.
48. Recall that a knight can make one of eight legal moves de-
picted in Figure 2.1. Figure 2.2 shows a closed knight’s tour on an 8 × 8
chessboard, that is, a circular tour of knight’s moves that visits every square
of the chessboard exactly once before returning to the first square. Prove by
induction that a closed knight’s tour exists for any 2k × 2k chessboard for all
k ≥ 3.
2 3
1 4
8 5
7 6
65. What is wrong with the following proof? We claim that 6n = 0 for all
n ∈ IN. The proof is by induction on n ≥ 0. Clearly, if n = 0, then 6n = 0.
Now, suppose that n > 0. Let n = a + b. By the induction hypothesis, 6a = 0
and 6b = 0. Therefore,
6n = 6(a + b) = 6a + 6b = 0 + 0 = 0.
16 Chap. 2. Mathematical Induction
66. What is wrong with the following reasoning? We show that for every
n ≥ 3, Fn is even. The base of the induction is trivial, since F3 = 2. Now
suppose that n ≥ 4 and that Fm is even for all m < n. Then Fn = Fn−1 +Fn−2 ,
and by the induction hypothesis Fn−1 and Fn−2 are even. Thus, Fn is the
sum of two even numbers, and must therefore be even.
2.10 GRAPHS
A graph is an ordered pair G = (V, E), where V is a finite set and E ⊆ V × V . The
elements of V are called nodes or vertices, and the elements of E are called edges.
We will follow the normal conventions of using n for the number of nodes and e for
the number of edges. Where convenient we will take V = {v2 , v2 , . . . , vn }.
67. Prove by induction that a graph with n vertices can have at most
n(n − 1)/2 edges.
72. Prove by induction that every hypercube has a Hamiltonian cycle, that
is, a cycle that visits every vertex exactly once.
73. Prove by induction that the vertices of a hypercube can be colored using
two colors so that no pair of adjacent vertices have the same color.
Sec. 2.11. Trees 17
Dimension 3 Dimension 4
2.11 TREES
A tree is a special kind of graph defined as follows1 . A single vertex v is a tree
with root v. Suppose Ti = (Vi , Ei ) are disjoint trees with roots ri respectively, for
1 ≤ i ≤ k. Suppose r ∈ V1 , V2 , . . . , Vk . Then, T = (V, E) is a tree, where
V = V1 ∪ V2 ∪ · · · ∪ Vk ∪ {r},
E = E1 ∪ E2 ∪ · · · ∪ Ek ∪ {(r, r1 ), (r, r2 ), . . . , (r, rk )}.
1 Technically, this is actually called a rooted tree, but the distinction is not of prime importance
here.
18 Chap. 2. Mathematical Induction
The root of a tree is said to be at level 1. For all i ≥ 1, a child of a level i node
is said to be at level i + 1. The number of levels in a tree is the largest level number
of any node in the tree. A binary tree is a tree in which all nodes have at most two
children. A complete binary tree is a binary tree in which all leaves are at the same
level.
76. Prove by induction that a tree with n vertices has exactly n − 1 edges.
77. Prove by induction that a complete binary tree with n levels has 2n − 1
vertices.
78. Prove by induction that if there exists an n-node tree in which all the
nonleaf nodes have k children, then n mod k = 1.
79. Prove by induction that for every n such that n mod k = 1, there exists
an n-node tree in which all of the nonleaf nodes have k children.
80. Prove by induction that between any pair of vertices in a tree there is a
unique path.
2.12 GEOMETRY
84. Prove that any set of regions defined by n lines in the plane can be
colored with two colors so that no two regions that share an edge have the
same color.
85. Prove by induction that n circles divide the plane into n2 − n + 2
regions if every pair of circles intersect in exactly two points, and no three
circles intersect in a common point. Does this hold for closed shapes other
than circles?
86. A polygon is convex if every pair of points in the polygon can be
joined by a straight line that lies in the polygon. Prove by induction on n ≥ 3
that the sum of the angles of an n-vertex convex polygon is 180(n−2) degrees.
Sec. 2.13. Miscellaneous 19
89. Prove that n straight lines in the plane, all passing through a single
point, divide the plane into 2n regions.
90. Prove that n planes in space, all passing through a single point, no
three of which meet in a straight line, divide space into n(n − 2) + 2 regions.
2.13 MISCELLANEOUS
91. Suppose Mi is an ri−1 × ri matrix, for 1 ≤ i ≤ n. Prove by induction on
n ≥ 1 that the matrix product M1 · M2 · · · Mn is an r0 × rn matrix.
A Gray code is a list of the 2n n-bit strings in which each string differs from the
previous one in exactly one bit. Consider the following algorithm for listing the
20 Chap. 2. Mathematical Induction
n-bit strings. If n = 1, the list is 0, 1. If n > 1, first take the list of (n − 1)-bit
strings, and place a 0 in front of each string. Then, take a second copy of the list of
(n − 1)-bit strings, place a 1 in front of each string, reverse the order of the strings,
and place it after the first list. So, for example, for n = 2 the list is 00, 01, 11, 10,
and for n = 3 the list is 000, 001, 011, 010, 110, 111, 101, 100.
96. Show that every n-bit string appears exactly once in the list generated
by the algorithm.
97. Show that the list has the Gray code property, that is, that each string
differs from the previous one in exactly one bit.
2.14 HINTS
37. You will need to use strong induction. And your base case may not be what
you expect at first,
48. Try combining four n/2 × n/2 tours into one. Your induction hypothesis must
be stronger than what you really need to prove — the tours you build must
have some extra structure to allow them to be joined at the corners.
49. Look carefully at the example given before the problem. It illustrates the
technique that you will need. Also, consider the hint to Problem 48.
69. First, prove that any graph in which all vertices have even degree must have a
cycle C (try constructing one). Delete C, leaving a smaller graph in which all
vertices have even degree. Apply the induction hypothesis (careful: there is
something I am not telling you). Put together C and the results of applying
the induction hypothesis to give the Eulerian cycle.
86. The hardest part of this is the base case, proving that the sum of the three
angles of a triangle is 180 degrees. It can be proved as follows. First, prove
Sec. 2.15. Solutions 21
92. You’ll need to make two cases in your inductive step, according to whether n
is a power of 2 (or something like that) or not.
2.15 SOLUTIONS
n
1. We are required to prove that for all n ≥ 0, i=1 i = n(n + 1)/2. The claim
is certainly true for n = 0,
in which case both sides of the equation are zero.
Suppose that n ≥ 0 and ni=1 i = n(n + 1)/2. We are required to prove that
n+1
i=1 i = (n + 1)(n + 2)/2. Now,
n+1
n
i = i + (n + 1)
i=1 i=1
= n(n + 1)/2 + (n + 1) (by the induction hypothesis)
= (n + 1)(n + 2)/2,
as required.
n
8. We are required to prove that for all n ≥ 1, i=1 1/2i = 1 − 1/2n . The claim
is true for n = 1, since in this case
both sides of the equation are equal to
1/2. Now suppose that n ≥ 1, and ni=1 1/2i = 1 − 1/2n . It remains to prove
that n+1 i
i=1 1/2 = 1 − 1/2
n+1
.
n+1
1 1 1 1
1/2i = + + + · · · + n+1
i=1
2 4 8 2
1 1 1 1 1 1
= + + + + ··· + n
2 2 2 4 8 2
n
1 1 1
= +
2 2 i=1 2i
1 1 1
− + · (1 − n ) (by the induction hypothesis)
2 2 2
= 1 − 1/2n+1 ,
as required.
n
11. We are required to prove that for all n ≥ 0, i=0 2i = 2n+1 − 1. The
0 i
i=0 2 = 1 = 2 − 1.
1
claim is certainly true for
nn =i 0, in which case
n+1
Suppose that n ≥ 0 and i=0 2 = 2 − 1. We are required to prove that
22 Chap. 2. Mathematical Induction
n+1
i=0 2i = 2n+2 − 1. Now,
n+1
n
2i = 2i + 2n+1
i=0 i=0
= (2n+1 − 1) + 2n+1 (by the induction hypothesis)
= 2n+2 − 1,
as required.
37. We are required to show that any amount of postage that is a positive integer
number of cents n > 7 can be formed by using only 3-cent and 5-cent stamps.
The claim is certainly true for n = 8 (one 3-cent and one 5-cent stamp), n = 9
(three 3-cent stamps), and n = 10 (two 5-cent stamps). Now suppose that
n ≥ 11 and all values up to n − 1 cents can be made with 3-cent and 5-cent
stamps. Since n ≥ 11, n − 3 ≥ 8, and hence by hypothesis, n − 3 cents can
be made with 3-cent and 5-cent stamps. Simply add a 3-cent stamp to this
to make n cents. Notice that we can prove something stronger with just a
little more work. The required postage can always be made using at most
two 5-cent stamps. Can you prove this?
n
52. We are required to prove that for all n ≥ 0, i=0 Fi = Fn+2 − 1. The claim
is certainly true for n = 0, since the left-hand side of the equation is F0 = 0,
and the right-hand side is F2 − 1 = 1 − 1 = 0. Now suppose that n ≥ 1, and
n−1
that i=0 Fi = Fn+1 − 1. Then,
n
n−1
Fi = Fi + Fn
i=0 i=0
= (Fn+1 − 1) + Fn (by the induction hypothesis)
= Fn+2 − 1,
as required.
Sec. 2.16. Comments 23
84. We are required to prove that any set of regions defined by n lines in the
plane can be colored with only two colors so that no two regions that share
an edge have the same color. The hypothesis is true for n = 1 (color one side
light, the other side dark). Now suppose that the hypothesis is true for n
lines. Suppose we are given n + 1 lines in the plane. Remove one of the lines
L, and color the remaining regions with two colors (which can be done, by
the induction hypothesis). Replace L. Reverse all of the colors on one side
of the line. Consider two regions that have a line in common. If that line is
not L, then by the induction hypothesis, the two regions have different colors
(either the same as before or reversed). If that line is L, then the two regions
formed a single region before L was replaced. Since we reversed colors on one
side of L only, they now have different colors.
2.16 COMMENTS
1. This identity was apparently discovered by Gauss at age 9 in 1786. There is
an apocryphal story attached to this discovery. Gauss’ teacher set the class
to add the numbers 1 + 2 + · · · + 100. One account I have heard said that the
teacher was lazy, and wanted to occupy the students for a long time so that
he could read a book. Another said that it was punishment for the class being
unruly and disruptive. Gauss came to the teacher within a few minutes with
an answer of 5050. The teacher was dismayed, and started to add. Some time
later, after some fretting and fuming, he came up with the wrong answer. He
refused to believe that he was wrong until Gauss explained:
24 Chap. 2. Mathematical Induction
1 + 2 + ··· + 50
+ 100 + 99 + · · · + 51
101 + 101 + · · · + 101 = 50 × 101 = 5050.
Can you prove this result without using induction? The case for even n can
easily be inferred from Gauss’ example. The case for odd n is similar.
3. I could go on asking for more identities like this. I can never remember the
exact form of the solution. Fortunately, there is not much need to. In general,
if you are asked to solve
n
ik
i=1
for some k ≥ 1, just remember that the solution is a polynomial in n with
largest exponent k + 1 (see Problem 21). Simply hypothesize that the sum is
ak+1 nk+1 + ak nk + · · · + a1 n + a0
for some choice of ai ∈ IR, 0 ≤ i ≤ k + 1, and try to prove it by induction.
The right values for the constants will drop right out of the proof. It gets a
bit tricky as k becomes larger, though. If you want to practice this technique,
start with k = 2. If you want a challenge to your ability to do arithmetic, try
k = 4.
4. Yes, I realize that you can solve this without induction by applying some
elementary algebra and the identities of Problems 1 and 2. What I want you
to do is practice your induction by proving this identity from first principles.
10. This can also be solved by applying the identity of Problem 9 twice.
11. This is a special case of the identity in Problem 9. There is an easy noninduc-
tive proof of this fact that appeals more to computer
n science students than
math students. The binary representation of i=0 2i is a string of n + 1 ones,
that is,
· · · 1 .
11
n+1
Sec. 2.16. Comments 25
2 n-1 2 n-1
n-1
2n
Figure
n 2.7. Using the method of areas to show that
n−1
i n i
i=1 i2 = (2n − 1)2 − i=1 2 .
· · · 0,
1 00
n+1
n n
that is, 2n+1 . Therefore, i=0 2i + 1 = 2n+1 or, equivalently, i=0 2i =
2n+1 − 1.
13. There is an elegant noninductive proof of this using the method of areas. This
technique works well for sums of products: Simply make a rectangle with area
equal to each term of the sum, fit them together to “nearly” make a rectangle,
and analyze the missing area.
n
n−1
i2i = (2n − 1)2n − 2i (see Figure 2.7)
i=1 i=1
= (2n − 1)2n − (2 − 2) n
(by Problem 1)
= (n − 1)2n+1 + 2.
15. This can also be solved using the method of areas sketched in the comment
to Problem 13.
16. This can also be solved using algebra and the identity from Problem 15.
19. And, therefore, 3n = O(n!). But we’re getting ahead of ourselves here. See
Chapter 3.
20. And, therefore, 2n = Ω(n2 ). But we’re getting ahead of ourselves here. See
Chapter 3.
48. There are actually closed knight’s tours on n × n chessboards for all even
n ≥ 6 (see Problem 369). Note that Problem 47 implies that there can be no
such tour for odd n. The formal study of the knight’s tour problem is said
26 Chap. 2. Mathematical Induction
to have begun with Euler [25] in 1759, who considered the standard 8 × 8
chessboard. Rouse Ball and Coxeter [8] give an interesting bibliography of
the history of the problem from this point.
51. This problem was suggested to the author by Nainan Kovoor in 1990.
ones. But since you have written exactly as many ones as zeros, and you have
written n2n bits in all, there must be n2n−1 ones.
64. Of course it is obvious that the proof is wrong, since all horses are not of the
same color. What I want you to do is point out where in the proof is there a
false statement made?
67. This can be proved using Problem 1, but what I’m looking for is a straight-
forward inductive proof.
69. An Eulerian cycle is not possible if the graph has at least one vertex of odd
degree. (Can you prove this?) Therefore, we conclude that a graph has a
Eulerian cycle iff it is connected and all vertices have even degree. The Swiss
mathematician Leonhard Euler first proved this in 1736 [24].
81. This problem together with Problem 82 show that “a connected graph with
no cycles” is an alternate definition of a tree. You’ll see this in the litera-
ture as often as you’ll see the recursive definition given at the beginning of
Section 2.11.
94. Yes, this is a little contrived. It’s easier to prove by contradiction. But it’s
good practice in using induction.
Big-O notation is useful for the analysis of algorithms since it captures the asymp-
totic growth pattern of functions and ignores the constant multiple (which is out
of our control anyway when algorithms are translated into programs). We will use
the following definitions (although alternatives exist; see Section 3.5). Suppose that
f, g : IN → IN.
• f (n) is O(g(n)) if there exists c, n0 ∈ IR+ such that for all n ≥ n0 , f (n) ≤
c · g(n).
• f (n) is Ω(g(n)) if there exists c, n0 ∈ IR+ such that for all n ≥ n0 , f (n) ≥
c · g(n).
• f (n) is Θ(g(n)) if f (n) is O(g(n)) and f (n) is Ω(g(n)).
• f (n) is o(g(n)) if limn→∞ f (n)/g(n) = 0.
• f (n) is ω(g(n)) if limn→∞ g(n)/f (n) = 0.
• f (n) ∼ g(n) if limn→∞ f (n)/g(n) = 1.
We will follow the normal convention of writing, for example, “f (n) = O(g(n))”
instead of “f (n) is O(g(n))”, even though the equality sign does not indicate true
equality. The proper definition of “O(g(n))” is as a set:
O(g(n)) = {f (n) | there exists c, n0 ∈ IR+ such that for all n ≥ n0 , f (n) ≤ c · g(n)}.
28
Sec. 3.1. Rank the Functions 29
Group these functions so that f (n) and g(n) are in the same group if and
only if f (n) = O(g(n)) and g(n) = O(f (n)), and list the groups in increasing
order.
99. Draw a line from each of the five functions in the center to the best
big-Ω value on the left, and the best big-O value on the right.
Ω(1/n) O(1/n)
Ω(1) O(1)
Ω(log log n) O(log log n)
Ω(log n) O(log n)
2 2
Ω(log√ n) O(log
√ n)
3 3
Ω( n) O( n)
Ω(n/ log n) 1/(log n) O(n/ log n)
Ω(n) 7n5 − 3n + 2 O(n)
Ω(n1.00001 ) (n2 + n)/(log2 n + log n) O(n1.00001 )
2
Ω(n2 / log2 n) 2log n O(n2 / log2 n)
Ω(n2 / log n) 3n O(n2 / log n)
Ω(n2 ) O(n2 )
Ω(n3/2 ) O(n3/2 )
Ω(2n ) O(2n )
Ω(5n ) O(5n )
Ω(nn ) O(nn )
2 2
Ω(nn ) O(nn )
For each of the following pairs of functions f (n) and g(n), either f (n) = O(g(n))
or g(n) = O(f (n)), but not both. Determine which is the case.
For each of the following pairs of functions f (n) and g(n), state whether f (n) =
O(g(n)), f (n) = Ω(g(n)), f (n) = Θ(g(n)), or none of the above.
127. f (n) = n2 + 3n + 4, g(n) = 6n + 7.
√
128. f (n) = n, g(n) = log(n + 3).
√
129. f (n) = n n, g(n) = n2 − n.
√
130. f (n) = n + n n, g(n) = 4n log(n2 + 1).
131. f (n) = (n2 + 2)/(1 + 2−n ), g(n) = n + 3.
132. f (n) = 2n − n2 , g(n) = n4 + n2 .
Sec. 3.3. Proving Big-O 31
2
156. Prove or disprove: 2(1+O(1/n)) = 2 + O(1/n).
Compare the following pairs of functions f , g. In each case, say whether f = o(g),
f = ω(g), or f = Θ(g), and prove your claim.
For each of the following pairs of functions f (n) and g(n), find c ∈ IR+ such that
f (n) ≤ c · g(n) for all n > 1.
173. Prove that if f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)), then f1 (n) +
f2 (n) = O(max{g1 (n), g2 (n)}).
Sec. 3.5. Alternative Definitions 33
174. Prove that if f1 (n) = Ω(g1 (n)) and f2 (n) = Ω(g2 (n)), then f1 (n) +
f2 (n) = Ω(min{g1 (n), g2 (n)}).
175. Suppose that f1 (n) = Θ(g1 (n)) and f2 (n) = Θ(g2 (n)). Is it true
that f1 (n) + f2 (n) = Θ(g1 (n) + g2 (n))? Is it true that f1 (n) + f2 (n) =
Θ(max{g1 (n), g2 (n)})? Is it true that f1 (n) + f2 (n) = Θ(min{g1 (n), g2 (n)})?
Justify your answer.
176. Prove that if f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)), then f1 (n) ·
f2 (n) = O(g1 (n) · g2 (n)).
177. Prove that if f1 (n) = Ω(g1 (n)) and f2 (n) = Ω(g2 (n)), then f1 (n)·f2 (n) =
Ω(g1 (n) · g2 (n)).
178. Prove or disprove: For all functions f (n) and g(n), either f (n) =
O(g(n)) or g(n) = O(f (n)).
179. Prove or disprove: If f (n) > 0 and g(n) > 0 for all n, then O(f (n) +
g(n)) = f (n) + O(g(n)).
180. Prove or disprove: O(f (n)α ) = O(f (n))α for all α ∈ IR+ .
181. Prove or disprove: O(x + y)2 = O(x2 ) + O(y 2 ).
√
182. Multiply log n + 6 + O(1/n) by n + O( n) and simplify your answer as
much as possible.
183. Show that big-O is transitive. That is, if f (n) = O(g(n)) and g(n) =
O(h(n)), then f (n) = O(h(n)).
184. Prove that if f (n) = O(g(n)), then f (n)k = O(g(n)k ).
185. Prove or disprove: If f (n) = O(g(n)), then 2f (n) = O(2g(n) ).
186. Prove or disprove: If f (n) = O(g(n)), then log f (n) = O(log g(n)).
187. Suppose f (n) = Θ(g(n)). Prove that h(n) = O(f (n)) iff h(n) = O(g(n)).
188. Prove or disprove: If f (n) = O(g(n)), then f (n)/h(n) = O(g(n)/h(n)).
189. Prove that if f (n) = O(g(n)), then f (n) = O1 (g(n)), or find a counterex-
ample to this claim.
34 Chap. 3. Big-O and Big-Ω
190. Prove that if f (n) = O1 (g(n)), then f (n) = O(g(n)), or find a counterex-
ample to this claim.
191. Prove that if f (n) = Ω(g(n)), then f (n) = Ω2 (g(n)), or find a counterex-
ample to this claim.
192. Prove that if f (n) = Ω2 (g(n)), then f (n) = Ω(g(n)), or find a coun-
terexample to this claim.
193. Prove that if f (n) = Ω1 (g(n)), then f (n) = Ω2 (g(n)), or find a coun-
terexample to this claim.
194. Prove that if f (n) = Ω2 (g(n)), then f (n) = Ω1 (g(n)), or find a coun-
terexample to this claim.
195. Prove or disprove: If f (n) = O(g(n)), then f (n) = Ω(g(n)). If
f (n) = O(g(n)), then f (n) = Ω2 (g(n)).
196. Define the relation ≡ by f (n) ≡ g(n) iff f (n) = Ω(g(n)) and g(n) =
Ω(f (n)). Similarly, define the relation ≡2 by f (n) ≡ g(n) iff f (n) = Ω2 (g(n))
and g(n) = Ω2 (f (n)). Show that ≡ is an equivalence relation, but ≡2 is not
an equivalence relation.
3.7 HINTS
98. There are a lot of groups.
136. When solving problems that require you to prove that f (n) = O(g(n)), it is a
good idea to try induction first. That is, pick a c and an n0 , and try to prove
that for all n ≥ n0 , f (n) ≤ c · g(n). Try starting with n0 = 1. A good way
of guessing a value for c is to look at f (n) and g(n) for n = n0 , · · · , n0 + 10.
If the first function seems to grow faster than the second, it means that you
must adjust your n0 higher — eventually (unless the thing you are trying to
prove is false), the first function will grow more slowly than the second. The
ratio of f (n0 )/g(n0 ) gives you your first cut for c. Don’t be afraid to make c
higher than this initial guess to help you make it easier to prove the inductive
step. This happens quite often. You might even have to go back and adjust
n0 higher to make things work out correctly.
171. Start by writing down the definitions for f1 (n) = O(g1 (n)) and f2 (n) =
O(g2 (n)).
3.8 SOLUTIONS
133. We are required to prove that (n + 1)2 = O(n2 ). We need to find a constant
c such that (n + 1)2 ≤ cn2 . That is, n2 + 2n + 1 ≤ cn2 or, equivalently,
(c − 1)n2 − 2n − 1 ≥ 0. Is√this possible? It is if c > 1. Take c = 4. The roots
of 3n2 − 2n − 1 are (2 ± 4 + 12)/2 = {−1/3, 1}. The second root gives us
the correct value for n0 . Therefore, for all n ≥ 1, (n + 1)2 ≤ 4n2 , and so by
definition, (n + 1)2 = O(n2 ).
136. We are required to prove that log n = O(n). By looking at log n for
small values of n, it appears that for all n ≥ 1, log n ≤ n. The proof is
by induction on n. The claim is certainly true for n = 1. Now suppose that
n > 1, and log(n − 1) ≤ n − 1. Then,
log n ≤ log(n − 1) + 1
≤ (n − 1) + 1 (by the induction hypothesis)
= n.
137. We are required to prove that 3n log n = O(n2 ). By looking at 3n log n for
small values of n, it appears that for all n ≥ 1, 3n log n ≤ 3n2 . The proof is
by induction on n. The claim is certainly true for n = 1. Now suppose that
36 Chap. 3. Big-O and Big-Ω
3n log n
≤ 3n( log(n − 1) + 1)
= 3(n − 1)( log(n − 1) + 1) + 3( log(n − 1) + 1)
= 3(n − 1) log(n − 1) + 3(n − 1) + 3( log(n − 1) + 1)
≤ 3(n − 1)2 + 3(n − 1) + 3( log(n − 1) + 1)
(by the induction hypothesis)
≤ 3(n − 1)2 + 3(n − 1) + 3n (see the solution to Problem 136)
= 3n2 − 6n + 3 + 3n − 3 + 3n
= 3n2 .
171. We are required to prove that if f1 (n) = O(g1 (n)) and f2 (n) = O(g2 (n)), then
f1 (n)+f2 (n) = O(g1 (n)+g2 (n)). Suppose for all n ≥ n1 , f1 (n) ≤ c1 ·g1 (n) and
for all n ≥ n2 , f2 (n) ≤ c2 ·g2 (n). Let n0 = max{n1 , n2 } and c0 = max{c1 , c2 }.
Then for all n ≥ n0 , f1 (n) + f2 (n) ≤ c1 · g1 (n) + c2 · g2 (n) ≤ c0 (g1 (n) + g2 (n)).
3.9 COMMENTS
173. This is often called the sum rule for big-Os.
Recurrence Relations
Recurrence relations are a useful tool for the analysis of recursive algorithms, as we
will see later in Section 6.2. The problems in this chapter are intended to develop
skill in solving recurrence relations.
37
38 Chap. 4. Recurrence Relations
Let X(n) be the number of different ways of parenthesizing the product of n values.
For example, X(1) = X(2) = 1, X(3) = 2 (they are (xx)x and x(xx)), and X(4) = 5
(they are x((xx)x), x(x(xx)), (xx)(xx), ((xx)x)x, and (x(xx))x).
223. Prove that if n ≤ 2, then X(n) = 1; and otherwise
n−1
X(n) = X(k) · X(n − k)
k=1
231. State and prove a general formula for recurrences of the form
d if n ≤ 1
T (n) =
aT (n/c) + b otherwise.
232. State and prove a general formula for recurrences of the form
d if n ≤ 1
T (n) =
aT (n/c) + bn2 otherwise.
233. State and prove a general formula for recurrences of the form
d if n ≤ 1
T (n) =
aT (n/c) + bnk otherwise.
243. Solve the following recurrence relation exactly. T (1) = 1 and for
all n ≥ 2, T (n) = T ( n/2 ) + 1.
244. Solve the following recurrence relation exactly. T (1) = 1 and for all
n ≥ 2, T (n) = T ( n/2 ) + 1.
245. Solve the following recurrence relation exactly. T (1) = 1, and for
n ≥ 2, T (n) = 2T ( n/2 ) + 6n − 1.
246. Solve the following recurrence relation exactly. T (1) = 2, and for all
n ≥ 2, T (n) = 4T ( n/3 ) + 3n − 5.
247. √ Solve the following recurrence: T (1) = 1, and for all n ≥ 2, T (n) =
T( n ) + 1.
248. Solve the following recurrence: T (1) = T (2) = 1, and for all
n ≥ 3, T (n) = T ( n/ log n ) + 1.
Sec. 4.6. Hints 41
4.6 HINTS
202. Try repeated substitution (see the comment to Problem 202 for a definition
of this term).
226. If you think about it, this is really two independent recurrence relations, one
for odd n and one for even n. Therefore, you will need to make a special case
for even n and one for odd n. Once you realize this, the recurrences in this
group of problems are fairly easy.
242. The solution is T (n) = n log n −2log n +1. This can be proved by induction.
Can you derive this answer from first principles?
243. This problem is difficult to solve directly, but it becomes easy when you
use the following fact from Graham, Knuth, and and Patashnik [30, Sec-
tion 6.6]. Suppose f : IR → IR is continuous and monotonically increasing, and
has the property that if f (x) ∈ ZZ, then x ∈ ZZ. Then, f ( x ) = f (x) and
f ( x ) = f (x) .
248. The answer is that for n ≥ 3, T (n) ≤ 1.76 log n/ log log n, but this may not
help you much.
4.7 SOLUTIONS
202. Suppose T (1) = 1, and for all n ≥ 2, T (n) = 3T (n − 1) + 2. If n is large
enough, then by repeated substitution,
i−1
T (n) = 3i T (n − i) + 2 3j . (4.1)
j=0
42 Chap. 4. Recurrence Relations
Then,
i−2
T (n) = 3i−1 T (n − i + 1) + 2 3j
j=0
i−2
= 3i−1 (3T (n − i) + 2) + 2 3j
j=0
i−1
= 3i T (n − i) + 2 3j ,
j=0
as required. Now that we’ve established identity (4.1), we can continue with
solving the recurrence. Suppose we take i = n − 1. Then, by identity (4.1),
n−2
n−1
T (n) = 3 T (1) + 2 3j
j=0
i−1
= T (n − 2i) + 3in − 6 j + 4i
j=0
= T (n − 2i) + 3in − 3i(i − 1) + 4i (by Problem 1)
= T (n − 2i) + 3in − 3i2 + 7i.
This can be verified easily by induction. Now, suppose that n is even. Take
i = n/2 − 1. Then,
Therefore, when n is even, T (n) = (3n2 + 14n − 16)/4, and when n is odd,
T (n) = (3n2 + 14n − 13)/4. Or, more succinctly, T (n) = (3n2 + 14n − 16 +
3(n mod 2))/4.
n−1
235. Suppose that T (1) = 1, and for all n ≥ 2, T (n) = i=1 T (i) + 1.
n−1
n−2
T (n) − T (n − 1) = ( T (i) + 1) − ( T (i) + 1)
i=1 i=1
= T (n − 1).
4.8 COMMENTS
202. The technique used to solve this problem in the previous section is called
repeated substitution. It works as follows. Repeatedly substitute until you
see a pattern developing. Write down a formula for T (n) in terms of n and
the number of substitutions (which you can call i). To be completely formal,
Sec. 4.8. Comments 45
verify this pattern by induction on i. (You should do this if you are asked to
prove that your answer is correct or if you want to be sure that the pattern
is right, but you can skip it is you are short of time.) Then choose i to be
whatever value it takes to make all of the T (·)s in the pattern turn into the
base case for the recurrence. You are usually left with some algebra to do,
such as a summation or two. Once you have used repeated substitution to
get an answer, it is prudent to check your answer by using induction. (It
is not normally necessary to hand in this extra work when you are asked to
solve the recurrence on a homework or an exam, but it’s a good idea to do it
for your own peace of mind.) Let’s do it for this problem. We are required
to prove that the solution to the recurrence T (1) = 1, and for all n ≥ 2,
T (n) = 3T (n − 1) + 2, is T (n) = 2 · 3n−1 − 1. The proof is by induction
on n. The claim is certainly true for n = 1. Now suppose that n > 1 and
T (n − 1) = 2 · 3n−2 − 1. Then,
T (n) = 3T (n − 1) + 2
= 3(2 · 3n−2 − 1) + 2 (by the induction hypothesis)
= 2 · 3n−1 − 1,
as required.
222. Hence, Fn = O(1.62n ). For more information on the constant multiple in the
big-O bound, see Graham, Knuth, and Patashnik [30].
Chapter 5
Correctness Proofs
How do we know that a given algorithm works? The best method is to prove it
correct. Purely recursive algorithms can be proved correct directly by induction.
Purely iterative algorithms can be proved correct by devising a loop invariant for
every loop, proving them correct by induction, and using them to establish that the
algorithm terminates with the correct result. Here are some simple algorithms to
practice on. The algorithms use a simple pseudocode as described in Section 1.3.
The loop invariants discussed in this section use the following convention: If
v is a variable used in a loop, then vj denotes the value stored in v immediately
after the jth iteration of the loop, for j ≥ 0. The value v0 means the contents of v
immediately before the loop is entered.
46
Sec. 5.1. Iterative Algorithms 47
procedure swap(x, y)
comment Swap x and y.
1. x := x + y
2. y := x − y
3. x := x − y
5.1.2 Arithmetic
250. Prove that the following algorithm for the addition of natural
numbers is correct.
function add(y, z)
comment Return y + z, where y, z ∈ IN
1. x := 0; c := 0; d := 1;
2. while (y > 0) ∨ (z > 0) ∨ (c > 0) do
3. a := y mod 2;
b := z mod 2;
4. if a ⊕ b ⊕ c then x := x + d;
5. c := (a ∧ b) ∨ (b ∧ c) ∨ (a ∧ c);
6. d := 2d; y := y/2 ;
z := z/2 ;
7. return(x)
251. Prove that the following algorithm for incrementing a natural number is
correct.
function increment(y)
comment Return y + 1, where y ∈ IN
1. x := 0; c := 1; d := 1;
2. while (y > 0) ∨ (c > 0) do
3. a := y mod 2;
4. if a ⊕ c then x := x + d;
5. c := a ∧ c;
6. d := 2d; y := y/2 ;
7. return(x)
252. Prove that the following algorithm for the multiplication of natural
numbers is correct.
48 Chap. 5. Correctness Proofs
function multiply(y, z)
comment Return yz, where y, z ∈ IN
1. x := 0;
2. while z > 0 do
3. if z mod 2 = 1 then x := x + y;
4. y := 2y; z := z/2 ;
5. return(x)
function multiply(y, z)
comment Return yz, where y, z ∈ IN
1. x := 0;
2. while z > 0 do
3. x := x + y · (z mod 2);
4. y := 2y; z := z/2 ;
5. return(x)
254. Prove that the following algorithm for the multiplication of natural
numbers is correct, for all integer constants c ≥ 2.
function multiply(y, z)
comment Return yz, where y, z ∈ IN
1. x := 0;
2. while z > 0 do
3. x := x + y · (z mod c);
4. y := c · y; z := z/c ;
5. return(x)
255. Prove that the following algorithm for division and remaindering of
natural numbers is correct.
function divide(y, z)
comment Return q, r ∈ IN such that y = qz + r
and r < z, where y, z ∈ IN
1. r := y; q := 0; w := z;
2. while w ≤ y do w := 2w;
3. while w > z do
4. q := 2q; w := w/2 ;
5. if w ≤ r then
6. r := r − w; q := q + 1
7. return(q, r)
Sec. 5.1. Iterative Algorithms 49
function power(y, z)
comment Return y z , where y ∈ IR, z ∈ IN
1. x := 1;
2. while z > 0 do
3. x := x · y
4. z := z − 1
5. return(x)
function power(y, z)
comment Return y z , where y ∈ IR, z ∈ IN
1. x := 1;
2. while z > 0 do
3. if z is odd then x := x · y
4. z := z/2
5. y := y 2
6. return(x)
258. Prove that the following algorithm for computing factorials is correct.
function factorial(y)
comment Return y!, where y ∈ IN
1. x := 1
2. while y > 1 do
3. x := x · y; y := y − 1
4. return(x)
5.1.3 Arrays
259. Prove that the following algorithm that adds the values in an array
A[1..n] is correct.
function sum(A) n
comment Return i=1 A[i]
1. s := 0;
2. for i := 1 to n do
3. s := s + A[i]
4. return(s)
50 Chap. 5. Correctness Proofs
260. Prove that the following algorithm for computing the maximum value
in an array A[1..n] is correct.
function max(A)
comment Return max A[1], . . . , A[n]
1. m := A[1]
2. for i := 2 to n do
3. if A[i] > m then m := A[i]
4. return(m)
procedure bubblesort(A[1..n])
comment Sort A[1], A[2], . . . , A[n] into nondecreasing order
1. for i := 1 to n − 1 do
2. for j := 1 to n − i do
3. if A[j] > A[j + 1] then
4. Swap A[j] with A[j + 1]
function match(P, S, n, m)
comment Find the pattern P [0..m − 1] in string S[1..n]
1. . := 0; matched := false
2. while (. ≤ n − m) ∧ ¬matched do
3. . := . + 1 ;
4. r := 0; matched := true
5. while (r < m) ∧ matched do
6. matched := matched ∧ (P [r] = S[. + r])
7. r := r + 1
8. return(.)
264. Prove that the following algorithm for evaluating the polynomial
an xn + an−1 xn−1 + · · · + a1 x + a0 is correct, where the coefficients are stored
in an array A[0..n], with A[i] = ai for all 0 ≤ i ≤ n. The algorithm is named
after its inventor, William G. Horner, and is often called Horner’s rule.
function fib(n)
1. comment Return Fn , the nth Fibonacci number
2. if n = 0 then return(0) else
3. last:=0; current:=1
4. for i := 2 to n do
5. temp:=last+current; last:=current; current:=temp
6. return(current)
• You will prove correctness by induction on the “size” of the problem being
solved (for example, the size of array chunk, number of bits in an integer,
etc.). Your first task is to identify what is to be used as the “size.”
52 Chap. 5. Correctness Proofs
• Then, you prove the base of the induction, which will usually involve only the
base of the recursion.
• Next, you need to prove that recursive calls are given subproblems to solve
(that is, there is no infinite recursion). This is often trivial, but it can be
difficult to prove and so should not be overlooked.
• Finally, you prove the inductive step: Assume that the recursive calls work
correctly, and use this assumption to prove that the current call works cor-
rectly.
5.2.1 Arithmetic
266. Prove that the following recursive algorithm computes 3n − 2n for all
n ≥ 0.
function g(n)
1. if n ≤ 1 then return(n)
2. else return(5 · g(n − 1) − 6 · g(n − 2))
267. Prove that the following recursive algorithm for incrementing a natural
number is correct.
function increment(y)
comment Return y + 1.
1. if y = 0 then return(1) else
2. if y mod 2 = 1 then
3. return(2 · increment( y/2 ))
4. else return(y + 1)
268. Prove that the following recursive algorithm for the addition of natural
numbers is correct.
function add(y, z, c)
comment Return the sum y + z + c, where c ∈ {0, 1}.
1. if y = z = 0 then return(c) else
2. a := y mod 2; b := z mod 2;
3. return(2 · add( y/2 , z/2 , (a + b + c)/2 ) + (a ⊕ b ⊕ c))
269. Prove that the following recursive algorithm for the multiplication
of natural numbers is correct.
function multiply(y, z)
comment Return the product yz.
1. if z = 0 then return(0) else
2. if z is odd then return(multiply(2y, z/2 )+y)
3. else return(multiply(2y, z/2 ))
Sec. 5.2. Recursive Algorithms 53
270. Prove that the following recursive algorithm for the multiplication of
natural numbers is correct.
function multiply(y, z)
comment Return the product yz.
1. if z = 0 then return(0) else
2. return(multiply(2y, z/2 ) + y · (z mod 2))
271. Prove that the following recursive algorithm for the multiplication of
natural numbers is correct, for all integers constants c ≥ 2.
function multiply(y, z)
comment Return the product yz.
1. if z = 0 then return(0) else
2. return(multiply(cy, z/c ) + y · (z mod c))
272. Prove that the following recursive algorithm for exponentiation is correct.
function power(y, z)
comment Return y z , where y ∈ IR, z ∈ IN.
1. if z = 0 then return(1)
2. if z is odd then return(power(y 2 , z/2 ) · y)
3. else return(power(y 2 , z/2 ))
273. Prove that the following recursive algorithm for computing factorials is
correct.
function factorial(n)
comment Return n!.
1. if n ≤ 1 then return(1)
2. else return(n · factorial(n − 1))
5.2.2 Arrays
274. Prove that the following recursive algorithm for computing the maximum
value in an array A[1..n] is correct.
function maximum(n)
comment Return max of A[1..n].
1. if n ≤ 1 then return(A[1])
2. else return(max(maximum(n − 1),A[n]))
function max(x, y)
comment Return largest of x and y.
3. if x ≥ y then return(x) else return(y)
54 Chap. 5. Correctness Proofs
275. Prove that the following recursive algorithm that adds the values in an
array A[1..n] is correct.
function sum(n)
comment Return sum of A[1..n].
1. if n ≤ 1 then return(A[1])
2. else return(sum(n − 1) + A[n])
function fib(n)
comment Return Fn , the nth Fibonacci number.
1. if n ≤ 1 then return(n)
2. else return(fib(n − 1)+fib(n − 2))
function fib(n)
comment Return (Fn−1 , Fn )
1. if n is odd then
2. (a, b) := even(n − 1)
3. return(b, a + b)
4. else return(even(n))
function even(n)
comment Return (Fn−1 , Fn ) when n is even
1. if n = 0 then return(1, 0)
2. else if n = 2 then return(1, 1)
3. else if n = 4 then return(2, 3)
4. (a, b) := fib(n/2 − 1)
5. c := a + b; d := b + c
6. return(b · d + a · c, c · (d + b))
278. Prove that the following recursive bubblesort algorithm is correct. The
values to be sorted are in an array A[1..n].
procedure bubblesort(n)
comment Sort A[1..n].
1. if n > 1 then
2. for j := 1 to n − 1 do
3. if A[j] > A[j + 1] then
4. Swap A[j] with A[j + 1]
5. bubblesort(n − 1)
279. Prove that the following variant of quicksort is correct. The values
to be sorted are in an array A[1..n].
1. procedure quicksort(., r)
2. comment sort S[...r]
3. i := .; j := r
a := some element from S[...r]
4. repeat
5. while S[i] < a do i := i + 1
6. while S[j] > a do j := j − 1
7. if i ≤ j then
8. swap S[i] and S[j]
9. i := i + 1; j := j − 1
10. until i > j
11. if . < j then quicksort(., j)
12. if i < r then quicksort(i, r)
5.4 HINTS
In each of the following hints, if the algorithm uses a variable v, then vi denotes the
contents of v immediately after the ith iteration of the single loop in the algorithm
(v0 denotes the contents of v immediately before the loop is entered for the first
time).
260. The loop invariant is the following: mj is the maximum of A[1], . . . , A[j + 1].
261. The loop invariant for the inner loop is the following: after the jth iteration,
for all 1 ≤ i < j, A[i] ≤ A[j]. The loop invariant for the outer loop is the
following: after the jth iteration, for all n − j + 1 ≤ i ≤ n, for all k < i,
A[k] ≤ A[i].
5.5 SOLUTIONS
250. This correctness proof will make use of the fact that for all n ∈ IN,
2 n/2 + (n mod 2) = n. (5.1)
(Can you prove this?) We claim that if y, z ∈ IN, then add(y, z) returns the
value y + z. That is, when line 8 is executed, x = y + z. For each of the
identifiers, use a subscript i to denote the value of the identifier after the ith
iteration of the while-loop on lines 3–7, for i ≥ 0, with i = 0 meaning the
time immediately before the while loop is entered and immediately after the
statement on line 2 is executed. By inspection,
aj+1 = yj mod 2
bj+1 = zj mod 2
yj+1 = yj /2
zj+1 = zj /2
dj+1 = 2dj .
From line 6 of the algorithm, cj+1 is 1 iff at least two of aj+1 , bj+1 , and cj
are 1 (see Problem 93). Therefore,
cj+1 = (aj+1 + bj+1 + cj )/2 .
Note that d is added into x in line 5 of the algorithm only when an odd
number of a, b, and c are 1. Therefore,
xj+1 = xj + dj ((aj+1 + bj+1 + cj ) mod 2).
We will now prove the following loop invariant: For all natural numbers j ≥ 0,
(yj + zj + cj )dj + xj = y0 + z0 .
Sec. 5.5. Solutions 57
(yj + zj + cj )dj + xj = y0 + z0 .
By the preceding,
This ends the proof of the loop invariant. It remains to use it to prove that
the algorithm is correct. We need to prove that the algorithm terminates
with x containing the sum of y and z. First, we will prove that it terminates.
By inspection, the values of y and z are halved (rounding down if they are
odd) on every iteration of the loop. Therefore, they will eventually both have
value zero, and stay that way. At the first point at which y = z = 0, either
c will equal zero or c will be assigned zero on the next iteration of the loop.
Therefore, eventually y = z = c = 0, at which point the loop terminates.
Now we will prove that x has the correct value on termination. Suppose the
loop terminates after t iterations, for some t ≥ 0. By the loop invariant,
(yt + zt + ct )dt + xt = y0 + z0 .
multiply(2y, (z + 1)/2 ) + y
= 2y (z + 1)/2 + y (by the induction hypothesis)
= 2y(z/2) + y (since z is even)
= y(z + 1).
multiply(2y, (z + 1)/2 )
= 2y (z + 1)/2 (by the induction hypothesis)
= 2y(z + 1)/2 (since z is odd)
= y(z + 1).
5.6 COMMENTS
253. The hard part about this problem (and other similar problems in this section)
is that you will need to find your own loop invariant.
265. See Section 2.7 for the definition of Fibonacci numbers. This is the obvious
iterative algorithm.
276. See Section 2.7 for the definition of Fibonacci numbers. This is the obvious
recursive algorithm.
277. See Section 2.7 for the definition of Fibonacci numbers. This is a sneaky
recursive algorithm.
Chapter 6
Algorithm Analysis
Algorithm analysis usually means “give a big-O figure for the running time of an
algorithm.” (Of course, a big-Θ bound would be even better.) This can be done by
getting a big-O figure for parts of the algorithm and then combining these figures
using the sum and product rules for big-O (see Problems 173 and 176). As we will
see, recursive algorithms are more difficult to analyze than nonrecursive ones.
Another useful technique is to pick an elementary operation, such as additions,
multiplications, or comparisons, and observe that the running time of the algorithm
is big-O of the number of elementary operations. (To find the elementary operation,
just ignore the “book keeping” code and look for the point in the algorithm where
the real work is done.) Then, you can analyze the exact number of elementary
operations as a function of n in the worst case. This is often easier to deal with
because it is an exact function of n and you don’t have the messy big-O symbols to
carry through your analysis.
59
60 Chap. 6. Algorithm Analysis
function mystery(n)
1. r := 0;
2. for i := 1 to n − 1 do
3. for j := i + 1 to n do
4. for k := 1 to j do
5. r := r + 1
6. return(r)
281. What is the value returned by the following function? Express your
answer as a function of n. Give, using big-O notation, the worst-case running
time.
function pesky(n)
1. r := 0;
2. for i := 1 to n do
3. for j := 1 to i do
4. for k := j to i + j do
5. r := r + 1
6. return(r)
282. What is the value returned by the following function? Express your
answer as a function of n. Give, using big-O notation, the worst-case running
time.
function pestiferous(n)
1. r := 0;
2. for i := 1 to n do
3. for j := 1 to i do
4. for k := j to i + j do
5. for . := 1 to i + j − k do
6. r := r + 1
7. return(r)
function conundrum(n)
1. r := 0;
2. for i := 1 to n do
3. for j := i + 1 to n do
4. for k := i + j − 1 to n do
5. r := r + 1
6. return(r)
Sec. 6.1. Iterative Algorithms 61
6.1.2 Arithmetic
284. Analyze the algorithm for the addition of natural numbers in Prob-
lem 250. How many additions does it use (that is, how many times is line 4
executed) in the worst case?
285. Analyze the algorithm for incrementing a natural number in Problem 251.
How many additions does it use (that is, how many times is line 4 executed)
in the worst case?
286. Analyze the algorithm for the multiplication of natural numbers in Prob-
lem 252. How many additions does it use (that is, how many times is line 3
executed) in the worst case?
287. Analyze the algorithm for the multiplication of natural numbers in Prob-
lem 253. How many additions does it use (that is, how many times is line 3
executed) in the worst case?
288. Analyze the algorithm for the multiplication of natural numbers in Prob-
lem 254. How many additions does it use (that is, how many times is line 3
executed) in the worst case? You may assume that c ≥ 2 is an integer.
289. Analyze the algorithm for division and remaindering of natural numbers
in Problem 255. How many subtractions does it use (that is, how many times
is line 6 executed) in the worst case?
290. Analyze the algorithm for exponentiation in Problem 256. How many
multiplications does it use (that is, how many times is line 3 executed) in the
worst case?
291. Analyze the algorithm for exponentiation in Problem 257. How
many multiplications does it use (that is, how many times is line 3 executed)
in the worst case?
292. Analyze the algorithm for computing factorials in Problem 258. How
many multiplications does it use (that is, how many times is line 3 executed)
in the worst case?
6.1.3 Arrays
293. Analyze the array addition algorithm in Problem 259. How many ad-
ditions does it use (that is, how many times is line 3 executed) in the worst
case?
294. Analyze the array maximum algorithm in Problem 260. How many
comparisons does it use (that is, how many times is line 3 executed) in the
worst case?
62 Chap. 6. Algorithm Analysis
295. Analyze the bubblesort algorithm in Problem 261. How many compar-
isons does it use (that is, how many times is line 3 executed) in the worst
case?
297. Analyze the matrix multiplication algorithm in Problem 263. How many
multiplications does it use (that is, how many times is line 5 executed) in the
worst case?
298. Analyze the algorithm for evaluating a polynomial in Problem 264. How
many multiplications does it use (that is, how many times is line 3 executed)
in the worst case?
6.2.1 Arithmetic
300. Analyze the recursive algorithm from Problem 266. How
many additions does it use (that is, how many times is line 2 executed) in the
worst case?
302. Analyze the recursive algorithm for the addition of natural numbers in
Problem 268. How many times is line 3 executed in the worst case?
304. Analyze the recursive algorithm for the multiplication of natural numbers
in Problem 270. How many additions does it use (that is, how many times is
line 2 executed) in the worst case?
305. Analyze the recursive algorithm for the multiplication of natural numbers
in Problem 271. How many additions does it use (that is, how many times is
line 2 executed) in the worst case? You may assume that c ≥ 2 is an integer.
306. Analyze the recursive algorithm for exponentiation in Problem 272. How
many multiplications does it use (that is, how many times is line 2 executed)
in the worst case?
307. Analyze the recursive algorithm for computing factorials in Problem 273.
How many multiplications does it use (that is, how many times is line 2
executed) in the worst case?
6.2.2 Arrays
308. Analyze the recursive algorithm for computing the maximum value in an
array in Problem 274. How many comparisons does it use (that is, how many
times is line 2 executed) in the worst case?
309. Analyze the recursive algorithm that adds the values in an array in
Problem 275. How many additions does it use (that is, how many times is
line 2 executed) in the worst case?
64 Chap. 6. Algorithm Analysis
312. Analyze the recursive version of bubblesort in Problem 278. How many
comparisons does it use (that is, how many times is line 3 executed) in the
worst case?
313. Analyze the variant of quicksort in Problem 279. How many com-
parisons does it use (that is, how many times are lines 5 and 6 executed) in
the worst case?
6.4 HINTS
283 If you use the technique from the solution of Problem 280 blindly, you’ll get
the answer n(n−1)/2. This is wrong. More subtlety is necessary. The correct
answer is n(n + 2)(2n − 1)/24 if n is even, and something different if n is odd.
(You didn’t think I would give you the whole answer, did you?) You can
partially verify your solution in a few minutes by coding the algorithm as a
program and printing n and r for a few values of n.
284. I am looking for a big-O analysis of the running time, and an exact function
of n (with no big-O at the front) for the number of additions. You can do the
running time first, or if you are clever, you can observe that additions are the
“elementary operation,” count them exactly, and then put a big-O around
the function you get as answer to the second part of the question (reducing
it to the simplest form, of course) to give the answer for the first part of the
question.
Sec. 6.5. Solutions 65
300. Fibonacci numbers are involved (see Section 2.7 for definitions). Specifically,
you will need to apply the results from Problems 52 and 222.
6.5 SOLUTIONS
280. The for-loop on lines 4–5 has the same effect as r := r + j. Therefore, the
for-loop on lines 3–5 has the following effect:
3–5. for j := i + 1 to n do
r := r + j,
equivalently, r := r + nj=i+1 j. Now,
n
n
i
j= j− j = n(n + 1)/2 − i(i + 1)/2.
j=i+1 j=1 j=1
n−1
(n(n + 1)/2 − i(i + 1)/2)
i=1
n−1 n−1
(n − 1)n(n + 1) 1 2 1
= − i − i
2 2 i=1 2 i=1
= (n − 1)n(n + 1)/2 − n(n − 1)(2n − 1)/12 − n(n − 1)/4
(by Problems 1 and 2)
= n(n2 − 1)/3.
300. Let A(n) be the number of additions used in function g(n). Then A(0) =
A(1) = 0 and for all n > 1, A(n) = A(n − 1) + A(n − 2) + 1. This recurrence
relation is a little tricky to solve, but repeated substitution will do it:
= 2A(n − 2) + A(n − 3) + 1 + 1
= 2(A(n − 3) + A(n − 4) + 1) + A(n − 3) + 1 + 1
= 3A(n − 3) + 2A(n − 4) + 1 + 1 + 2
= 3(A(n − 4) + A(n − 5) + 1) + 2A(n − 4) + 1 + 1 + 2
= 5A(n − 4) + 3A(n − 5) + 1 + 1 + 2 + 3.
Hence, taking i = n − 1,
n−1
A(n) = Fn A(1) + Fn−1 A(0) + Fj
j=1
n−1
= Fj
j=1
= Fn+1 − 1 (see Problem 52).
The running time of procedure g(n) is clearly big-O of the number of addi-
tions, and is hence O(Fn+1 ) = O(1.62n ) (for the latter, see the comment to
Problem 222).
303. Let n be the number of bits in z, and A(n) be the number of additions
used in the multiplication of y by z. Then A(0) = 0 and for all n ≥ 1,
A(n) = A(n − 1) + 1. The solution to this recurrence is A(n) = n. (Can you
verify this by induction on n? If I didn’t tell you the answer, could you have
derived it for yourself?) The running time of procedure multiply is clearly
big-O of the number of additions, and is hence O(n).
6.6 COMMENTS
291. Which algorithm would you use in practice, the one in this problem or the
one from Problem 290?
311. Which algorithm would you use in practice, the one in this problem, the one
from Problem 299, or the one from Problem 310?
313. The worst-case analysis of quicksort is often presented in class, but if not,
then it makes a good exercise.
Chapter 7
Divide-and-Conquer
function maximum(x, y)
comment return maximum in S[x..y]
1. if y − x ≤ 1 then return(max(S[x], S[y]))
2. else
3. max1:=maximum(x, (x + y)/2 )
4. max2:=maximum( (x + y)/2 + 1, y)
5. return(max(max1,max2))
314. Prove that the algorithm is correct. You may assume that n is a
power of 2.
315. Write down a recurrence relation for the worst-case number of com-
parisons used by maximum(1, n). Solve this recurrence relation. You may
assume that n is a power of 2.
316. What is the running time of maximum(1, n)? Explain your answer.
67
68 Chap. 7. Divide-and-Conquer
usually described for n a power of 2. The following problems ask you to extend this
to arbitrary n.
procedure maxmin2(S)
comment computes maximum and minimum of S[1..n]
in max and min resp.
1. if n is odd then max:=S[n]; min:=S[n]
2. else max:=−∞; min:=∞
3. for i := 1 to n/2 do
4. if S[2i − 1] ≤ S[2i]
5. then small:=S[2i − 1]; large:=S[2i]
6. else small:=S[2i]; large:=S[2i − 1]
7. if small < min then min:=small
8. if large > max then min:=small
Complete the design of the following algorithm for performing integer multiplication
in time O(n1.63 ). (This is slower than the standard algorithm, but its verification
and analysis will test your abilities.) We use a technique similar to the standard
divide-and-conquer algorithm. Instead of dividing the inputs y and z into two parts,
we divide them into three parts. Suppose y and z have n bits, where n is a power
of 3. Break y into three parts, a, b, c, each with n/3 bits. Break z into three parts,
d, e, f , each with n/3 bits. Then,
r1 := ad
r2 := (a + b)(d + e)
r3 := be
r4 := (a + c)(d + f )
r5 := cf
r6 := (b + c)(e + f )
x := r1 24n/3 + (r2 − r1 − r3 )2n + (r3 + r4 − r1 − r5 )22n/3
+ (r6 − r3 − r5 )2n/3 + r5 .
const Precision=5000;
type Bit=0..1;
longinteger=packed array[1..Precision] of Bit;
carry:Bit;
begin{LongAdd}
carry:=0;
for index:=1 to Precision do
begin{for}
digitsum:=result[index]+summand[index]+carry;
carry:=digitsum div 2;
result[index]:=digitsum mod 2;
end;{for}
if carry>0 then LongError(LongOverflow)
end;{LongAdd}
Joe carefully implemented the naive multiplication algorithm from Problem 252
as procedure LongMultiply, and the divide-and-conquer multiplication algorithm as
procedure FastLongMultiply. Both of these procedures worked perfectly. However,
he found that LongMultiply was faster than FastLongMultiply, even when he tried
multiplying 2n − 1 by itself (which is guaranteed to make LongMultiply slow). Joe’s
data show that as n increases, FastLongMultiply becomes slower than LongMultiply
by more than a constant factor. Figure 7.1 shows his running times, and Figure 7.2
their ratio.
325. Show that the worst case for the naive integer multiplication algorithm
is, as claimed above, multiplying 2n − 1 by itself.
Complete the design of the following algorithm for performing integer multiplica-
tion using O(n1.585 /(log n)0.585 ) bit-operations. First, construct a table of all k-bit
products. To multiply two n-bit numbers, do the following. Perform the divide-
and-conquer algorithm with the base of recursion n ≤ k. That is, if n ≤ k, then
simply look up the result in the table. Otherwise, cut the numbers in half and
perform the recursion as normal.
328. Show that a table of all k-bit products can be constructed using O(k4k )
bit-operations.
329. Devise and solve a recurrence relation for the divide-and-conquer
multiplication algorithm when the base of recursion is T (n) = O(1) for all
n ≤ k. Your solution must be a function of both n and k.
330. Show that the best value of k is Θ(log n), and that this gives an
algorithm that uses O(n1.585 /(log n)0.585 ) bit-operations, as claimed.
Sec. 7.2. Integer Multiplication 71
60.00 Ratio
55.00
50.00
45.00
40.00
35.00
30.00
25.00
20.00
15.00
10.00
5.00
0.00
n x 103
0.00 0.50 1.00 1.50 2.00 2.50
333. The current record for matrix multiplication is O(n2.376 ), due to Cop-
persmith and Winograd [18]. How small would m need to be in Problem 332
in order to improve on this bound?
A = m2 + m3
B = t1 + m 5 + m 6
C = t2 − m 7
D = t2 + m 5 .
335. Analyze its running time. Is it likely to be faster or slower than the
standard version of Strassen’s algorithm in practice?
336. Show that the number of comparisons used by the binary search
algorithm when n is not necessarily a power of 2 is at most log n .
337. What is wrong with the following binary search algorithm?
function search(A, x, ., r)
comment find x in A[...r]
if . = r then return(.)
else
m := (. + r)/2
if x ≤ A[m]
then return(search(A, x, ., m))
else return(search(A, x, m, r))
338. Is the following algorithm for binary search correct? If so, prove it. If
not, give an example on which it fails.
function search(A, x, ., r)
comment find x in A[...r]
if . = r then return(.)
else
m := . + (r − . + 1)/2
if x ≤ A[m]
then return(search(A, x, ., m))
else return(search(A, x, m + 1, r))
The following problems ask you to modify binary search in some way. For each of
them, write pseudocode for the algorithm, devise and solve a recurrence relation for
the number of comparisons used, and analyze the running time of your algorithm.
341. Modify the binary search algorithm so that it splits the input not into two
sets of almost-equal sizes, but into two sets of sizes approximately one-third
and two-thirds.
342. Modify the binary search algorithm to give a ternary search algorithm
that splits the input not into two sets of almost-equal sizes, but into three sets
of sizes approximately one-third.
7.5 QUICKSORT
The following is a high-level version of Hoare’s quicksort algorithm. Suppose S is a
set of numbers.
function quicksort(S)
1. if |S| ≤ 1
2. then return(S)
3. else
4. Choose an element a from S
5. Let S1 , S2 , S3 be the elements of S that are respectively <, =, > a
6. return(quicksort(S1 ),S2 ,quicksort(S3 ))
344. The median of a set of n values is the n/2 th smallest value. Suppose
quicksort were to always pivot on the median value. How many comparisons
would be made then in the worst case?
345. Suppose quicksort were to always pivot on the n/3 th smallest value.
How many comparisons would be made then in the worst case?
n is a power of 3. If n = 3, then simply sort the 3 values and return the median.
Otherwise, divide the n items into n/3 groups of 3 values. Sort each group of
3, and pick out the n/3 medians. Now recursively apply the procedure to find a
pseudomedian of these values.
346. Let T (n) be the number of comparisons used by the preceding algorithm
for computing a pseudomedian. Write a recurrence relation for T (n), and
solve it exactly. Hence, show that the algorithm runs in time O(n).
347. Let E(n) be the number of values that are smaller than the value
found by the preceding algorithm. Write a recurrence relation for E(n), and
hence prove that the algorithm does return a pseudomedian. What is the
value of α?
348. Any odd number can be used instead of 3. Is there any odd number
less than 13 which is better? You may use the following table, which gives
the minimum number of comparisons S(n) needed to sort n ≤ 11 numbers.
n 1 2 3 4 5 6 7 8 9 10 11
S(n) 0 1 3 5 7 10 13 16 19 22 26
350. How many comparisons does it use to sort n values in the worst case?
Sec. 7.6. Towers of Hanoi 77
355. Show that the following algorithm solves the Towers of Hanoi
problem. Think of the pegs as being arranged in a circle, with clockwise moves
being from peg 1 to peg 2 to peg 3 to peg 1. If n is odd, then start by moving
the smallest disk one jump in the counterclockwise direction. Alternate be-
tween moving the smallest disk in this manner and the only other legal move
available. If n is even, then start by moving the smallest disk one jump in
the clockwise direction. Alternate between moving the smallest disk in this
manner and the only other legal move available.
356. Prove that any algorithm that solves the Towers of Hanoi problem must
make at least 2n − 1 moves.
procedure dfs(v)
1. mark v used
2. for each w ∈ V such that (v, w) ∈ E do
3. if w is unused then
4. mark (v, w) used
5. dfs(w)
The set of marked edges form a depth-first spanning tree of a connected component
of G.
Sec. 7.8. Applications 79
+ /
2 * 5
3 4
7.8 APPLICATIONS
The following problems ask you to solve new problems using divide-and-conquer.
For each of your algorithms, describe it in prose or pseudocode, prove it correct,
and analyze its running time.
80 Chap. 7. Divide-and-Conquer
Design an efficient (that is, linear time) algorithm to find the weighted me-
dian. (Note that the ki s need not be given in sorted order.) Prove that your
algorithm performs as claimed.
Sec. 7.9. Hints 81
7.9 HINTS
317. Separate the n numbers into disjoint subsets (finding the right decomposition
is crucial) and first run the known recursive algorithm on the subsets.
324. Observe that x is computed using only six multiplications of n/3-bit integers,
plus some additions and shifts. Solve the resulting recurrence relation.
326. Start by deriving and solving a recurrence relation for the number of times
that Joe’s procedure LongAdd is called by procedures LongMultiply and Fast-
LongMultiply.
355. Show that this algorithm makes exactly the same moves as the standard
divide-and-conquer algorithm.
357. No, it is not enough to use the standard algorithm and replace every move
from peg 1 to peg 3 (for example) with two moves to and from peg 2. That
may require you to place a large disk on top of a small one.
359. No, it is not enough to use the standard algorithm and replace every counter-
clockwise move with two clockwise moves. That may require you to place a
large disk on top of a small one. It is possible to devise a divide-and-conquer
82 Chap. 7. Divide-and-Conquer
algorithm for moving all of the disks one peg clockwise using (4n −1)/3 moves.
This can be used to solve the Towers of Hanoi problem (which is to move all of
the disks one peg counterclockwise) in 2(4√n − 1)/3 moves. The best algorithm
known to the author uses less than (1 + 3)n − 1 = O(2.73n ) moves.
363. Surprisingly, the standard algorithm solves this problem. You can prove it
by induction on n, but be careful about the bottom disk on each peg. The
induction hypothesis will have to be stronger than just the statement of the
required result. To get the idea, play with some examples.
368. It’s easy to do this in time O(n + e), but that is not what is asked for. A
problem from Chapter 2 is relevant.
369. In order to construct closed knight’s tours on square chessboards, you will
need a method for constructing tours on rectangular boards. If you are smart
enough, you will only need n × (n + 2) tours. You will also need some extra
structure in your tours to enable them to be joined at the corners (see Prob-
lem 48). You will have to devise an exhaustive search algorithm for finding
small tours for the base of your divide-and-conquer (see Problem 513), or you
can make use of the tours shown in Figure 7.4.
374. Use the adjacency list representation. Store for each v ∈ V an additional
field called its degree. Start by giving an algorithm that sets this field to the
correct value. Then write a divide-and-conquer algorithm that, given a vertex
v of degree less than k, deletes that vertex from the graph, and all vertices of
degree less than k that are adjacent to it. Apply this algorithm to solve the
required problem.
r r r r r r r r r r r r r r r r✟r r r❍r r r
r❍❆✟✁r❍✟❍r✟❆❍ ✟
✟r❍
✁r❍ ✟
✟
❍✟✁r❍✟r✟ ❆✟❍ ✟r✟
r✟ ❆❍✁r r❍ ❆✟ ✟
❍
✁r✟
❍✁r✟❍
❍✟ r ✁❆ ✟❍
✟r r✟❍
r❍✟ ✟❍
❆ r ✁r❍✟
❍✟ r❍ r✟
✁❆ ❍ ✁r
r❍ ❆r❍ r r❆✁ r r ❆r❍✁r❍ ✟ ✟
r r❍ ✟rr✟❍ r r✟r✟r❍r r✟r ✁✁ ❍✁ r❍❆ ✁❆❆✁r r✟ ✁ r r✁✁r✟
✁❆✁❆r✟ ❆r ✁r ❆✁
r ✁❆
r r✁ r✁ ❆✁❆r
r❍ ❍ ✟❍ r✟❍r✟ r✁❍ ✁r❍r r✟r✟ ❍ ✁ r❍
✟ ✟r❆❆r❍r❍
❆✟
❍r❍r rr❍rr❍✟
✁r
✟❍✟ ❍✟ ✟ ✁r❍✟ ❍
✟
❍r❍
r❆✁r ✁❆r❍❍ ✁r❆❍
✁r✁ r❆✁❆r r✟ ✁❆r ✁r❍ ✁r✟✟ r✁
❍✁ r
❆✟ r✁ ❆✁ ✁❆r
r❍ r r❆ r❍ ❍r✟
❍❍✟ ✟❆
r✁ ✁❆ ✁ r r r ❆
✁
❍ ✁❆r ✁✁r❍ ❆ ✁ ❆
✁ r✁❍✁ r❍✁❆r❍r❆❆✁r r❍✁r✁ r✟ ✁❆ r✟ r✟
✁ r✟❆r r ✁
❆ r r ❆✁❆r
r❍ ✁r✟ ❍✁r✟
❍
❆ r✟ ❍ ❍ ❍❍✟ ✁❍ ✟
✁ ✁r✟
❍ ✁r❍r✟✟ ❍r❆❍
✟
❍✁r❆ r❆
✁
✟ ❆r r✁✟ ✁❆r❍r✟
❆✟ ❆✁❆r❍ ❆r❆ r✁✟r ✁r✟ ✁✁❆r ❆r✁❍ ✁❍r❍r ❆r❆ r❍r✟ ✟rr ✁rr✁✟rr✁❆✟
❆r❍r❆✁❆r
r✁ r❆
❆ ✁ r✁
❆ ✁r
✟ r
✁
❆
❍ r❆
✟ r✁❆✟
r❍ ❆r❆ ❆r r✟ ❆✁r ❆✟r r✁❆❆r r✟ ✁❆r ✁r✟✁✁r✁✟r ✁r ❆✟
✟ ❆ r❍
r❍ ✁r ✁❆✟
r ✁r❍ ✁r✟ r✁✁❆r
❆✁r✟❆r✁❆r❆ r❍ ✟ r❆✁✁r✟r✟r✟ ❆r✟❆ r✁✟r✁ r✟
✟ ✟
❍ ❍✟ ✁❆r❍
r❆✟ ✟ ❆r❍❆r
✟r r r❍ r r ❆✁r ✟ ✁ r❆✟ ✁r r✟ ✁r ❆r✟ ✁ r✁❆ r✟
✁✁r r✟✁❍r✟r❍ ❆ r❆✁❍✁r
r❆✁✁❆r r ✁r
✟ ❆r
❆ ✟ ✟r❍
❆ ✁❆✁ r r✁❆✁❆r✟ r✟r✟r✟❆r ✁r r✟
✁❆ ✁r❆✁❆✁r ✁r❍ ❆✁❆r✟
✟r✟❍r❍ r ✁r❆ r✟
❆✁ r❆✟r❍r ❆r❆✁❆✁r
r❍ r❍
✟r❍✟
❍r✟ ❍r❍
✟ ✟r❍✟❍ ✟❆r✟ r✁❍❆✁ ✟❍ ✟❍
❍ ✟r✟❍ ❍
✟r✟ ❍❆r❍r❍ ✟ ❍❆r✟❍
✟r❍ ✟r❍ ❍r✟
✁ ✟ ❆✟❍✁ ❍r✟ ❆✟❍ ✟ r✟ ✁❍ ❆r ✟ r❍✟ r ❆✟
r❍✟ r❍✁ ❍✟ r r✁✟❍✁ ❍ ❆r ✁r✟ r✟
❆❍ ✟r❍✁ r✟ ❍ ✟❍❆r❍ ✟ ❍✁❆r
7.10 SOLUTIONS
314. Here is a full solution when n is not necessarily a power of 2. The size of
the problem is the number of entries in the array, y − x + 1. We will prove
by induction on n = y − x + 1 that maximum(x, y) will return the maximum
value in S[x..y]. The algorithm is clearly correct when n ≤ 2. Now suppose
n > 2, and that maximum(x, y) will return the maximum value in S[x..y]
whenever y − x + 1 < n. In order to apply the induction hypothesis to the
first recursive call, we must prove that (x + y)/2 − x + 1 < n. There are
two cases to consider. If y − x + 1 is even, then y − x is odd, and hence y + x
is odd. Therefore,
(The last inequality holds since n > 2.) If y − x + 1 is odd, then y − x is even,
and hence y + x is even. Therefore,
(The last inequality holds since n > 1.) In order to apply the induction
hypothesis to the second recursive call, we must prove that y − ( (x + y)/2 +
84 Chap. 7. Divide-and-Conquer
If y − x + 1 is odd, then
x+y x+y y−x+1 n 1
y− +1 +1=y− = − 1/2 = − < n.
2 2 2 2 2
Procedure maximum divides the array into two parts. By the induction hy-
pothesis, the recursive calls correctly find the maxima in these parts. There-
fore, since the procedure returns the maximum of the two maxima, it returns
the correct values.
315. Let T (n) be the number of comparisons used by maximum(x, y), where n =
y − x + 1 is the size of the array chunk processed. Then if n is a power of 2,
T (1) = 0, and for all n > 1 a power of 2, T (n) = 2T (n/2) + 1. Hence, using
the techniques of Chapter 4, T (n) = n − 1.
316. The running time of procedure maximum is clearly a constant times the num-
ber of comparisons. Hence, by the preceding, it runs in time O(n).
354. Let T (n) be the number of moves it takes to move n disks from peg i to peg
j. Clearly,
1 if n = 1
T (n) =
2T (n − 1) + 1 otherwise.
Hence, using the techniques of Chapter 4, T (n) = 2n − 1.
Let
• “D” denote moving the smallest disk in direction D,
• “D” denote moving the smallest disk in direction D, and
• “O” denote making the only other legal move.
Case 1. n + 1 is odd. Then n is even, and so by the induction hypothesis,
moving n disks in direction D uses moves:
DODO · · · OD.
(Note that by Problem 354, the number of moves is odd, so the preceding
sequence of moves ends with D, not O.) Hence, moving n+1 disks in direction
D uses
DODO· · · OD O DODO· · · OD ,
n disks n disks
as required.
Case 2. n + 1 is even. Then n is odd, and so by the induction hypothesis,
moving n disks in direction D uses moves
DODO · · · OD.
(Note that by Problem 354, the number of moves is odd, so the preceding
sequence of moves ends with D, not O.) Hence, moving n+1 disks in direction
D uses moves:
DODO· · · OD O DODO· · · OD ,
n disks n disks
as required. Hence, by induction, the claim holds for any number of disks.
7.11 COMMENTS
321. See also Problem 329.
330. So where does this algorithm get its speed from? The secret is in Problem 320.
Although it is faster than the standard divide-and-conquer, there is a faster
algorithm than this one (see Schönhage and Strassen [70]).
336. Binary search is normally described and analyzed for n a power of 2. This
problem asks you to extend this to any value of n.
354. According to legend, there is a set of 64 gold disks on 3 diamond needles being
solved by priests of Brahma. The Universe is supposed to end when the task is
complete. If done correctly, it will take T (64) = 264 − 1 = 1.84 × 1019 moves.
At one move per second, that’s 5.85 × 1011 years. More realistically, at 1
minute per move (since the largest disks must be very heavy) during working
hours, it would take 1.54 × 1014 years. The current age of the Universe is
estimated at ≈ 1010 years. Perhaps the legend is correct.
360. There is an algorithm that will do this using at most 2n − 1 moves. There is
even one that is provably optimal.
370. There is a faster algorithm for this problem. See Problem 409.
Dynamic Programming
378. Find three other orders in which the cost table m can be filled in
(ignoring the diagonal).
87
88 Chap. 8. Dynamic Programming
379. Prove that there are Ω(2n ) different orders in which the cost table m can
be filled in.
Fill in the cost table m in the dynamic programming algorithm for iterated matrix
products on the following inputs:
380. n = 4; r0 = 2, r1 = 5, r2 = 4, r3 = 1, r4 = 10.
381. n = 4; r0 = 3, r1 = 7, r2 = 2, r3 = 5, r4 = 10.
Find counterexamples to the following algorithms for the iterated matrix products
problem. That is, find n, r0 , r1 , . . . , rn such that, when the product of the matrices
with these dimensions is evaluated in the order given, the cost is higher than optimal.
387. Start by cutting the product in half, and then repeat the same thing
recursively in each half.
function knapsack(s1 , s2 , . . . , sn , S)
1. t[0, 0] :=true
2. for j := 1 to S do t[0, j] :=false
3. for i := 1 to n do
4. for j := 0 to S do
5. t[i, j] := t[i − 1, j]
6. if j − si ≥ 0 then t[i, j] := t[i, j] ∨ t[i − 1, j − si ]
7. return(t[n, S])
391. Fill in the table t in the dynamic programming algorithm for the knap-
sack problem on a knapsack of size 19 with rods of the following lengths:
15, 5, 16, 7, 1, 15, 6, 3.
Fill in the table t in the dynamic programming algorithm for the knapsack problem
on a knapsack of size 10 with rods of the following lengths:
392. 1, 2, 3, 4.
393. 5, 4, 3, 6.
394. 2, 2, 3, 3.
Find counterexamples to the following algorithms for the knapsack problem. That
is, find S, n, s1 , . . . , sn such that when the rods are selected using the algorithm
given, the knapsack is not completely full.
395. Put them in the knapsack in left to right order (the first-fit algorithm).
The optimal binary search tree problem is the following. Given the following
three sets of data,
construct a binary search tree (see also Section 11.2) that has the minimum number
of expected comparisons.
The standard dynamic programming algorithm for this problem is as follows. Let
Ti,j be the min-cost binary search tree for xi+1 , . . . , xj , and c[i, j] be the expected
number of comparisons for Ti,j .
function bst(p1 , p2 , . . . , pn , q0 , q1 , . . . , qn )
1. for i := 0 to n do
2. w[i, i] := qi ; c[i, i] := 0
3. for . := 1 to n do
4. for i := 0 to n − . do
5. j := i + .
6. w[i, j] := w[i, j − 1] + pj + qj
7. c[i, j] := mini<k≤j (c[i, k − 1] + c[k, j] + w[i, j])
8. return(c[0, n])
Fill in the cost table c in the dynamic programming algorithm for the optimal
binary search tree problem on the following inputs.
function floyd(C, n)
1. for i := 1 to n do
2. for j := 1 to n do
3. if (i, j) ∈ E
4. then A[i, j] := cost of edge (i, j)
5. else A[i, j] := ∞
6. A[i, i] := 0
7. for k := 1 to n do
8. for i := 1 to n do
9. for j := 1 to n do
10. if A[i, k] + A[k, j] < A[i, j] then A[i, j] := A[i, k] + A[k, j]
11. return(A)
401. Fill in the cost table A in Floyd’s algorithm on the graphs shown in
Figure 8.1.
402. Show that Floyd’s algorithm on an undirected graph chooses a mini-
mum cost edge (v, w) for each vertex v. Can an all-pairs shortest-path algo-
rithm be designed simply by choosing such a cheapest edge for each vertex?
What if all edge costs are different?
Modify Floyd’s algorithm to solve the following problems on directed graphs. An-
alyze your algorithms.
92 Chap. 8. Dynamic Programming
403. Determine the number of paths (possibly infinite) between each pair
of vertices.
404. Determine the number of even-length paths and of odd-length paths
between each pair of vertices.
405. Determine the number of shortest paths between each pair of vertices.
406. Given an n-vertex graph whose vertices are labeled with distinct in-
tegers from 1 to n, the label of each path is obtained by taking the labels of
vertices on the path in order. Determine the label and length of the lexico-
graphically first shortest path between each pair of vertices.
8.5 APPLICATIONS
The problems in this section ask you to apply dynamic programming in new situa-
tions. One way to proceed is to start by devising a divide-and-conquer algorithm,
replace the recursion with table lookups, and devise a series of loops that fill in the
table in the correct order.
t h i s i s a s t r i ngo f c ha r s
Total: 49 units
411. Find counterexamples to the following algorithms for the string-cutting prob-
lem introduced in Problem 410. That is, find the length of the string, and
several places to cut such that when cuts are made in the order given, the
cost in higher than optimal.
(a) Start by cutting the string as close to the middle as possible, and
then repeat the same thing recursively in each half.
94 Chap. 8. Dynamic Programming
t h i s i s a s t r i ngo f c ha r s
Total: 38 units
(b) Start by making (at most) two cuts to separate the smallest sub-
string. Repeat this until finished. Start by making (at most) two cuts
to separate the largest substring. Repeat this until finished.
412. There are two warehouses V and W from which widgets are to be
shipped to destinations Di , 1 ≤ i ≤ n. Let di be the demand at Di , for 1 ≤ i ≤
n, and rV , rW be the number of widgets available at V and W , respectively.
Assume that there are enough widgets available to fill the demand, that is,
that n
rV + rW = di . (8.1)
i=1
is minimized.
(a) Let gj (x) be the cost incurred when V has an inventory of x widgets,
and supplies are sent to destinations Di for all 1 ≤ i ≤ j in the optimal
Sec. 8.5. Applications 95
We wish to minimize the sum, over all lines except the last, of the cubes of
the number of blanks at the end of each line (for example, if there are k lines
with b1 , b2 , . . . , bk blanks at the ends, respectively, then we wish to minimize
b31 + b32 + · · · + b3k−1 ). Give a dynamic programming algorithm to compute this
value. Analyze your algorithm.
414. A “smart” terminal changes one string displayed on the screen into
another by a series of simple operations. The cursor is initially on the first
character of the string. The operations are shown in Table 8.1. For example,
one way to transform the string algorithm to the string altruistic is shown
in Figure 8.4.
96 Chap. 8. Dynamic Programming
Operation String
Algorithm
advance aLgorithm
advance alGorithm
replace with t alTorithm
advance altOrithm
delete altRithm
advance altrIthm
insert u altruIthm
advance altruiThm
insert s altruisThm
advance altruistHm
insert i altruistiHm
insert c altruisticHm
kill altruistic
There are many sequences of operations that achieve the same result. Sup-
pose that on a particular brand of terminal, the advance operation takes a
milliseconds, the delete operation takes d milliseconds, the replace operation
takes r milliseconds, the insert operation takes i milliseconds, and the kill
operation takes k milliseconds. So, for example, the sequence of operations to
transform algorithm into altruistic takes time 6a + d + r + 4i + k.
(a) Show that everything to the left of the cursor is a prefix of the new string,
and everything to the right of the cursor is a suffix of the old string.
(b) Devise a dynamic programming algorithm that, given two strings x[1..n]
and y[1..n], determines the fastest time needed to transform x to y.
Analyze your algorithm.
417. You have $1 and want to invest it for n months. At the beginning of
each month, you must choose from the following three options:
(a) Purchase a savings certificate from the local bank. Your money will be
tied up for one month. If you buy it at time t, there will be a fee of CS (t)
and after a month, it will return S(t) for every dollar invested. That is,
if you have $k at time t, then you will have $(k − CS (t))S(t) at time
t + 1.
(b) Purchase a state treasury bond. Your money will be tied up for six
months. If you buy it at time t, there will be a fee of CB (t) and after
six months, it will return B(t) for every dollar invested. That is, if you
have $k at time t, then you will have $(k − CB (t))B(t) at time t + 6.
(c) Store the money in a sock under your mattress for a month. That is, if
you have $k at time t, then you will have $k at time t + 1.
Suppose you have predicted values for S, B, CS , CB for the next n months. De-
vise a dynamic programming algorithm that computes the maximum amount
of money that you can make over the n months in time O(n).
422. Modify your dynamic programming algorithm for the warehouse prob-
lem (Problem 412) so that the values xi and yi for 1 ≤ i ≤ n can be recovered.
Write an algorithm for recovering these values from the information generated
by your modified dynamic programming algorithm.
423. Modify your dynamic programming algorithm for the printing prob-
lem (Problem 413) so that the number of words on each line can be recovered.
Write an algorithm for recovering these values from the information generated
by your modified dynamic programming algorithm.
424. Modify your dynamic programming algorithm for the fastest edit-
sequence problem (Problem 414) so that the fastest edit sequence can be
recovered. Write an algorithm for recovering this sequence of operations from
the information generated by your modified dynamic programming algorithm.
426. Modify your algorithm for Problem 416 to determine the sequence of
currencies i1 , i2 , . . . , ik that maximizes
427. Modify your algorithm for Problem 417 so that it determines the
optimal investment strategy.
After graduation, you start working for a company that has a hierarchical supervisor
structure in the shape of a tree, rooted at the CEO. The personnel office has ranked
each employee with a conviviality rating between 0 and 10. Your first assignment
is to design a dynamic programming algorithm to construct the guest list for the
office Christmas party. The algorithm has input consisting of the hierarchy tree,
and conviviality ratings for each employee. The output is to be a guest list that
maximizes the total conviviality, and ensures that for every employee invited to
the party, his or her immediate supervisor (that is, their parent in the tree) is not
invited.
429. You are not sure if the CEO should get invited to the party, but you
suspect that you might get fired if he is not. Can you modify your algorithm
to ensure that the CEO does get invited?
8.7 HINTS
378. Start by drawing a picture of the array entries on which m[i, j] depends. Make
sure that the array is filled in such way that the required entries are there
when they are needed.
409. Construct an array B[1..n] such that B[i] contains the length of the longest
ascending subsequence that starts with A[i], for 1 ≤ i ≤ n. Fill B from back
to front. From the information in B, you should be able to find the length of
the longest ascending subsequence. You should also, with a little thought, be
able to output a longest ascending subsequence (there may be more than one
of them, obviously) in time O(n).
413. In addition to the table of costs for subproblems, you will need to keep track of
the number of blanks in the last line of text in the solution of each subproblem.
The base case is more than just filling in the diagonal entries.
415. An O(nC) time algorithm can be constructed by exploiting the similarity be-
tween the coin changing problem and the knapsack problem (see Section 8.2).
It is easy to formulate an inferior algorithm that appears to run in time
O(nC 2 ), but with a little thought it can be analyzed using Harmonic numbers
to give a running time of O(C 2 log n), which is superior for C = o(n/ log n).
8.8 SOLUTIONS
383. 1 2 3 4 5
1 0 48 352 1020 1096
2 0 114 792 978
3 0 684 936
4 0 2394
5 0
407. For each nonterminal symbol S ∈ N , keep a Boolean table tS [1..n, 1..n]. We
100 Chap. 8. Dynamic Programming
will fill the tables so that for 1 ≤ i ≤ j ≤ n, tS [i, j] is true iff the string
xi xi+1 · · · xj is in the language generated by S. tS [i, j] is true iff either i = j
and S → xi , or there exists k such that i ≤ k < j and S → T U , where tT [i, k]
and tU [k + 1, j] are true.
function cfl(x1 x2 · · · xn )
1. for each nonterminal S do
2. for i := 1 to n do
3. tS [i, i] :=true iff S → xi
4. for d := 1 to n − 1 do
5. for i := 1 to n − d do
6. j := i + d
7. for each nonterminal S do
8. tS [i, j] :=false
9. for each production S → T U do
10. tS [i, j] := tS [i, j] ∨ j−1
k=i (tT [i, k] ∧ tU [k + 1, j])
11. return(tR [1, n]) where R is the root
Since there are O(1) nonterminals:
• The for-loop on lines 1–3 costs O(n).
• Line 10 costs O(n) (a single for-loop).
• The for-loop on lines 7–10 costs O(n).
• The for-loop on lines 5–10 costs O(n2 ).
• The for-loop on lines 4–10 costs O(n3 ).
Therefore, the algorithm runs in time O(n3 ).
8.9 COMMENTS
380. Although the easiest way to do this is to write a program to do it for you,
I recommend strongly that you do it by hand instead. It will help you to
understand the subtleties of the algorithm. However, there is nothing wrong
with using such a program to verify that your answer is correct.
415. In some cases, a greedy algorithm for this problem is superior to dynamic
programming (see Problem 471). See also Problem 95.
Chapter 9
Greedy Algorithms
A greedy algorithm starts with a solution to a very small subproblem and augments
it successively to a solution for the big problem. This augmentation is done in
a “greedy” fashion, that is, paying attention to short-term or local gain, without
regard to whether it will lead to a good long-term or global solution. As in real life,
greedy algorithms sometimes lead to the best solution, sometimes lead to pretty
good solutions, and sometimes lead to lousy solutions. The trick is to determine
when to be greedy.
s := S; i := 1
while si ≤ s do
xi := 1;s := s − si ; i := i + 1
xi := s/si
for j := i + 1 to n do xj := 0
101
102 Chap. 9. Greedy Algorithms
432. Show the contents of arrays D and P after each iteration of the main
for-loop in Dijkstra’s algorithm executed on the graphs shown in Figure 9.1,
with source vertex 1. Use this information to list the shortest paths from
vertex 1 to each of the other vertices.
433. Show that an arbitrary collection of single-source shortest paths in
an undirected graph does not necessarily form a spanning tree.
434. Show that the single-source shortest paths constructed by Dijkstra’s
algorithm on a connected undirected graph form a spanning tree. (See also
Problem 433.)
435. Is the spanning tree formed by Dijkstra’s algorithm on a connected
undirected graph a min-cost spanning tree? (See also Problem 434.)
Sec. 9.3. Min-Cost Spanning Trees 103
437. Suppose you are the programmer for a travel agent trying to de-
termine the minimum elapsed time from DFW airport to cities all over the
world. Since you got an A in my algorithms course, you immediately think
of using Dijkstra’s algorithm on the graph of cities, with an edge between
two cities labeled with the flying time between them. But the shortest path
in this graph corresponds to shortest flying time, which does not include the
time spent waiting at various airports for connecting flights. Desperate, you
instead label each edge from city x to city y with a function fxy (t) that maps
from your arrival time at x to your arrival time at y in the following fashion:
If you are in x at time t, then you will be in y at time fxy (t) (including waiting
time and flying time). Modify Dijkstra’s algorithm to work under this new
formulation, and explain why it works. State clearly any assumptions that
you make (for example, it is reasonable to assume that fxy (t) ≥ t).
1. T := ∅; V1 := {1}; Q := empty
2. for each w ∈ V such that (1, w) ∈ E do
3 insert(Q, (1, w))
4. while |V1 | < n do
5. e :=deletemin(Q)
6. Suppose e = (u, v), where u ∈ V1
7. if v ∈ V1 then
8. T := T ∪ {e}
9. V1 := V1 ∪ {v}
10. for each w ∈ V such that (v, w) ∈ E do
11. insert(Q, (v, w))
1. T := ∅; F := ∅
2. for each vertex v ∈ V do F := F ∪ {{v}}
3. Sort edges in E in ascending order of cost
4. while |F | > 1 do
5. (u, v) := the next edge in sorted order
6. if find(u) = find(v) then
7. union(u, v)
8. T := T ∪ {(u, v)}
438. Draw the spanning forest after every iteration of the main while-loop in
Kruskal’s algorithm on each of the graphs in Figure 9.1, ignoring the directions
on the edges.
439. Draw the spanning forest after every iteration of the main while-loop in
Prim’s algorithm on each of the graphs in Figure 9.1, ignoring the directions
on the edges.
440. Draw the spanning forest after every iteration of the main while-loop in
Kruskal’s algorithm on the graph shown in Figure 9.2. Number the edges in
order of selection.
441. Draw the spanning forest after every iteration of the main while-loop
in Prim’s algorithm on the graph shown in Figure 9.2. Number the edges in
order of selection.
442. Would Prim’s algorithm start by making the choice of edges shown in
Figure 9.4 for the graph shown in Figure 9.3?
443. Would Kruskal’s algorithm start by making the choice of edges shown in
Figure 9.5 on the graph shown in Figure 9.3?
Sec. 9.3. Min-Cost Spanning Trees 105
6 8
2 2 9 11
16 4 11
13 5 20
1
12 10 6 12
7 3 5
10 4 19
1 8 14
15 18
3 9
21 7 17
3
5
2 4
2 12
1 4 5
6 10
1 8 11
9
7
6 7
3
3 3 3 3
5 5 5 5
2 4 2 4 2 4 2 4
2 12 2 12 2 12 2 12
1 4 5 1 4 5 1 4 5 1 4 5
6 10 6 10 6 10 6 10
1 8 11 1 8 11 1 8 11 1 8 11
9 9 9 9
7 7 7 7
6 7 6 7 6 7 6 7
3 3 3 3
3 3 3 3
5 5 5 5
2 4 2 4 2 4 2 4
2 12 2 12 2 12 2 12
4 5 1 4 5 1 4 5 1 4 5
1
6 10 6 10 6 10 6 10
1 8 11 1 8 11 1 8 11 1 8 11
9 9 9 9
7 7 7 7
6 7 6 7 6 7 6 7
3 3 3 3
3 3 3 3
5 5 5 5
2 4 2 4 2 4 2 4
2 12 2 12 2 12 2 12
1 4 5 1 4 5 1 4 5 1 4 5
6 10 6 10 6 10 6 10
1 8 11 1 8 11 1 8 11 1 8 11
9 9 9 9
7 7 7 7
6 7 6 7 6 7 6 7
3 3 3 3
444. Could a greedy algorithm for min-cost spanning trees start by making
the choice of edges shown in Figure 9.6 on the graph shown in Figure 9.3?
445. Could a greedy algorithm for min-cost spanning trees start by making
the choice of edges shown in Figure 9.7 on the graph shown in Figure 9.3?
446. Show that Prim’s algorithm and Kruskal’s algorithm always con-
struct the same min-cost spanning tree on a connected undirected graph in
which the edge costs are distinct.
447. Show that Prim’s algorithm and Kruskal’s algorithm always con-
struct the same min-cost spanning tree on a connected undirected graph in
which no pair of edges that share a vertex have the same cost.
3 3 3 3
1 1 1 1
2 4 2 4 2 4 2 4
2 12 2 12 2 12 2 12
4 5 5 4 5 5 4 5 5 4 5
5
6 10 6 10 6 10 6 10
1 8 2 1 8 2 1 8 2 1 8 2
9 9 9 9
7 7 7 7
6 7 6 7 6 7 6 7
3 3 3 3
450. Here is a version of Prim’s algorithm that is different from the one
given earlier in this section. Is it correct? If so, prove it correct. If not, explain
why it is wrong and give a graph on which it fails.
1. T := ∅; V1 := {1}; Q := empty
2. for each w ∈ V such that (1, w) ∈ E do
3 insert(Q, (1, w))
4. while |V1 | < n do
5. e :=deletemin(Q)
6. Suppose e = (u, v), where u ∈ V1
7. T := T ∪ {e}
8. V1 := V1 ∪ {v}
9. for each w ∈ V such that (v, w) ∈ E do
10. if w ∈ V1 then insert(Q, (v, w))
In the vanilla version of Prim’s algorithm and Kruskal’s algorithm, it is not specified
what to do if a choice must be made between two edges of minimum cost.
451. Prove that for Prim’s algorithm, it doesn’t matter which of the two edges
is chosen.
452. Prove that for Kruskal’s algorithm, it doesn’t matter which of the two
edges is chosen.
453. Suppose that there are exactly two edges that have the same cost. Does
Prim’s algorithm construct the same min-cost spanning tree regardless of
which edge is chosen?
108 Chap. 9. Greedy Algorithms
454. Suppose that there are exactly two edges that have the same cost. Does
Kruskal’s algorithm construct the same min-cost spanning tree regardless of
which edge is chosen?
455. Suppose Prim’s algorithm and Kruskal’s algorithm choose the lexi-
cographically first edge first. That is, if a choice must be made between two
distinct edges, e1 = (u1 , v1 ) and e2 = (u2 , v2 ) of the same cost, where u1 < v1
and u2 < v2 , then the following strategy is used. Suppose the vertices are
numbered 1, 2, . . . , n. Choose e1 if u1 < u2 , or u1 = u2 and v1 < v2 . Other-
wise, choose e2 . Prove that under these conditions, both algorithms construct
the same min-cost spanning tree.
456. Design an algorithm for the max-cost spanning tree.
A directed spanning tree of a directed graph is a spanning tree in which the directions
on the edges point from parent to child.
457. Suppose Prim’s algorithm is modified to take the cheapest edge di-
rected out of the tree.
(a) Does it find a directed spanning tree of a directed graph?
(b) If so, will it find a directed spanning tree of minimum cost?
458. Suppose Kruskal’s algorithm is used to construct a directed spanning
tree.
(a) Does it find a directed spanning tree of a directed graph?
(b) If so, will it find a directed spanning tree of minimum cost?
459. Consider the following algorithm. Suppose Prim’s algorithm is modi-
fied as described in Problem 457, and then run n times, starting at each vertex
in turn. The output is the cheapest of the n directed spanning trees.
(a) Does this algorithm output a directed spanning tree of a directed graph?
(b) If so, will it find a directed spanning tree of minimum cost?
Consider the following algorithm for finding a min-cost spanning tree. Suppose the
graph has vertices numbered 1, 2, . . . , n. Start with a forest containing n trees, each
consisting of a single vertex. Repeat the following until a single tree remains. For
each vertex v = 1, 2, . . . , n in turn, consider the cheapest edge with at least one
endpoint in the tree that contains v. If it has both endpoints in the same tree, then
discard it. If not, then add it to the spanning forest.
460. Is this algorithm correct? Justify your answer.
461. Explain how to implement this algorithm in time O(e log n).
Sec. 9.4. Traveling Salesperson 109
462. Show that this greedy algorithm may fail to find a cycle.
463. Show that even when the greedy algorithm finds a cycle, it may not be
a Hamiltonian cycle, even when one exists in the graph.
464. Suppose the greedy algorithm does find a Hamiltonian cycle. Prove
that it is a Hamiltonian cycle of minimum cost.
465. Suppose the edge costs are distinct, and the greedy algorithm does
find a Hamiltonian cycle. Show that it is the only min-cost Hamiltonian cycle
in the graph.
466. Here is another greedy algorithm for the traveling salesperson prob-
lem. Simply run the vanilla greedy algorithm described above n times, start-
ing at each vertex in turn, and output the cheapest Hamiltonian cycle found.
Suppose this greedy algorithm does output a Hamiltonian cycle. Can it be a
different cycle from the one found by the vanilla greedy algorithm? Justify
your answer.
Another greedy algorithm for the traveling salesperson problem proceeds as follows.
Start at node number 1, and move along the edge of minimum cost that leads to
an unvisited node, repeating this until the algorithm arrives at a vertex from which
all edges lead to visited nodes.
467. Show that this greedy algorithm may fail to find a cycle.
468. Suppose the graph has a Hamiltonian cycle. Will this algorithm find it?
469. Suppose this greedy algorithm finds a Hamiltonian cycle. Is it neces-
sarily a Hamiltonian cycle of minimum cost? Justify your answer.
9.5 APPLICATIONS
The problems in this section ask you to apply the greedy method in new situations.
One thing you will notice about greedy algorithms is that they are usually easy to
design, easy to implement, easy to analyze, and they are very fast, but they are
almost always difficult to prove correct.
110 Chap. 9. Greedy Algorithms
function cover(V, E)
1. U := ∅
2. repeat
3. let v ∈ V be a vertex of maximum degree
4. U := U ∪ {v}; V := V − {v}
5. E := E − {(u, w) such that u = v or w = v}
6. until E = ∅
7. return(U)
Does this algorithm always generate a minimum vertex cover? Justify your
answer.
in order of deadline (the early deadlines before the late ones). Show that if a
feasible schedule exists, then the schedule produced by this greedy algorithm
is feasible.
9.6 HINTS
430. Show that a greedy solution (x1 , x2 , . . . , xn ) can be changed to an optimal
solution (y1 , y2 , . . . , yn ) without affecting its length by doing the following for
i = 1, 2, . . . , n. Change xi to yi and adjust the remaining xi+1 , . . . , xn so that
the length remains the same.
446. This can be proved directly by induction. But a more insightful way to do it
is to prove that the min-cost spanning tree is unique under these conditions.
471. For part (a), just play around until you get a counterexample. For part (b),
show that there is only one solution in which the number of coins of each type
is less than c, and that the min-cost solution must have this form. These two
facts should enable you to easily prove that the greedy algorithm does the
right thing.
9.7 SOLUTIONS
430. Let X = (x1 , x2 , . . . , xn ) be the solution generated by the greedy algorithm.
If all the xi are 1, then the solution is clearly optimal (it is the only solution).
Otherwise, let j be the smallest number such that xj = 1. From the algorithm,
xi = 1 for 1 ≤ i < j
0 ≤ xj < 1
xi = 0 for j < i ≤ n.
Therefore,
j
xi si = S. (9.1)
i=1
It must be the case that yk < xk . To see this, consider the three possible
cases:
Case 1. k < j. Then xk = 1. Therefore, since xk = yk , yk must be smaller
than xk .
Case 2. k = j. By the definition of k, xk = yk . If yk > xk ,
n
k−1
n
yi si = yi si + yk sk + yi si
i=1 i=1 i=k+1
k−1
n
= xi si + yk sk + yi si
i=1 i=k+1
n
= S + (yk − xk )sk + yi si (by Equation 9.1, since k = j)
i=k+1
> S,
n
j
n
yi si = yi si + yi si
i=1 i=1 i=j+1
j
n
= xi s i + yi si
i=1 i=j+1
n
= S+ yi si (by Equation 9.1)
i=j+1
> S.
This is not possible, hence Case 3 can never happen. In the other two cases,
yk < xk as claimed. Now that we have established that yk < xk , we can
return to the proof. Suppose we increase yk to xk , and decrease as many of
yk+1 , . . . , yn as necessary to make the total length remain at S. Call this new
solution Z = (z1 , z2 , . . . , zn ). Then,
Therefore,
n
k−1
n
zi wi = zi wi + zk wk + zi wi
i=1 i=1 i=k+1
k−1
n
= yi wi + zk wk + zi wi
i=1 i=k+1
n n n
= yi wi − yk wk − yi wi + zk wk + zi wi
i=1 i=k+1 i=k+1
n n
= yi wi + (zk − yk )wk + (zi − yi )wi
i=1 i=k+1
n n
= yi wi + (zk − yk )sk wk /sk + (zi − yi )si wi /si
i=1 i=k+1
n n
≤ yi wi + (zk − yk )sk wk /sk + (zi − yi )si wk /sk
i=1 i=k+1
(by Equation 9.4 and density)
n
n
= yi wi + (zk − yk )sk + (zi − yi )si wk /sk
i=1 i=k+1
n
= yi wi (by Equation 9.4).
i=1
n
n
zi wi < yi wi .
i=1 i=1
Hence, Y and Z have the same weight. But Z looks “more like” X than Y
does, in the sense that the first k entries of Z are the same as X, whereas
only the first k − 1 entries of Y are the same as X. Repeating this proce-
dure sufficiently often transforms Y into X and maintains the same weight.
Therefore, X must have minimal weight.
k
To retrieve the kth file on the tape, fik , costs j=1 mij . Therefore, the cost
of retrieving them all is
n
k n
m ij = (n − k + 1)mik . (9.5)
k=1 j=1 k=1
which is the right-hand side of Equation (9.5). The greedy algorithm picks
files fi in nondecreasing order of their size mi . It remains to prove that this
is the minimum cost permutation.
Claim: Any permutation in nondecreasing order of mi ’s has minimum cost.
Proof: Let Π = (i1 , i2 , . . . , in ) be a permutation of 1, 2, . . . , n that is not
in nondecreasing order of mi ’s. We will prove that it cannot have minimum
cost. Since mi1 , mi2 , . . . , min is not in nondecreasing order, there must exist
1 ≤ j < n such that mij > mij+1 . Let Π be permutation Π with ij and ij+1
swapped. The cost of permutation Π is
n
C(Π) = (n − k + 1)mik
k=1
j−1
= (n − k + 1)mik + (n − j + 1)mij + (n − j)mij+1
k=1
n
+ (n − k + 1)mik .
k=j+2
Hence,
9.8 COMMENTS
436, 456. These problems were suggested to the author by Farhad Shahrokhi in
1993.
471. See also Problem 95. In general, a dynamic programming algorithm is the
best solution to this problem (see Problem 415). This question is designed
to illustrate the point that a greedy algorithm is better in some (but not all)
cases.
Chapter 10
Exhaustive Search
Often it appears that there is no better way to solve a problem than to try all pos-
sible solutions. This approach, called exhaustive search, is almost always slow, but
sometimes it is better than nothing. It may actually be practical in real situations if
the problem size is small enough. The most commonly used algorithmic technique
used in exhaustive search is divide-and-conquer (see Chapter 7).
The approach taken to exhaustive search in this chapter is very different from
that taken by most textbooks. Most exhaustive search problems can be reduced to
the problem of generating simple combinatorial objects, most often strings, permu-
tations, and combinations. Sections 10.1–10.4 cover some simple algorithms that
form our basic tools. Section 10.6 covers the application of these tools to various
interesting exhaustive search problems. Section 10.5 compares exhaustive search
algorithms to backtracking. Sometimes backtracking is faster, sometimes not.
10.1 STRINGS
A bit-string is an array of n bits. Bit-strings are often used to represent sets
(the term usually used is the bit-vector representation of a set). The following
is a recursive algorithm for generating bit-strings. We assume the existence of a
procedure process(C) that processes a bit-string stored in an array C. Our aim is
to process all bit-strings of a fixed length.
procedure binary(m)
comment process all binary strings of length m
1. if m = 0 then process(C) else
2. C[m] := 0; binary(m − 1)
3. C[m] := 1; binary(m − 1)
474. Show that the algorithm is correct, that is, that a call to binary(n)
processes all binary strings of length n.
475. How many assignments to array C are made (that is, how often are
lines 2 and 3 executed)? Hence, deduce the running time of the procedure.
116
Sec. 10.1. Strings 117
The following algorithm generates all strings of n bits in an array C[1..n]. A sentinel
is used at both ends of C, so C is actually an array indexed from 0 to n + 1. The
operation “complement a bit” used in line 8 means change it from a 0 to a 1, or
vice-versa. It works by treating C as a binary number and repeatedly incrementing
it through all 2n possible values.
1. for i := 0 to n + 1 do C[i] := 0
2. while C[n + 1] = 0 do
3. process(C)
4. increment(C)
procedure increment(C)
5. i := 1
6. while C[i − 1] = 0 do
7. Complement C[i]
8. i := i + 1
476. Prove that the algorithm is correct, that is, that procedure process(C)
is called once with C[1..n] containing each string of n bits.
477. Prove that for all n ≥ 1, the number of times that a bit of C is comple-
mented (that is, the number of times that line 7 is executed) is 2n+1 − (n + 2).
Hence, deduce the running time of the algorithm.
procedure generate(n)
1. if n = 0 then process(b)
2. else
3. generate(n − 1)
4. Complement b[n]
5. generate(n − 1)
478. Prove that this algorithm does generate the 2n bit-strings of length
n in minimal-change order.
479. Prove that for all n ≥ 1, the number of times that a bit of b is comple-
mented (that is, the number of times that line 4 is executed) is 2n − 1. Hence,
deduce the running time of the algorithm.
118 Chap. 10. Exhaustive Search
A k-ary string is a string of digits drawn from {0, 1, . . . , k − 1}. For example, if
k = 2, then a k-ary string is simply a binary string.
480. Design an optimal exhaustive search algorithm for generating all k-ary
strings of length n in lexicographic order in time O(kn ). For example, if k = 3
and n = 4, then your algorithm should generate the strings
0000, 0001, 0002, 0010, 0011, 0012, 0020, 0021, 0022, 0100, . . . , 2221, 2222.
482. Analyze the running time of your algorithm for Problem 480.
485. Analyze the running time of your algorithm for Problem 483.
10.2 PERMUTATIONS
Consider the problem of exhaustively generating all permutations of n things. Sup-
pose A[1..n] contains n distinct values. The following algorithm generates all permu-
tations of A[1..n] in the following sense: A call to procedure permute(n) will result
in procedure process(A) being called once with A[1..n] containing each permutation
of its contents.
procedure permute(n)
1. if n = 1 then process(A) else
2. B := A
3. permute(n − 1)
4. for i := 1 to n − 1 do
5. swap A[n] with A[1]
6. permute(n − 1)
7. A := B cyclically shifted one place right
486. Prove that this algorithm calls process(A) with A containing each
permutation of n entries exactly once.
Sec. 10.2. Permutations 119
Consider the following algorithm for generating all permutations. To generate all
permutations of any n distinct values in A[1..n], call permute(1).
procedure permute(i)
1. if i = n then process(A) else
4. permute(i + 1)
2. for j := i + 1 to n do
3. swap A[i] with A[j]
4. permute(i + 1)
5. swap A[i] with A[j]
procedure oddpermute(n)
5. if n = 1 then process else
6. evenpermute(n − 1)
7. for i := 1 to n − 1 do
8. swap A[1] and A[n]
9. evenpermute(n − 1)
procedure evenpermute(n)
10. if n = 1 then process else
11. oddpermute(n − 1)
12. for i := 1 to n − 1 do
13. swap A[i] and A[n]
14. oddpermute(n − 1)
120 Chap. 10. Exhaustive Search
491. Show that the algorithm is correct, that is, it generates all
permutations in minimal-change order.
492. Show that Heap’s algorithm uses the minimum number of swaps,
(that is, n! − 1 swaps). What is its running time?
10.3 COMBINATIONS
Consider the problem of generating all combinations of r things chosen from n, for
some 0 ≤ r ≤ n. We will, without loss of generality, assume that the n items to be
chosen from are the natural numbers 1, 2, . . . , n.
The following algorithm generates all combinations of r things drawn from n
in the following sense: Procedure process(A) is called once with A[1..r] containing
each combination. To use it, call procedure choose(1, r).
procedure choose(b, c)
comment choose c elements out of b..n
1. if c = 0 then process(A)
2. else for i := b to n − c + 1 do
3. A[c] := i
4. choose(i + 1, c − 1)
494. Prove that the combinations generated by this algorithm are in descend-
ing order, that is, when process(A) is called, A[i] > A[i + 1] for all 1 ≤ i < n.
n
496. Show that for all 1 ≤ r ≤ n/2, T (n, r) ≤ 2 r − r. Hence, deduce
that the algorithm is optimal for r ≤ n/2.
procedure up(b, c)
1. if c = 0 then process(a)
2. else for i := b to n do
3. a[r − c + 1] := i
4. if i is odd then up(i + 1, c − 1) else down(i + 1, c − 1)
procedure down(b, c)
5. if c = 0 then process(a)
6. else for i := n downto b do
7. a[r − c + 1] := i
8. if i is odd then up(i + 1, c − 1) else down(i + 1, c − 1)
497. Prove that the algorithm is correct, that is, that procedure
process(a) is called with a containing a sequence of combinations in minimal-
change order.
498. How many assignments to a does the algorithm make? Hence, deduce
its running time.
499. Prove that the combinations generated by this algorithm are in ascending
order, that is, when process(a) is called, a[i] < a[i + 1] for all 1 ≤ i < n.
500. Let M (n) be the number of perfect matchings of n values. Show that if
n is even, then
n/2
M (n) = (2i − 1).
i=1
122 Chap. 10. Exhaustive Search
Suppose you have m marbles and n jars, each of which can hold at most c marbles.
504. Use divide-and-conquer to design an algorithm that generates all of
the ways of distributing the marbles into the jars so that no jar holds more
than c marbles, where m ≤ cn. Your algorithm must print the number of
marbles in each jar for each of the different ways of distributing the marbles.
So, for example, with m = 10, n = 3, and c = 5, the output of your algorithm
would be
0 5 5 3 2 5 4 3 3 5 3 2
1 4 5 3 3 4 4 4 2 5 4 1
1 5 4 3 4 3 4 5 1 5 5 0
2 3 5 3 5 2 5 0 5
2 4 4 4 1 5 5 1 4
2 5 3 4 2 4 5 2 3
10.5 BACKTRACKING
Backtracking is a type of exhaustive search in which the combinatorial object is
constructed recursively, and the recursion tree is pruned, that is, recursive calls are
not made when the part of the current object that has already been constructed
cannot lead to a valid or optimal solution. For example, the recursive algorithm for
generating bit-strings from Section 10.1 can be modified easily to give a backtracking
algorithm for generating all bit-strings of length n with k or fewer bits. To use the
algorithm, call binary(n, 0).
procedure binary(m, c)
comment process all binary strings of length m
1. if m = 0 then process(C) else
2. C[m] := 0; binary(m − 1, c)
3. if c < k then C[m] := 1; binary(m − 1, c + 1)
Sec. 10.6. Applications 123
Note that the pruning is done in line 3: If enough ones have been put into C
already, then the recursion tree is pruned to prevent more ones from being put in.
507. Show that the algorithm is correct, that is, that a call to binary(n, 0)
processes all binary strings of length n with k or fewer bits.
508. How many assignments to array C are made (that is, how often are
lines 2 and 3 executed)? Hence, deduce the running time of the procedure. Is
it asymptotically optimal?
509. Show that the algorithm is correct, that is, that a call to permute(n)
processes all permutations of length n.
510. How many assignments to array A are made (that is, how often are
lines 3 and 4 executed)? Hence, deduce the running time of the procedure. Is
it asymptotically optimal?
10.6 APPLICATIONS
The problems in this section are to be solved by reduction to one of the standard
exhaustive search algorithms from Sections 10.1–10.4. Some of them require back-
tracking, and some do not.
511. A clique is an induced complete subgraph. The clique problem is the
following: Given a graph G and a natural number r, does G contain a clique
onr vertices?
Devise an algorithm for the clique problem that runs in time
O r 2 nr . Briefly describe how to improve the running time by a factor
of Ω(r).
124 Chap. 10. Exhaustive Search
514. Suppose you have n students in a class and wish to assign them into
teams of two (you may assume that n is even). You are given an integer array
P [1..n, 1..n] in which P [i, j] contains the preference of student i for working
with student j. The weight of a pairing of student i with student j is defined to
be P [i, j] · P [j, i]. The pairing problem is the problem of pairing each student
with another student in such a way that the sum of the weights of the pairings
is maximized. Design an algorithm for the pairing problem that runs in time
proportional to the number of pairings. Analyze your algorithm.
515. The 15-puzzle, invented by Sam Loyd, has been a source of enter-
tainment for over a century. The puzzle consists of 15 square tiles numbered
1 through 15, arranged in row-major order in a square grid with a one-tile
gap. The aim is to randomize the puzzle and then attempt to return it to
the initial configuration by repeatedly sliding an adjacent tile into the gap
(see, for example, Figure 10.1). The general form of this puzzle, based on an
n × n grid, is called the (n2 − 1)-puzzle. Design an algorithm that, given some
configuration of the (n2 − 1)-puzzle, finds a sequence of moves that return the
puzzle to the initial configuration, if such a sequence of moves exists. Your
algorithm should run in time O(n2 + 4k ), where k is the number of moves
required for solution of the puzzle.
516. The following puzzle is called Hi-Q. You are given a board in which 33
holes have been drilled in the pattern shown in Figure 10.2. A peg (indicated
by a square) is placed in every hole except the center one (indicated by a
circle). Pegs are moved as follows. A peg may jump over its neighbor in the
horizontal or vertical direction if its destination is unoccupied. The peg that
is jumped over is removed from the board. The aim is to produce a sequence
of jumps that removes all pegs from the board except for one. The generalized
HI-Q puzzle has n(n + 4m) holes arranged in a symmetrical cross-shape as
shown in Figure 10.3, for any m and any odd n. Design an algorithm that
Sec. 10.6. Applications 125
6 2 4 6 2 4 1 6 2 4 1 6 2 4 1 2 4
1 5 3 8 1 5 3 8 5 3 8 5 3 8 5 6 3 8
13 10 7 11 13 10 7 11 13 10 7 11 13 10 7 11 13 10 7 11
14 9 15 12 14 9 15 12 14 9 15 12 14 9 15 12 14 9 15 12
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 4
5 6 7 8 5 6 7 8 5 6 7 8 5 6 8 5 6 3 8
13 9 10 11 13 10 11 13 10 11 13 10 7 11 13 10 7 11
14 15 12 14 9 15 12 14 9 15 12 14 9 15 12 14 9 15 12
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4
5 6 7 8 5 6 7 8 5 6 7 8 5 6 7 8 5 6 7 8 5 6 7 8
13 9 10 11 9 10 11 9 10 11 9 10 11 9 10 11 9 10 11 12
14 15 12 13 14 15 12 13 14 15 12 13 14 15 12 13 14 15 12 13 14 15
solves the Hi-Q puzzle. Let p = n(n + 4m). Your algorithm should run in
time O((4p)p−2 ).
m n m
518. The I-C Ice Cream Company plans to place ice cream carts
at strategic intersections in Dallas so that for every street intersection, either
there is an ice cream cart there, or an ice cream cart can be reached by
walking along only one street to the next intersection. You are given a map of
Dallas in the form of a graph with nodes numbered 1, 2, . . . , n representing the
intersections, and undirected edges representing streets. For each ice cream
cart at intersection i, the city will levy $ci in taxes. The I-C Ice Cream
Company has k ice cream carts, and can make a profit if it pays at most
$d in taxes. Write a backtracking algorithm that outputs all of the possible
locations for the k carts that require less than $d in taxes. Analyze your
algorithm.
Sec. 10.6. Applications 127
1+2+3
1+5
2+4
6
1+2+3+4
1+2+7
1+3+6
1+4+5
1+9
2+3+5
2+8
3+7
4+6
10
523. If you were given a graph and asked to find all Hamiltonian cycles
in it, which algorithm would you use, the algorithm from Problem 520 or the
algorithm from Problem 522? Explain your answer.
A weighing matrix is an n × n matrix with entries from {−1, 0, 1} such that the
scalar product of any two rows is zero. That is, a matrix
m1,1 m1,2 . . . m1,n
m2,1 m2,2 . . . m2,n
M = .. ,
.
mn,1 mn,2 . . . mn,n
Preference P 1 2 3 4 N 1 2 3 4
Matrices:
1 2 3 1 1 1 4 1 3 2
2 1 1 4 3 2 4 2 3 1
3 1 2 3 4 3 1 1 1 4
4 2 3 2 1 4 3 2 3 3
Candidate Pilot: 1 2 3 4
Matching:
Navigator: 1 2 3 4
Weight: P[1,2]N[2,1]+P[2,4]N[4,2]+P[3,1]N[1,3]+P[4,3]N[3,4]
= 3.4+3.2+1.3+2.4 = 12+6+3+8 = 29
Suppose you are given n pilots and n navigators. You have an integer array
P [1..n, 1..n] in which P [i, j] is the preference of pilot i for navigator j, and an
integer array N [1..n, 1..n] in which N [i, j] is the preference of navigator i for pilot
j. The weight of a pairing of pilot i with navigator j is defined to be P [i, j] · N [j, i].
The flight-assignment problem is the problem of pairing each pilot with a navigator
in such a way that the sum of the weights of the pairings is maximized. Figure 10.6
shows an example of the preference arrays, a candidate matching, and the sum of
the weights of the pairs in this matching:
529. Design an algorithm for the flight-assignment problem that runs in time
O((n + 1)!). Analyze your algorithm.
530. Describe how to improve the running time of your algorithm from Problem 529
to O(n!).
The postage stamp problem is defined as follows. The Fussytown Post Office has n
different denominations of stamps, each a positive integer. Fussytown regulations
allow at most m stamps to be placed on any letter.
531. Design an algorithm that, given n different postage stamp values in
an array P [1..n] and the value of m, computes the length of the consecutive
series of postage values that can legally be realized under these rules starting
with the value 1. For example, if n = 4, m = 5, and the stamps have face
value 1, 4, 12, and 21, then all postage values from 1 to 71 can be realized.
Your algorithm should run in time O(nm ).
130 Chap. 10. Exhaustive Search
533. Design an algorithm that runs in time O(3n ) to generate all n-ominos,
including all rotations and reflections.
10.7 HINTS
487. Be very careful how you choose your recurrence relation. The obvious choice
of recurrence is quite difficult to solve. The right choice can make it easy.
489. Suppose that A[i] = i for 1 ≤ i ≤ n, and pretend that procedure process(A)
prints the contents of A. Trace the algorithm by hand to see what it generates
when n = 1, 2, 3, 4. This should give you some intuition as to how it works,
what your induction hypothesis should be, and how the basic structure of
your proof should look.
491. See the hint for Problem 489. Your induction hypothesis should look some-
thing like the following:
When n is even, a call to evenpermute(n)
• processes all permutations of the values in A[1..n], and
• finishes with the entries in A permuted in some order in which, if re-
peated n − 1 times, cycles A[1] through all possible values from 1 to n.
Determining and describing this permutation is the difficult part.
When n is odd, a call to oddpermute(n)
• processes all permutations of the values in A[1..n], and
• finishes with the first and last entries in A swapped.
Sec. 10.8. Solutions 131
502. You may need to use the identity from Problem 23.
510. The
n answer is at most 1.718(n + 1)!. You will need to use the fact that
i=1 i! = e − 1 ≈ 1.718 (where e is the base of the natural logarithm).
520. The running time gives away the technique to be used. Reduce the problem
to that of generating permutations.
525. Reduce the problem to that of generating combinations, and take advantage
of Section 10.3.
10.8 SOLUTIONS
487. Let T (n) be the running time of permute(n). Line 6 can be done in time
O(n). Therefore, T (1) = c, and for n > 2, T (n) = nT (n − 1) + d(n − 1), for
some c, d ∈ IN. We claim that for all n ≥ 1, T (n) = (c + d)n! − d. The proof
is by induction on n. The claim is certainly true for n = 1, since T (1) = c
and (c + d)1! − d = c. Now suppose the hypothesis is true for n. Then,
T (n + 1) = (n + 1)T (n) + dn
= (n + 1)((c + d)n! − d) + dn
= (c + d)(n + 1)! − d.
Hence, the running time is O(n!), which is linear in the number of permuta-
tions.
520. The following algorithm prints all of the Hamiltonian cycles of a graph G.
132 Chap. 10. Exhaustive Search
procedure process(P )
if hamiltonian(P ) then print(P )
521. The solution is similar to the solution of Problem 520, but we use a minimal-
change algorithm such as Heap’s algorithm (see Section 10.2) to generate
permutations.
We keep a Boolean array D[0..n − 1], where D[i] is true iff (P [i], P [(i + 1) mod
n) ∈ E, for 0 ≤ i < n, and a count of missing edges, called “missing.”
We initialize the array and count when processing the first permutation. A
minimal change to P requires a minimal change to the array and the count.
Procedure process is exactly as in the solution to Problem 520.
missing:=0
P := first permutation
process(P )
for each permutation P in minimal-change order do
Suppose P [x] and P [y] were just swapped
iprocess(P, x, y)
procedure edge(i, j)
D[i] := (i, j) ∈ E
if (i, j) ∈ E then missing:=missing+1
procedure iprocess(P, x, y)
if ihamiltonian(P, x, y) then print(P )
procedure iedge(i, j)
if D[i] ∧ ((i, j) ∈ E) then
D[i]:=false; missing:=missing + 1
else if (not D[i]) ∧ ((i, j) ∈ E) then
D[i]:=true; missing:=missing − 1
The time needed to process the first permutation is O(n). The other n! − 1
permutations require O(1) time each to process. The overhead for generating
the permutations is O(n!) with a minimal-change algorithm such as Heap’s
algorithm. Therefore, the total running time is O(n + (n! − 1) + n!) = O(n!).
As before, this running time can be reduced by a factor of Ω(n) by observing
that all of the permutations can begin with vertex 1.
10.9 COMMENTS
478. The algorithm generates the binary reflected Gray code (see also Problems 96
and 97).
497. The algorithm is adapted from Lam and Soicher [51]. You will find a cor-
rectness proof there, but there is an easier and clearer explanation. Another
interesting algorithm with a stronger minimal-change property is given by
Eades and McKay [23].
512. For more information on Ramsey numbers, consult Graham and Spencer [32],
and Graham, Rothschild, and Spencer [31].
513. See also Problems 48 and 369 (and associated comments), which deal with
closed knight’s tours. Backtracking for n × n closed knight’s tours is only
practical for n = 6 (there are no tours for smaller n), in which case there
are 9, 862 closed knight’s tours. It is not known exactly how many 8 × 8
tours exist. Rouse Ball and Coxeter [8] describe an interesting random walk
algorithm that they attribute to Euler. A heuristic attributed to Warnsdorff
in 1823 by Conrad, Hindrichs, Morsy, and Wegener [15, 16] appears to help
both the exhaustive search and random walk algorithms find solutions faster:
When there is a choice of squares to move to, move to the one with the least
number of unused legal moves out of it. Takefuji and Lee [79] (reprinted in
Takefuji [78, Chapter 7]) describe a neural network for finding closed knight’s
tours, but empirical evidence indicates that their algorithm is impractical
(Parberry [62]).
517. Although an algorithm that runs in time O(n!) is considered very slow, it
134 Chap. 10. Exhaustive Search
n count time
6 4 < 0.001 seconds
7 40 < 0.001 seconds
8 92 < 0.001 seconds
9 352 0.034 seconds
10 724 0.133 seconds
11 2,680 0.6 seconds
12 14,200 3.3 seconds
13 73,712 18.0 seconds
14 365,596 1.8 minutes
15 2,279,184 11.6 minutes
16 14,772,512 1.3 hours
17 95,815,104 9.1 hours
18 666,090,624 2.8 days
is still usable in practice for very small n. Table 10.1 shows the number
of solutions to the n-queens problem and the running time required to find
them using Berkeley Unix Pascal on Sun Sparc 2. It is known that there is
at least one solution to the n-queens problem for all n ≥ 4 (see, for example,
Bernhardsson [11]). Heuristics for finding large numbers of solutions (but not
necessarily all of them) include Takefuji [78], and Susic and Gu [77].
Data Structures
Algorithms and data structures are obviously very closely related. Firstly, you must
understand algorithms in order to design data structures, since an important issue
in choosing a particular data structure is the availability of efficient algorithms for
creating, updating, and accessing it. Secondly, you must understand data structures
in order to design algorithms, since the proper organization of data is often the
key to developing efficient algorithms. Some schools separate algorithms and data
structures into separate classes, but I prefer to teach them together. In case your
algorithms class requires it, here are some problems on advanced data structures,
principally heaps, binary search trees, AVL trees, 2–3 trees, and the union-find data
structure.
11.1 HEAPS
A heap is a binary tree with the data stored in the nodes. It has two important
properties: balance and structure. Balance means that it is as much like a complete
binary tree as possible. Missing leaves, if any, are on the last level at the far right
(see, for example, Figure 11.1). Structure means that the value in each parent is no
greater than the values in its children (see, for example, Figure 11.2).
535. Prove that the value in each node is no greater than the values in its
descendants (that is, its children, its children’s children, etc.)
135
136 Chap. 11. Data Structures
9 5
11 20 10 24
21 15 30 40 12
536. Prove by induction that a heap with n vertices has exactly n/2 leaves.
540. Give the array representation heap shown in Figure 11.2, and redraw the
heap and its array representation after each of the following operations have
been applied in sequence:
541. Construct a heap containing the items 10, 2, 9, 16, 8, 6, 1, 3, 12. You
are to draw the heap (as a binary tree) at each of the major steps in the
construction.
(a) Inserting the items in the order given into an empty heap.
(b) Building the heap from the bottom up, as in heapsort.
542. Write a procedure that, given parameters A, n, and i, where A is the
array representation of an n-element heap, and 1 ≤ i ≤ n, deletes the element
A[i] from the heap in O(log n) time.
543. Halving is the process of taking a set S of integers and separating it
into two half-sized sets S1 , S2 , where all of the values in S1 are smaller than
all of the values in S2 . Show that halving is equivalent to finding the median,
that is:
Sec. 11.1. Heaps 137
(a) If the median can be found using T (n) comparisons, then a set of size n
can be halved with T (n) + n − 1 comparisons.
(b) If a set of size n can be halved using T (n) comparisons, then the median
can be found using T (n) + n/2 − 1 comparisons.
The standard heapsort algorithm uses binary heaps, that is, heaps in which each
node has at most two children. In a k-ary heap, each node has at most k children.
Heapsort works not only with binary heaps, but with k-ary heaps for any k =
1, 2, 3, . . ..
548. For k > 1, analyze the number of binary comparisons used by k-ary
heapsort in terms of n and k, determining the coefficient (depending on k) of
the most significant n-term. Which k minimizes this factor? Why is k = 2
probably best anyway?
A parallel priority queue is a priority queue in which the insert operation inserts
p values, and the deletemin operation deletes the p smallest values in the priority
queue and returns them in no particular order. A parallel heap is a data structure
with the same tree structure as a heap, but with p values per node. It also has the
property that all of the values in a node are smaller than the values in its children,
and all of the values in one sibling are smaller than the values in the other (it doesn’t
matter which is the “smaller” sibling). For example, Figure 11.3 shows a parallel
heap with p = 4.
Sequential algorithms for parallel heaps are fairly straightforward: An insert
operation is processed by first placing the p new items in the next unused node d,
which we will call dirty. Combine the values in d and its parent r, and place the
smallest p values into r, and the largest p values into d. Combine the values in d
and its sibling (if it has one), and place the largest p values into the sibling that
had the largest value between them, and the smallest p values into the other sibling.
Node r then becomes the dirty node, which is processed in a similar fashion. For
example, Figure 11.4 shows an insertion into a 4-node parallel heap with p = 4.
138 Chap. 11. Data Structures
6 1
2 5
19
15 20 25
9
8 30 40
32 26 18
25
27 45 22 16
A deletemin operation is processed by removing the values from the root and
replacing them with the values from the last used node. Call the root dirty. Let
d be the dirty node. Combine the values in d with those of its smallest child s,
and place the smallest p values into d and the largest p values into s. Combine the
values in d and its sibling (if it has one), and place the largest p values into the
sibling that had the largest value between them, and the smallest p values into the
other sibling. Node s then becomes the dirty node, which is processed in a similar
fashion.
549. Show that the insert algorithm is correct, that is, it results in a
parallel heap.
550. Show that the deletemin algorithm, that is, it results in a parallel
heap.
551. Analyze the running time of the insert and deletemin algo-
rithms.
6 1 6 1
2 5 2 5
19 19
20 25 15 20 25 15
9 9
30 40 8 30 40 8
32 26 32 26 50
3
27 45 27 45 10
23
6 1 6 1
2 5 2 5
10 19 10 19
3 15 3 15
20 23 8 9 20 23 8 9
50 30 25 5045
32 26 26 27
27 45 25 40 32 40
30
3 1 3 1
2 5 2 5
10 19 15 10
6 15 19
9 20 8 6
20 23 8 23 9
25 5045 25 5045
26 27 26 27
30 32 40 30 32 40
100
50 150
30 60 120 160
10 40
100
50 150
30 70 200
20 60 80
An AVL tree is a binary search tree with one extra structure condition: For every
node, the magnitude of the difference in height between its left and right subtrees is
at most one. (the height of an AVL tree is the number of edges in the longest path
from the root to a leaf). This difference in height is maintained during insertions and
deletions using operations called single rotations and double rotations. Operations
insert, delete, member, and findmin can be performed in time O(log n).
556. Find an AVL tree for which the deletion of a node requires two single
rotations. Draw the tree, indicate the node to be deleted, and explain why
two rotations are necessary and sufficient.
Sec. 11.3. 2–3 Trees 141
100
50 150
30 120 200
210
557. Find an AVL tree for which the deletion of a node requires two double
rotations. Draw the tree, indicate the node to be deleted, and explain why
two rotations are necessary and sufficient.
558. Show that for all n ∈ IN, there exists an AVL tree for which the
deletion of a node requires n (single or double) rotations. Show how such
a tree is constructed, indicate the node to be deleted, and explain why n
rotations are necessary and sufficient.
559. Prove that the height of an AVL tree with n nodes is at most
1.4404 log n.
1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
Suppose we modify the union-find algorithm so that the root of the shallower tree
points to the root of the deeper tree (instead of the root of the smaller tree pointing
to the root of the deeper tree).
563. Does this affect the running time of the algorithm in the worst case?
564. Does it affect the amortized running time when path compression is
used?
11.5 APPLICATIONS
This section asks you to use your knowledge of data structures to design algorithms
for new problems.
Sec. 11.5. Applications 143
565. Design a data structure for a set in which insertions, deletions, and
membership queries can be processed in O(1) time in the worst case. You may
assume that the set elements are integers drawn from a finite set {1, 2, . . . , n}.
566. Let S = {s1 , s2 , . . . , s } and T = {t1 , t2 , . . . , tm } be two sets of inte-
gers such that 1 ≤ si , tj ≤ n for all 1 ≤ i ≤ . and 1 ≤ j ≤ m. Design an
algorithm for determining whether S = T in time O(. + m).
The operations insert, delete, member, and next are to take time O(log n),
and union is to take time O(n).
All operations involving n-element sets are to take time O(log n).
570. The bin packing problem is the following. You are given n metal ob-
jects, each weighing between zero and one kilogram. You also have a collection
of large, but fragile bins. Your aim is to find the smallest number of bins that
will hold the n objects, with no bin holding more than one kilogram.
144 Chap. 11. Data Structures
The first-fit heuristic for the bin packing problem is the following. Take each
of the objects in the order in which they are given. For each object, scan the
bins in increasing order of remaining capacity, and place it into the first bin
in which it fits. Design an algorithm that implements the first-fit heuristic
(taking as input the n weights w1 , w2 , . . . , wn and outputting the number of
bins needed when using first-fit) in O(n log n) time.
572. Reduce the running time in Problem 571 to O(1) when (A, ·) is a
group, and group inversion is allowed for free.
574. Suppose we are to support a set of ordered pairs (p, k), where k is an
integer key and p is an integer priority. Devise a data structure in which the
following operations can be implemented in O(log n) time:
575. Design a data structure that uses O(n log n) space and answers queries
in O(log n) time.
576. Design a data structure that uses O(n) space and answers queries in
O(log n) time.
Sec. 11.6. Hints 145
577. Use your solution to Problem 576 to design a data structure to quickly
answer least-common-ancestor queries in a tree. That is, your data structure
will be constructed from a description of a tree, and using this data structure
you must answer queries of the form: “What is the least-common-ancestor of
tree vertices v and w?”
11.6 HINTS
535. Prove it by induction on level.
544. Use two heaps A and B, each containing n/2 elements (if n is odd, store
one item separately). Make A a min-heap, B a max-heap, and ensure that
for each 1 ≤ i ≤ n/2 , A[i] ≤ B[i]. Then, the root of A will be the smallest
value and the root of B will be the largest value.
559. Let N (h) be the minimum number of nodes in an AVL tree of height h.
Devise a recurrence relation for a lower bound for N (h) (start with N (0) = 1,
N (1) = 2). Show that this recurrence has solution
√ h
1+ 5
N (h) > .
2
562. It’s the standard union-find algorithm with path compression. The difficult
part is the amortized analysis.
570 You will need a data structure that allows you to find the “first bin in which
it fits” in time O(log n).
11.7 SOLUTIONS
545. A k-ary heap is a k-ary tree with the data stored in the nodes. It has two
important properties: balance and structure. Balance means that it is as
much like a complete k-ary tree as possible. Missing leaves, if any, are on the
last level at the far right. Structure means that the value in each parent is
no greater than the values in its children. A k-ary heap on n vertices can be
efficiently implemented in storage using contiguous locations of a linear array
A[1..n], such that the root of the heap is stored in A[1], the vertices are stored
in level order, and for any level, the vertices in it are stored in left-to-right
146 Chap. 11. Data Structures
order. It can be shown that the parent of A[m] is A[ (m + k − 2)/k ] and its
children are A[km − k + 2], . . . , A[min(km + 1, n)].
559. Let N (h) be the minimum number of nodes in an AVL tree of height h.
Clearly, N (0) = 1 and N (1) = 2, and for h ≥ 2, N (h) > N (h − 1) + N (h − 2).
We claim that N (h) ≥ ch for some choice of c. The proof is by induction on
h. The claim is true for h ≤ 2 provided c < 2. For h > 2, by the induction
hypothesis,
N (h) > N (h − 1) + N (h − 2) ≥ ch−1 + ch−2 .
Therefore,
log n
h< √ ≈ 1.4404 log n.
log((1 + 5)/2)
567. We use a 2–3 tree augmented with extra fields in each internal node recording
how many leaves are descendants of the left, middle, and right children. The
insertion is done as normal, with extra code to increment the leaf-count of the
nodes on the path from the root to the new leaf, and to take care of the new
fields during node splitting. The deletion code is modified in a similar way,
with the extra modification that the location of the leaf to be deleted is done
using the new leaf-count fields in the obvious way (recall that in a 2–3 tree,
the leaves are in ascending order from left to right). The member operations
are done as normal.
11.8 COMMENTS
537. You’ll see this result in all competent data structures textbooks, but you’ll
seldom see a proof of it.
570. This problem is N P-complete (see Chapter 12 if you don’t know what this
means), and so it is unlikely that there is a polynomial-time algorithm for
finding an exact solution.
Chapter 12
N P -completeness
Define P to be the set of decision problems (that is, problems that have a yes/no
answer) that can be solved in polynomial time. If x is a bit string, let |x| denote the
number of bits in x. Define N P to be the set of decision problems of the following
form, where R ∈ P, c ∈ IN: “Given x, does there exist y with |y| ≤ |x|c such that
(x, y) ∈ R.” That is, N P is the set of existential questions that can be verified in
polynomial time. Clearly, P ⊆ N P. It is not known whether P = N P, although
the consensus of opinion is that probably P = N P. But it is known that there are
problems in N P with the property that if they are members of P, then P = N P.
That is, if anything in N P is outside of P, then they are, too. They are called
N P-complete problems.
A problem A is reducible to B if an algorithm for B can be used to solve A.
More specifically, if there is an algorithm that maps every instance x of A into an
instance f (x) of B such that x ∈ A iff f (x) ∈ B. A problem A is polynomial time
reducible to B, written A ≤m p B, if there is a polynomial time algorithm that maps
every instance x of A into an instance f (x) of B such that x ∈ A iff f (x) ∈ B. Note
that the size of f (x) can be no greater than a polynomial of the size of x.
Listing problems on N P-completeness is a little bit redundant. The truly dedi-
cated student can practice by picking two random N P-complete problems from the
long list in the back of Garey and Johnson [28] and attempting to prove that one
is reducible to the other. But here are a few problems that I find useful.
12.1 GENERAL
This section contains a few general questions about N P-completeness. Many of
them use variants of the satisfiability problem. The standard terminology is used
here: A literal is a complemented or uncomplemented variable, a clause is a dis-
junction of literals, and a Boolean formula is a conjunction of clauses. Then SAT
is defined as follows:
SAT
Instance: A Boolean formula F .
147
148 Chap. 12. N P -completeness
579. 2SAT is the satisfiability problem with at most two literals per clause.
Show that 2SAT ∈ P.
DNF SAT
Instance: A Boolean formula F in disjunctive normal form.
Question: Is there a truth assignment to the variables of F that
makes F true?
581. Show that the following problem can be solved in polynomial time:
STRONG 3SAT
Instance: A Boolean formula F in conjunctive normal form with
at most three literals per clause.
Question: Is there a truth assignment to the variables of F that
makes at least two literals in each clause of F true?
582. Show that the following problem can be solved in polynomial time:
PARITY 3SAT
Instance: A Boolean formula F in conjunctive normal form with
at most three literals per clause.
Question: Is there a truth assignment to the variables of F that
makes an odd number of literals in each clause of F true?
algorithm for A as a subroutine, and this algorithm makes polynomially many calls
to A and runs in polynomial time if the calls to A are not counted. This type of
reduction is sometimes called a Cook reduction after Cook [17].
SAT1
Instance: A Boolean formula F .
Question: Is there a truth assignment to the variables of F that makes
F true?
SAT2
Instance: A Boolean formula F .
Output: A truth assignment to the variables of F that makes F true,
if one exists.
The former is the decision problem for satisfiability, and the second requires the
construction of a solution. Clearly the decision problem is Cook reducible to the
solution problem. The term self-reducibility, coined by Mike Paterson, is used to
describe problems for which the opposite is true.
TSP1
Instance: A directed graph G with positive costs on the edges, and a
positive integer B.
Question: Is there a Hamiltonian cycle in G of cost at most B?
TSP2
Instance: A directed graph G with positive costs on the edges, and a
positive integer B.
Output: A Hamiltonian cycle in G of cost at most B, if one exists.
150 Chap. 12. N P -completeness
TSP3
Instance: A directed graph G with positive costs on the edges, and a
positive integer B.
Output: A Hamiltonian cycle in G of minimum cost, if one exists.
588. Prove that TSP2 ≤Tp TSP1.
590. Prove directly that TSP3 ≤Tp TSP1, without using the fact that “≤Tp ”
is transitive.
Since DNF-SAT ∈ P (see Problem 580), a satisfying assignment for the new
formula, and hence for the old formula, can be found in polynomial time. This
shows that 3SAT ∈ P, and hence that P = N P.
593. The knapsack problem is defined as follows:
KNAPSACK
Instance: Positive integers s1 , s2 , . . . , sn , S
Question: Does there exist X ⊆ {1, 2, . . . , n} such that i∈X si =
S?
Sec. 12.3. What is Wrong? 151
12.4 Circuits
An architecture is a finite combinational circuit (that is, a circuit constructed with-
out feedback loops) with the gate functions left unspecified. A task for an archi-
tecture is an input-output pair, and a task set is a finite set of tasks. The loading
problem is the following: Given an architecture and a task set, find functions for
each of the gates that enable the circuit to give the correct output for each cor-
responding input in the task set. The following questions deal with the decision
problem version of the loading problem: Given an architecture and a task set, de-
cide whether there exist functions for each of the gates that enable the circuit to
give the correct output for each corresponding input in the task set. The problems
in this section are from Parberry [63].
A cyclic circuit is a network of gates that may have cycles. Time is quantized,
and at each step, one or more of the gates reads its inputs, and computes a new
output. All connections are directed, and self-loops are allowed. The gate-functions
are limited to are conjunction, disjunction, and complement. The circuit is said to
have converged to a stable configuration if the output of all gates remains fixed over
time. Consider the following problems:
The Stable Configuration Problem (SNN)
Instance: A cyclic circuit M .
Question: Does M have a stable configuration?
599. Show that SNN is N P-complete.
600. Show that SNN is N P-complete for cyclic circuits of fan-in 2.
12.6 FACTORIZATION
Let pi denote the ith prime number, starting with p1 = 2. The factorization problem
is defined as follows:
FACTORIZATION
Instance: A natural number, N .
Output: A list of pairs (pi , ai ) for 1 ≤ i ≤ k, such that N = pa1 1 pa2 2 · · · pakk .
12.7 HINTS
594. A lot of the details of this proof are missing. Most of them are correct. At
least one of them is wrong. It is up to you to discover which.
604. Prove a Turing reduction to the problem “is there a factor less than M .”
Sec. 12.8. Solutions 155
12.8 SOLUTIONS
595–597. A solution to these problems can be found in Judd [41].
603. A solution to this problem can be found in Parberry [60]. However, the
reduction is from O3SAT3, not O3SAT1 as requested; so technically you’ll
have to solve Problem 602 in order to complete this proof.
12.9 COMMENTS
578. Most textbooks (and classes) prove that if A ≤m p B and B ∈ P, then A ∈ P,
and omit the proof of this result saying only that it is “similar.”
595. For more information on the complexity of variants of the loading problem,
consult also Judd [39, 40, 42], Lin and Vitter [53], and Blum and Rivest [12,
13].
Chapter 13
Miscellaneous
Here are some miscellaneous problems, defined to be those that do not necessarily
fit into the earlier chapters, and those for which part of the problem is to determine
the algorithmic technique to be used. You’re on your own!
607. Show that n positive integers in the range 1 to k can be sorted in time
O(n log k).
609. Show that finding the median of a set S and finding the kth smallest
element are reducible to each other. That is, if |S| = n, any algorithm for
finding the median of S in time T (n) can be used to design an O(T (n)) time
algorithm for finding the kth smallest, and vice-versa, whenever the following
holds: T is monotone increasing and there exists a constant 0 < c < 1 such
that T (n/2) < c · T (n) for n ∈ IN.
The nuts and bolts problem is defined as follows. You are given a collection of n
bolts of different widths, and n corresponding nuts. You are allowed to try a nut
and bolt together, from which you can determine whether the nut is too large, too
small, or an exact match for the bolt, but there is no way to compare two nuts
together, or two bolts together. You are to match each bolt to its nut.
610. Show that any algorithm for the nuts and bolts problem must
take Ω(n log n) comparisons in the worst case.
156
Sec. 13.2. Lower Bounds 157
611. Devise an algorithm for the nuts and bolts problem that runs
in time O(n log n) on average .
612. Suppose that instead of matching all of the nuts and bolts, you wish to
find the smallest bolt and its corresponding nut. Show that this can be done
with only 2n − 2 comparisons.
613. Show a matching lower bound for the sorting problem of Problem 607
in a comparison-based model.
on the number of comparisons required to find the median. You may use
Stirling’s approximation, √
n! ∼ 2πn(n/e)n .
158 Chap. 13. Miscellaneous
Let G = (V, E) be a directed labeled graph with n vertices and e edges. Suppose
the cost of edge (v, w) is .(v, w). Suppose c = (v0 , v2 . . . , vk−1 ) is a cycle, that is,
(vi , v(i+1)modk ) ∈ E for all 0 ≤ i < k. The mean cost of c is defined by
k−1
1
µ(c) = w(vi , v(i+1)modk ).
k i=0
Sec. 13.3. Graph Algorithms 159
Let µ be the cost of the minimum mean-cost cycle in G, that is, the minimum value
of µc over all cycles c in G.
Assume without loss of generality that every v ∈ V is reachable from a source
vertex s ∈ V . Let δ(v) be the cost of the cheapest path from s to v, and δk (v) be
the cost of the cheapest path of length exactly k from s to v (if there is no path
from s to v of length exactly k, then δk (v) = ∞).
δn (v) − δk (v)
max ≥ 0.
0≤k≤n−1 n−k
628. Let c be a cycle of cost zero, and let u, v be any pair of vertices
on c. Suppose that the cost of the path from u to v along c is x. Prove that
δ(v) = δ(u) + x.
δn (v) − δk (v)
max = 0.
0≤k≤n−1 n−k
δn (v) − δk (v)
min max = 0.
v∈V 0≤k≤n−1 n−k
631. Show that if we add a constant c to the cost of every edge of G, then
µ is increased by exactly c. Hence, show that
δn (v) − δk (v)
µ = min max .
v∈V 0≤k≤n−1 n−k
The maximum flow problem is to find a flow f that has maximum total flow.
Algorithms for the maximum flow problem include the Ford-Fulkerson algo-
rithm, the Dinic algorithm, and the Edmonds-Karp algorithm.
Consider a flow problem on a graph G = (V, E) with n vertices and e edges with all
capacities bounded above by a polynomial in n. For any given flow, let the largest
edge flow be
max f (u, v).
(u,v)∈E
636. Show that two different maximum flows in G can have different largest
edge flows. Design an algorithm to find the minimum possible
largest edge flow over all maximum flows in G in time O(ne2 log n).
Sec. 13.5. Matrix Reductions 161
Suppose we are given the maximum flow in a flow network G = (V, E) with source
s, sink t, and integer capacities c(u, v) on each edge (u, v) ∈ E. Suppose G has n
vertices and e edges. The following modified version of the standard Ford-Fulkerson
algorithm can be used to find a maximum flow in G.
function max-flow-by-scaling(G, s, t)
1. c := max(u,v)∈E c(u, v)
2. initialize flow f to zero
3. k := 2log c
4. while k ≥ 1 do
5. while there exists an augmenting path p of capacity ≥ k do
6. augment flow f along p
7. k := k/2
8. return(f )
644. Show that for all k ≥ 2, multiplying two n×n matrices is equivalent
to raising an n × n matrix to the kth power.
162 Chap. 13. Miscellaneous
645. Define the closure of an n × n matrix A to be ∞ i
i=0 A , where A
0
13.6 GENERAL
652. You are facing a high wall that stretches infinitely in both directions.
There is a door in the wall, but you don’t know how far away or in which
direction. It is pitch dark, but you have a very dim lighted candle that will
enable you to see the door when you are right next to it. Show that there
is an algorithm that enables you to find the door by walking at most O(n)
steps, where n is the number of steps that you would have taken if you knew
where the door is and walked directly to it. What is the constant multiple in
the big-O bound for your algorithm?
The schedule lists the times at which each event begins and ends. The events
may overlap. Naturally, you want to attend as many events as you can. Design
an efficient algorithm to find the largest set of nonoverlapping events. Prove
that your algorithm is correct. How fast is it?
The following group of questions deals with the generalized version of the popular
Sam Loyd 15-puzzle, called the (n2 − 1)-puzzle. For definitions, see Problem 515.
659. Design an algorithm for the (n2 − 1)-puzzle that runs in time
3
O(n ). Your algorithm must input a legal state of the puzzle and output a
series of O(n3 ) moves that solves it.
660. Show that the worst case number of moves needed to solve the
(n2 − 1)-puzzle is at least n3 − O(n2 ).
164 Chap. 13. Miscellaneous
661. Show that the average case number of moves needed to solve the
(n2 − 1)-puzzle is at least 2n3 /3 − O(n2 ).
662. Show that for some constants 0 < α, β < 1, a random legal con-
figuration of the (n2 − 1)-puzzle requires at least αn3 moves to solve with
2
probability 1 − 1/e−βn .
Suppose you are given a collection of gold coins that includes a clever counterfeit
(all the rest are genuine). The counterfeit coin is indistinguishable from the real
coins in all measurable characteristics except weight. You have a scale with which
you can compare the relative weight of coins. However, you do not know whether
the counterfeit coin is heavier or lighter than the real thing. Your problem is to find
the counterfeit coin and determine whether it is lighter or heavier than the rest.
663. Show that if there are 12 coins, the counterfeit coin can be
found in three weighings.
664. Show that if there are 39 coins, the counterfeit coin can be found in
four weighings.
665. Show that if there are n ≥ 3 coins, the counterfeit coin can be
found in log3 2n weighings.
666. Show that if there are n ≥ 3 coins, at least log3 2n weighings are
needed to find the counterfeit coin.
Your job is to seat n rambunctious children in a theater with n balconies. You are
given a list of m statements of the form “i hates j.” If i hates j, then you do not
want to seat i above or in the same balcony as j, otherwise i will throw popcorn at
j instead of watching the play.
667. Give an algorithm that assigns the balconies (or says that it is not
possible) in time O(m + n).
668. Design and analyze an algorithm for finding the minimum number of
balconies needed.
13.7 HINTS
614. Show that if the search for two different elements x, y, where x < y, in the set
leads to the same leaf of the comparison tree, then the search for any element
a of the universe with x < a < y must also lead to that leaf, which contradicts
the correctness of the algorithm.
Sec. 13.7. Hints 165
629. Show that a min-cost path to any vertex on the minimum mean-cost cycle
can be extended along the cycle to the next vertex on the cycle.
The capacity of all edges is one. Show that the minimum number of paths
that cover V in G is equal to n − f , where f is the maximum flow in G .
641–645. Consider Problems 647–651. The lower triangular property can actually
make the problem easier, if you think about it for long enough. On the other
hand, there do exist nontriangular solutions, as the solution to Problem 641
given later demonstrates.
660. The Manhattan distance of a tile in square (i, k) that belongs in square (k, .)
is defined to be |i − k| + |j − .|. The Manhattan distance of a configuration
of the puzzle is defined to be the sum of the Manhattan distances of its tiles.
Show that the Manhattan distance is a lower bound on the number of moves
needed to solve the puzzle, and that there exists a configuration that has
Manhattan distance at least n3 − O(n2 ).
662. Find a lower bound for the Manhattan distance of a random configuration.
The best value that I can get for α is 16/243. You will need to use Chernoff
bounds. Let B(m, N, p) be the probability of obtaining at least m successes
166 Chap. 13. Miscellaneous
663. Start by dividing the coins into three piles of four coins. Pick two piles and
compare their weights.
665. Start by solving Problems 663 and 664. Then generalize your solution.
13.8 SOLUTIONS
641. The trick is to embed A and B into a larger matrix that, when squared, has
AB has a submatrix. The solution can then be read off easily in optimal (that
is, O(n2 )) time. There are many embeddings that work, including
2
0 A AB 0
= .
B 0 0 AB
663. Actually, I’m not going to give you a solution. This problem is so popular
with mathematicians that you ought to be able to find one in the literature
somewhere. It even appears in a science fiction novel by Piers Anthony [6,
Chapter 16].
13.9 COMMENTS
610. The nuts and bolts problem is from Rawlins [66].
653. This problem was suggested to the author by Arny Rosenberg. Interestingly,
one can prove by a counting argument that one can so permute an n×k matrix,
but an actual efficient algorithm is known only for k = 1, 2. For k = 1, this
is a relative of the Erdös-Szekeres Theorem , which states that there exists
in every sequence of n integers
√ a monotonic subsequence (either ascending or
descending) of length n (for more recent proofs of this theorem, see, for
example, Dijkstra [22], and Seidenberg [73]).
659. Horden [36] gives an interesting history of this puzzle and its variants. See
also Gardner [27]. Early work on the 15-puzzle includes Johnson [38] and
Storey [76]. The minimum number of moves to solve each legal permutation
of the 8-puzzle has been found by exhaustive search by Schofield [69]. The
minimum number of moves needed to solve the 15-puzzle in the worst case
is unknown, but has been the subject of various papers in the AI literature,
Sec. 13.9. Comments 167
including Michie, Fleming and Oldfield [57] and Korf [48]. Ratner and War-
muth [65] have proved that the problem of determining the minimum length
number of moves for any given legal configuration of the (n2 − 1)-puzzle is
N P-complete, and they demonstrate an approximation algorithm that makes
no more than a (fairly large) constant factor number of moves than necessary
for any given legal configuration. Kornhauser, Miller, and Spirakis [49] have
shown an algorithm for the (n2 − 1)-puzzle and its generalizations that always
runs in time O(n3 ).
Bibliography
[5] D. Angluin and L. Valiant. Fast probabilistic algorithms for Hamiltonian cir-
cuits and matchings. In Proceedings of the Ninth Annual ACM Symposium on
Theory of Computing, pages 30–41. ACM Press, 1977.
[11] B. Bernhardsson. Explicit solutions to the n-queens problem for all n. SIGART
Bulletin, 2(2):7, 1991.
168
Bibliography 169
[14] G. Brassard and P. Bratley. Algorithmics: Theory and Practice. Prentice Hall,
1988.
[20] P. Cull and J. DeCurtins. Knight’s tour revisited. Fibonacci Quarterly, 16:276–
285, 1978.
[23] P. Eades and B. McKay. An algorithm for generating subsets of fixed size
with a strong minimal change property. Information Processing Letters, pages
131–133, 1984.
[25] L. Euler. Solution d’une question curieuse qui ne paroit soumise à aucune
analyse. Mem. Acad. Sci. Berlin, pages 310–337, 1759.
[40] J. S. Judd. Neural Network Design and the Complexity of Learning. PhD
thesis, University of Massachusetts, Amherst, MA, 1988.
[42] J. S. Judd. Neural Network Design and the Complexity of Learning. MIT Press,
1990.
[46] D. E. Knuth. Sorting and Searching, volume 3 of The Art of Computer Pro-
gramming. Addison-Wesley, 1973.
[51] C. W. H. Lam and L. H. Soicher. Three new combination algorithms with the
minimal change property. Communications of the ACM, 25(8), 1982.
[52] Lewis and Denenberg. Data Structures and Their Algorithms. Harper Collins,
1991.
[53] J.-H. Lin and J. S. Vitter. Complexity results on learning by neural nets.
Machine Learning, 6:211–230, 1991.
[54] X.-M. Lu. Towers of Hanoi problem with arbitrary k ≥ 3 pegs. International
Journal of Computer Mathematics, 24:39–54, 1988.
[63] I. Parberry. Circuit Complexity and Neural Networks. MIT Press, 1994.
[64] P. W. Purdom Jr. and C. A. Brown. The Analysis of Algorithms. Holt, Rine-
hart, and Winston, 1985.
[65] D. Ratner and M. K. Warmuth. The (n2 − 1)-puzzle and related relocation
problems. Journal for Symbolic Computation, 10:11–137, 1990.
[75] D. Solow. How to Read and Do Proofs. Wiley, second edition edition, 1990.
Bibliography 173
174
Index 175
This note contains corrections to Parberry [5]. An up-to-date copy of this document and
other useful information can be found on the World-Wide Web (Parberry [7]). I welcome
further errata and comments from readers — I prefer electronic mail, but if you must use
more primitive modes of communication, full contact information can be found in [6].
Then anyone who leaves behind him a written manual, and likewise anyone who
receives it, in the belief that such writing will be clear and certain, must be ex-
ceedingly simple-minded.
— Plato, Phaedrus
General Errata
Apparently I couldn’t make up my mind whether the keyword “return” in my algorithms
should be in bold face or Roman type. It appears in bold face in Chapters 5, 7–9, and 13;
in Roman face in Chapters 5, 6, and 10. Out of 71 occurrences, 39 are in bold face.
Specific Errata
Page 3, Line 1: The first comma should be a period.
Page 6, Line 14: The second “but” should be deleted.
Page 7, Last Paragraph: Wilf’s textbook is now out of print, and is very difficult to
obtain in its original form. Fortunately, the author has made it generally available in
postscript form from http://www.cis.upenn.edu/~wilf/index.html.
∗
Copyright c Ian Parberry. This Errata may be reproduced without fee provided that the copies are not
made for profit, and the full text, including the author information and this notice, are included.
†
Author’s address: Department of Computer Sciences, University of North Texas, P.O. Box 13886, Den-
ton, TX 76203-6886, U.S.A. Electronic mail: [email protected].
1
Page 10, Problem 22: That should be induction on n ≥ 2. If n = 0, the right side of
inequality doesn’t make sense, and if n = 1 equality holds.
Page 12, Problem 47: This problem does not have a comment associated with it; the
should be deleted.
Page 16, Line 10: In the set V , the first occurrence of v2 should be v1 .
Page 16, Problem 67: I intended the graph here to be undirected and with no self-loops,
but this did not make it into the definition. I need to add to the definition of a graph
at the start of Section 2.10 the extra property that V is an ordered set and for all
(u, v) ∈ E, u < v.
Page 20, Hint to Problem 37: The comma at the end of the second sentence should be
a period.
Page 21, Solution to Problem 8: The fourth line of the multiline equation should begin
with “=”, not “−”.
Page 22, Solution to Problem 37, Line 5: Change “all values up to n − 1 cents” to “all
values from 8 to n − 1 cents”.
Page 25, Comment to Problem 13, Line 6: Change “by Problem 1” to “by Problem 11”.
Page 35, Solution to Problem 133: On line 4, the denominator of the fraction should
be 6, not 2.
Page 40, Problem 241: This problem should ask you to show that 2n − 3log n − 1 ≤
T (n) ≤ 2n − log n − 1.
Page 40, Problem 242: That should be T (1) = 0, and T (n) = T ( n/2 )+T (n/2)+n−1
(otherwise it is inconsistent with the answer given on p. 41); it is the number of
comparisons used by mergesort.
Page 45, Line 3: “skip it is” should read “skip it if”.
Page 52, Problem 266: In Line 1 of the algorithm, the Roman “n” in the return statement
should be an italic “n”.
Page 52, Problem 269: In Line 2 of the algorithm, the final Roman “y” in the return
statement should be an italic “y”.
0
Page 56, Hint to Problem 258: The loop invariant is xj = yk=y j−1
k, for all j ≥ 1.
Page 56, Solution to Problem 250:
2
make this clearer. The algorithm of Problem 318 almost meets the required bound.
It makes 2 more comparisons than necessary, and should be rewritten slightly to save
them (hint: consider the second and third comparisons).
Page 68, Problem 320: The second question is not worded very clearly. The algorithm
could be called upon to solve duplicate subproblems at the first level of recursion, given
suitable inputs (can you find one?), but I was looking for more. There comes a point
before the bottom-most level of recursion at which the algorithm is obliged to solve
duplicate sub-problems. This is also the key to the algorithm in Problems 328-330.
Page 68, Problem 321: This problem has a comment, but is missing the .
Page 68, Problem 322: The solution to this problem is a little bit more subtle than I first
thought. It begins: “Break the n-bit number into n/m m-bit numbers.” The remainder
of the analysis from that point is straightforward, but in that first step lies a small
difficulty. It can be done by repeatedly shifting the large number to the right, removing
single bits and reconstructing the pieces in m registers. One needs to keep two counters,
one of which runs from 0 to m, and the other off which runs from 0 to n/m. Consider
the second counter. A naive analysis would show that this requires O(n/m) increment
operations, each of which takes O(log n − log m) bit operations. However, one needs to
notice that the total is O(n/m) amortized time (see Problems 476, and 477). I need
to increase the difficulty level of
√ this problem
√ and add a hint to this effect.
Page 74, Problem 340: Change n to N .
Page 78, preamble to Section 7.7: Line 4 of the depth-first search algorithm should read
“mark w used”.
Page 79, Problem 366: This problem should be deleted. There is no known way to find
triangles in time O(n + e), using depth-first search or otherwise. Of course, it can be
solved in time O(ne). This problem could be replaced with the much easier one of
finding a cycle (of any size) in a directed graph in time O(n).
Page 80, Problem 371: The array A should contain integers, not natural numbers. Oth-
erwise the problem is trivial.
Page 82, Hint to Problem 361: Delete the second adjacent occurrence of the word “in”.
Page 85, Solution to Problem 369: Perhaps this should be called a comment, not a
solution.
Page 87, Preamble to Section 8.1: In Line 6 of the algorithm, the Roman “m” should
be replaced by an italic “m”.
Page 88, Problem 384, Line 3: The second occurrence of “M2 ” should be “Mi+2 ”.
Page 88, Problem 390, Line 3: “M2 ” should be “Mi+2 ”.
Page 90, Preamble to Section 8.3, Lines 11–12: The operation “member” should be
“search” (two occurrences).
Page 91, Preamble to Section 8.4, Line 4: “all pairs shortest paths” should be “all-
pairs shortest-paths”.
Page 98, Problem 426: The function to be maximized should be:
Page 99, Hint to Problem 415: The last three lines are wrong, and should be deleted.
Page 106, Problem 447: The stated proposition is wrong! Here is a counterexample. The
3
problem should be changed to “prove or disprove”.
2
2 3
1 1
1 4
2
4
Page 141, Preamble to Section 11.3, Line 1: Change the first occurrence of “in” to
“is”.
Page 142, Problem 562, Line 1: Insert “of” after the word “sequence”.
Page 143, Problem 570: This problem has a hint and a comment, but lacks the appro-
priate icons .
Page 144, Problem 573: Change “as as” to “as” in line 2.
Page 145, Hint to Problem 544: This hint is misleading, and should be deleted. (Less
significantly, n/2 should be n/2 on lines 1 and 3).
Page 145, Hint to Problem 570: Insert a period after the problem number.
Page 150, Preamble to Section 12.3: “Riemann” is mis-spelled ‘Reimann” (also in the
Index on p. 178). Fermat’s Last Theorem may have been proved recently.
Page 155, Solutions: These aren’t really solutions, just bibliographic notes. Perhaps they
should be moved to the Comments section.
Page 155, Solutions to Problems 599–600: Change “this problem” to “these problems”.
Page 157, Problem 620: This problem should be changed from finding the median to
that of halving, which is dividing n values into two sets X, Y , of size n/2 and n/2,
respectively, such that for all x ∈ X and y ∈ Y , x < y. A comment should be added
explaining that this problem is merely practice in using decision trees, since there is a
3n/2 lower bound using the adversary method due to Blum et al. [2].
Page 164, Problem 665 This problem is much harder than I thought. An upper bound of
log3 n + 1 weighings is easy. A little more thought will get you log3 9n/4 weighings.
The best algorithm known to me uses log3 2n + 2 weighings (see Aigner [1]).
Page 171, references [46, 47]: For some reason BibTEX chose to list Knuth Vol. 3 before
Vol. 2.
Page 171, reference [52]: The authors’ initials are missing — H. R. Lewis and L. Denen-
berg.
Page 172, references [66, 67]: These appear to be the same book. The correct reference
is [67].
Page 172, reference [75]: The second occurrence of the word “Edition” should be deleted.
Acknowledgements
The author is very grateful to the following individuals who helpfully pointed out many of the
above errors in the manuscript: Timothy Dimacchia, Sally Goldman, Narciso Marti-Oliet,
G. Allen Morris III, Paul Newell, Mike Paterson, Steven Skiena, Mike Talbert, Steve Tate,
and Hung-Li Tseng.
References
[1] M. Aigner. Combinatorial Search. Wiley-Teubner, 1988.
[2] M. Blum, R. L. Floyd, V. Pratt, R. L. Rivest, and R. E. Tarjan. Time bounds for
selection. Journal of Computer and System Sciences, 7:448–461, 1973.
5
[3] H. N. Gabow. An efficient implementation of Edmond’s algorithm for maximum matching
on graphs. Journal of the ACM, 23(2):221–234, 1976.
[4] H. N. Gabow and R. E. Tarjan. Faster scaling algorithms for general graph-matching
problems. Journal of the ACM, 38(4):815–853, 1991.
[6] I. Parberry. Ian Parberry’s Home Page. A World-Wide Web Document with URL
http://hercule.csci.unt.edu/ian, Created 1994.
6
Supplementary Problems
By Bill Gasarch
Mathematical Induction
1 Games
1. Let A be any natural number. We define a sequence as an as follows:
(1) a10 = A. (2) To obtain an , first write an−1 in base n − 1. Then
let x be the number that is that string of symbols if it was in base n.
Let an = x − 1. (EXAMPLE: if a20 = 1005 then I write 1005 in base
20 as 2T 5 = 2 · 202 + T · 20 + 5 · 1 where T is the symbol for ten. So
x = 2 · (21)2 + 10 · 21 + 5 · 1. So a21 = x − 1 = 1097 − 1 = 1096.) For
which natural numbers A will the sequence go to infinity?
1
(d) Let k ∈ N . Modify the game so that you can remove 1, 2, or 4
sticks. Fill in the following sentence and justify: “Player I has a
forced win iff X(n).”
(e) Let a1 , . . . , ak ∈ N . Modify the game so that you can remove
a1 , . . . , ak sticks. Show that there exists A ∈ N and X ⊆ {0, . . . , A−
1} such that, for all but a finite number of n ∈ N , if n sticks are
in the pile then
Player I wins iff (∃x ∈ X)[n ≡ x (mod A)].
3. (a) Consider the following game played between FILLER and EMP-
TYIER There is initially a box with a finite (nonempty) number
of balls in it. Each ball has a natural number on it, called its
rank. (There may be many balls of the same rank. For ex-
ample, a box could have 100 balls of rank 1, 20000 of rank 2,
and 4 of rank 1000.) In every round EMPTYIER makes the
first move. EMPTYIER’s move consists of taking a ball from
the box. FILLER then counters be replacing the ball with a
finite number of balls of lower rank. (For example, he could re-
place a ball of rank 1000 with 999999999999999 balls of rank 999,
888888888876234012 balls of rank 8, and 4 balls of rank 1.) EMP-
TYIER wants the box to be empty eventually. FILLER wants
the box to never be empty. Show that no matter what the initial
box of balls has in it, EMPTYIER can win.
(b) Consider the same game as above except that the balls are labeled
with integers. Fill in the following sentence and prove it: “Player
XXX can always win.”
(c) Consider the same game as above except that the balls are labeled
with positive rationals. Fill in the following sentence and prove
it: “Player XXX can always win.”
(d) Consider the same game as above except that the balls are labeled
with ordered pairs of numbers, and we consider (a, b) to be of
lower rank than (c, d) if either a < c or a = c and b < d. Fill
in the following sentence and prove it: “Player XXX can always
win.”
(e) Fill in the following sentence and prove it: “If the game can
be played with a set A with ordering ≤ such that the ordering
has property XXX then the FILLER can win; however, if it has
property NOT(XXX) then the EMPTYIER can win.”
2
4. Consider the following game of “NIM”. There are two piles of stones
of different sizes. Players alternate turns. At each turn a player may
take as many stones as he or she likes from one (and only one) pile,
but must take at least one stone. The goal is to take the last stone.
The following is a winning strategy for the first player: Take stones
from the larger pile so as to make the two piles have equal size. Prove
that this really is a winning strategy using mathematical induction.
5. Consider the following 2-player game. There are n sticks on the table.
Player I can remove 1,2 or 3 sticks. Player II can remove 2,4, or 6
sticks. The player who removes the last stick WINS. Show that:
if n ≡ 1 mod 2 then player I has a winning strategy.
3
(a) P (10).
(b) (∀k)[P (k) → P (k − 2)].
a0 = 6
a1 = 21
a2 = 11
an = 3an−1 + an−3 + 10n + 2
10. Let P (n) be some property of the natural number n (e.g. P (n) could
be ‘217n ≡ 10 (mod 8)’). Assume that P (10) and P (11) are known
to be true. Assume that
12. Let an be defined as below. Show that (∀n ≥ 1)[an ≡ 0 (mod 3)].
a0 = 3
a1 = 6
(∀n ≥ 3)[an = (an−1 + an−2 )2 + 6n]
13. Let P (n) be some property of the natural numbers. Assume that
the following are known (1) For all primes q, P (q) is true, and (2)
(∀n ∈ N )[P (n) → P (n2 )]. For which n ∈ N can we conclude P (n) is
true? Express your answer clearly. No explanation is needed.
4
14. Let P (z) be some property of the integers. Assume that the following
are known
(a) P (−23).
(b) (∀z ∈ Z)[P (z) → P (z + 3)].
17. Describe the set of all a ∈ N such that the following is true:
Show that
n
(∀n ≥ 1)[an ≤ 4 · 3(3 ) ].
Show that
(∀n ≥ 0)[an ≡ 4 (mod 5)].
5
21. Let P (q) be some property of the rational q. Assume that the following
are known
(a) P (1).
(b) (∀q ∈ Q)[P (q) → P ( 2q )].
Let A be the set of all q such that we can conclude P (q) is true.
Write down the set A clearly but DO NOT use the notation “. . .” No
justification required.
24. Let T (n) ≤ T ( 34 n) + T ( 14 n) + 4n. Find a value of d such that for all
n, T (n) ≤ dn log n.
25. Let T (n) ≤ T (an) + T (bn) + cn. For what values of a, b and c is it
possible to find a d such that for all n, T (n) ≤ dn log n.
T (n) = T (n − log n) + n.
6
(a)
n
n(n + 1)(n + 2)
i(i + 1) =
i=1
3
(b)
n
i2i−1 = (n − 1)2n − 1
i=0
30. Assume every node of a tree with n nodes has either 0 or 3 children.
How many leaves does the tree have? Prove your answer.
n n−j
n
1
i=1 j=i k=1
for i = 1 to n do
for j = i to 2*i do
for k = n-j+1 to n do
output ‘foobar’
34. a. Assume that Christmas has n days. Exactly how many presents
did my “true love” send me? [If you do not understand this
question, you should do some research.]
7
b. Using part (a), exactly how many presents did my “true love”
send during the twelve days of Christmas?
∞
35. Evaluate the sum k=0 (3k − 1)x3k for 0 < x < 1.
n
36. Evaluate k=1 4 · 3k
for i=1 to n do
for j=i to 2*i do
output ‘foobar’
8
40. Given an m × n array of (positive and negative) numbers, find the
largest sum of values in a (contiguous) rectangle.
a. Write down an English description of the “brute force” algorithm
for the “maximum contiguous rectangle problem”. One or two
sentences should suffice.
b. Write down the “brute force” algorithm in pseudocode.
c. How many times is the inner loop executed? Write it using sum-
mations.
d. Simplify your answer. Justify your work. [If you do this right,
the solution involves very little calculation.]
e. Find a better algorithm for the “maximum contiguous rectangle
problem”. How well can you do?
n 3
41. Approximate k=1 k using integrals. Make sure to get an upper and
lower bound.
n 4
42. Approximate k=1 3k−2
∞ 1
43. Use integrals to give upper and lower bounds for k=1 k 3/2 .
√
44. Appoximate nj=0 j using an integral.
45. Consider the following code fragment.
for i=1 to n do
for j=1 to lg i do
print ‘foobar’
9
47. Solve the recurrence
T (n) = T (n − 2) + n n>1
0 otherwise
assuming n is even. Show your calculations.
10
56. Consider the following recurrence, defined for n a power of 5:
7 if n = 1
T (n) =
4T (n/5) + 2n − 3 otherwise
a. Let n be a power of 2.
4 if n = 1
T (n) = 2
5T (n/2) + 3n otherwise
b. Let n be a power of 4.
3 if n = 1
T (n) =
2T (n/4) + 4n otherwise
11
61. Consider the following recurrence.
0 if n = 0
T (n) =
T ( n/2 ) + T ( n/4 ) + 3n otherwise
64. Let
2T (n/2) + 2 lg n if n > 1
T (n) =
0 if n = 1
12
(b) The nodes on level i are all nodes distance i from the root. Tree
Sk will have nodes on levels 0, . . . , k (and thus have k + 1 levels).
For S3 , how many nodes are on levels 0, 1, 2, and 3.
(c) Make a conjecture for how many nodes are on level i of Sk . (You
may also want to look at S4 .)
(d) Prove your conjecture.
13
Floors and Ceilings
1. Let r be a real number. Determine all real numbers such that the
following statement is true: r + = r.
6. For which real numbers x does x satisfy the following two conditions:
(a) 1 < x < 2, and (b) x2 = x2 . (HINT: Express x as 1 + where
0 < < 1.)
14
Combinatorics
1. For this problem we express numbers in base 10. The last digit of a
number is the rightmost digit of the number (e.g., the last digit of
39489 is 9). Find the largest x ∈ N such that the following statements
is true:
(a) How many different lunchs can Bill make for Carolyn ?
(b) If Carolyn does not like to have egg salad with apple pie, or tuna
fish with an apple, then how many lunches can Bill make for
Carolyn.
(c) Why does Bill call Carolyn his ‘gnilrad’ ?
15
Probability
2. You are given a deck of fifty two cards which have printed on them a
pair of symbols: an integer between 1 and 13, followed by one of the
letters “S,” “H,” “D,” or “C,” There are 4 × 13 = 52 such possible
combinations, and you may assume that you have one of each type,
handed to you face down.
(a) Suppose the cards are randomly distributed and you turn them
over one at a time. What is the average cost (i.e. number of
cards turned over) of finding the card [11,H]?
(b) Suppose you know that the first thirteen cards have one of the
letters “H” or “D” on them, but otherwise all cards are randomly
distributed. Now what is the average cost of finding [11,H]?
(a) The only inputs are {1, 2, . . . , n}, n is large, and n is divisible by
2.
(b) If the input i is one of {1, 2, . . . , n2 } then P takes i4 steps.
(c) If the input i is one of { n2 + 1, . . . , n} then P takes 2i steps
16
Find a distribution on the inputs such that every input has a nonzero
probability of occurring and the expected running time is θ(n17 ).
4. Suppose we have one loaded die and one fair die, where the probabil-
ities for the loaded die are
(a) What is the probability that a toss of these two dice produces a
sum of either seven or eleven?
(b) What is the expected value of the sum of the values of the dice?
(c) the product of the values of the dice?
(a) The only inputs are {1, 2, . . . , n}, n is large, and n is divisible by
3.
(b) If the input is one of {1, 2, . . . , n3 } then P takes log n steps.
17
(c) If the input is one of { n3 + 1, . . . , 2n
3 } then P takes n steps
(d) If the input is one of { 2n n
3 + 1, . . . , n} then P takes 2 steps
Find a probability distribution on the inputs such that every input has
a nonzero probability of occurring, but the expected time is θ(n).
9. Consider the following experiments. For each of them find the expected
value and the variance.
(a) Roll a 6-sided die and a 4-sided die. Add the two values together.
Both dice are fair.
(b) Roll a 2-sided die and a 3-sided die. Multiply the two values
together. Both dice are fair.
(c) Roll a 3 sided die and another 3-sided die. Add the two values
together. The first die has probability p1 of being 1, p2 of being
2, and p3 of being 3. The second die is fair. (In this case E and
V will be in terms of p1 , p2 , and p3 .)
18
11. Given n (n > 10) numbers in a sorted list, we know that in general,
the worst case cost for linear search is n, and its average cost is (n +
1)/2, whereas the average case for binary search is (roughly) log n (its
actually (log n)− c for some constant c). Suppose instead that we have
a special situation where the probability that the ith item is sought is
pi , and (1) for i ≤ 10 we have pi = p, and (2) for i > 10 we have the
probability that it is in position i is pi = 1−10p
n−10 . What is the average
case cost of linear search with this probability distribution (the answer
will be in terms of p). For this problem do NOT use Θ-notation, you
will need to look at exact results. Using a calculator find values of
n > 10 and p < 1 such that linear search performs better than log n
average case.
12. We are going to pick a number from {1, . . . , n} There are constants
p1 , p2 , . . ., and a quantity A(n), such that the probability of picking i
is pi = A(n)i. What is A(n)?
13. Let n be a large number. Assume we pick numbers from the set
{1, . . . , n}, and after we get a number we square it. (Formally the
sample space is {1, . . . , n} and the random variable is X(a) = a2 .)
Find a probability distribution such that E(X) = θ(log n).
(a) What is the expected run time if the distribution is uniform? You
may use Θ-notation.
(b) Find a distribution where the expected time is Θ(n2 ). (In this
distribution all elements must occur with non-zero probability.)
19
16. Let n be a large number. Let f be a function. Assume we pick
numbers from the set {1, . . . , n}, and after we get a number we apply
f to the result. (Formally the sample space is {1, . . . , n} and the
random variable is X(i) = f (i).)
17. You play a game of throwing two six sided dice (with integer values
1, . . . , 6). You lose a dollar if the two dice have different values. You
win d dollars if the two dice land on the same value d (i.e. you throw
double d’s).
18. Consider an s-sided die, where the sides are marked with the numbers
1, 2, . . . , s, and each side has equal probability (1/s) of being rolled.
20
(f) What is the probability of two rolls of this die both being s given
that you know that at least one of the two rolls was an s. (Note:
The roll that was an s is NOT necessarily the first roll.)
19. A combination lock works by turning to the right three times and
stopping a on particular integer from 1 to 60 (inclusive), turning to
the left two times and stopping on a particular integer from 1 to 60,
and then turning to the right and stopping on a final integer from 1
to 60. How many possible combinations are there.
21
b. What is the variance?
c. What is the standard deviation?
(b) Assume n independent such dice are rolled and their values are
summed.
a. What is the expected value?
b. What is the variance?
c. What is the standard deviation?
24. Assume every child born has probability 1/2 of being a boy or a girl.
Assume that (even within a family) children’s sexes are independent.
(a) Mr. Jones has two children. The older one is a boy. What is the
probability that they are both boys? Why?
(b) Mr. Smith has two children. At least one is a boy. What is the
probability that they are both boys? Why?
(c) Mr. Johnson has three children. At least one is a boy. What is
the probability that at least one is a girl? Why?
22
Asympototics
1. For each of the following expressions f (n) find a simple g(n) such that
f (n) = Θ(g(n)).
(a) f (n) = ni=1 1i .
(b) f (n) = ni=1 1i .
(c) f (n) = ni=1 log i.
(d) f (n) = log(n!).
23
7. Let a, b be real numbers such that a, b > 0. Consider the recurrence
T (n) = T ( na )+ T ( nb )+ n. Specify an infinite number of values for (a, b)
such that T (n) = Θ(n).
T (0) = 1
T (n) = 81T ( n3 ) + cn
Find a function g(n) such that T (n) = Θ(g(n)). (The function g may
involve several cases depending on what c is. The cases should be
clearly and easily specified.)
n ia
10. (a) For what values of a, b does the following hold? i=0 bi = O(1).
n
(b) For which constants a is it the case that 3 + 2n = O(an )?
2
n
11. (a) Find a function f such that i=1 i log i = Θ(f (n)).
√
(b) Find a simple function f such that ni=1 i log i = Θ(f (n)). Ex-
plain your answer.
12. For each of the following expressions f (n) find a simple g(n) such that
f (n) = Θ(g(n)). (You should be able to prove your result by exhibiting
the relevant parameters, but this is not required for the hw.)
n 4
(a) f (n) = i=1 3i + 2i3 − 19i + 20.
n
(b) f (n) = i i
i=1 3(4 ) + 2(3 ) − i19 + 20.
n i 2i
(c) f (n) = i=1 5 + 3 .
24
n i
(b) i=1 3 = Θ(3n ).
n i
(c) i=1 3 = Θ(3n+1 ).
∞ i5
14. Show that i=0 i = O(1).
43
(a) Find a function fa that does not involve summations such that
n −a
i=1 i = Θ(fa (n)). (The function fa might involve several
cases and might depend on a.)
(b) Find a function gb that does not involve summations such that
n i
i=0 b = Θ(gb (n)).
(c) For what values of a and b does fa (n) = O(gb (n))? (Only consider
the cases where 0 ≤ a ≤ 1, and b ≥ 0.)
(d) For what values of a and b does gb (n) = O(fa (n))? (Only consider
the cases where 0 ≤ a ≤ 1, and b ≥ 0.)
16. Let a, b be real numbers such a > 0 and b > 1. Consider the recurrence
T (n) = aT ( nb ) + n. You are given the option of either reducing a or b
by 1. You want to make T (n) as small as possible. For which values
of a, b would reducing a by 1 reduce T (n)? For which values of a, b
would reducing b by 1 reduce T (n)?
17. Let a, b be real numbers such that a, b > 0. Consider the recurrence
T (1) = 10, T (n) = T ( na ) + T ( nb ) + n. Specify an infinite number of
values for (a, b) such that T (n) = Θ(n).
√
18. Let T (1) = 10 and T (n) = T ( n) + n. Show that T (n) = O(n).
19. Let a, b be real numbers such that −100 ≤ a ≤ 100 and b > 0.
(a) Find a function fa that does not involve summations such that
n2 7a
i=0 i = Θ(fa (n)). (The function fa might involve several
cases and might depend on a. Each case should be simple.)
(b) Find a function gb that does not involve summations such that
√n 12i
i=1 b = Θ(gb (n)). (The function gb might involve several
cases and might depend on b. Each case should be simple.)
20. For each of the following functions f find a simple function g such that
f (n) = Θ(g(n)).
25
(a) f1 (n) = (1000)2n + 4n .
√
(b) f2 (n) = n + n log n + n.
(c) f3 (n) = log(n20 ) + (log n)10 .
(d) f4 (n) = (0.99)n + n100 .
22. Let a, b be real numbers such that a > 0 and −100 ≤ b ≤ 100.
23. (a) Find a function fa that does not involve summations such that
n2 7i
i=1 a = Θ(fa (n)). (The function fa might involve several
cases and might depend on a. Each case should be simple.)
(b) Find a function gb that does not involve summations such that
n 5b
i=0 i = Θ(gb (n)). (The function gb might involve several cases
and might depend on b. Each case should be simple.)
(c) For what values of a and b does fa (n) = O(gb (n))? (Only consider
the cases a > 0 and −100 ≤ b ≤ 100.)
(d) For what values of a and b does fa (n) = Ω(gb (n))? (Only consider
the cases a > 0 and −100 ≤ b ≤ 100.)
Find a function ga,b (n) such that T (n) = Θ(g(n)). The function ga,b
should be as simple as possible. (The function ga,b may involve several
cases depending on what a, b is. The cases should be clearly and easily
specified. NO PROOF REQUIRED.)
26
A B
(a) n100 2n
√
(b) (lg n)12 n
√
(c) n ncos(πn/8)
(d) 10n 100n
(e) n lg n (lg n)n
(f ) lg (n!) n lg n
27
Graph Problems
1. Let Kn denote the complete graph with n vertices (i.e. each pair
of vertices contains one edge between them), and let Kn denote the
modification of Kn in which one edge is removed.
3. Find a number a, 0 < a < 1 such that the following statement is true
“If G is a planar graph on n vertices then at least an vertices have
degree ≤ 99.”
(a) For which values of (n1 , n2 ) does C(n1 , n2 ) have an Eulerian cy-
cle?
(b) For which values of (n1 , n2 ) does C(n1 , n2 ) have a Hamiltonian
path?
(c) For which values of (n1 , n2 ) is C(n1 , n2 ) 5-colorable but not 4-
colorable?
28
(a) Eulerian but not Hamiltonian.
(b) Hamiltonian but not Eulerian.
(c) Hamiltonian and 2-colorable.
8. Consider the following real world problem: Given n cities you want to
build a highway system such that, for any two cities x and y, there is
a way to go from x to y that only passes through at most one other
city. You want to minimize the number of roads you build. STATE
this problem rigorously as a problem in graph theory.
9. For every n, find the maximal number of edges that a graph with n
vertices can have and still be unconnected.
10. Show that if G is a planar graph then at least half the vertices have
degree ≤ 10.
11. Give an example of a planar graph with an Eulerian path that has 2
vertices of degree 5.
13. Let Wn be the ‘wheel graph’ (pictured below). Formally it has vertices
{1, 2, . . . , n, n + 1} and edge set is the union of the sets
{(1, 2), (2, 3), . . . , (n − 2, n − 1), (n − 1, n), (n, 1)}
and
{(1, n + 1), (2, n + 1), . . . , (n, n + 1)}.
We only consider n ≥ 3.
29
3
If G is a connected planar graph then at least at least 4 of the vertices
have to have degree less than c.
You must prove that your answer is correct. You may assume that the
number of vertices in G is divisible by 4.
18. Let G be a bipartite graph with m nodes on the left side and n nodes
on the right side. What is the maximum number of edges G can
have? What is the maximum number of edges G can have and still be
disconnected?
30
20. The courses 150, 251, 451, and 650 must be taught. Bill can teach 150
or 650. Clyde can teach 150 or 451. Diane can teach 451 or 650. Edna
can teach 150 or 251.
21. Let T be a weighted tree. Let f be some function that is easily com-
puted (it goes from Naturals to Naturals). An f -minimal spanning tree
is a spanning tree where the weights are w1 , ..., wm and mi=1 f (wi ) is
minimal among all spanning trees. Give an efficient algorithm to find
the f -minimal spanning tree of a weighted tree.
24. Prove that in a graph G = (V, E), the number of vertices that have
odd degree are even. (Hint: First show that v∈V deg(v) = 2m where
m is the number of edges in G.)
31
25. Describe concisely and accurately a linear-time algorithm for solving
the single-source shortest paths problem in a weighted DAG. Give a
rigorous, concise proof that your algorithm is correct.
Hints: Topological sort; relaxation lemma; try your alg. on examples.
26. Using depth first search, outline an algorithm that determines whether
the edges of a connected, undirected graph can be directed to produce
a strongly connected directed graph. If so, the algorithm outputs such
an orientation of the edges. (Hint: Show that this can be done if and
only if removing an edge leaves a connected graph.)
27. An Euler Circuit for an undirected graph is a path which starts and
ends at the same vertex and uses each edge exactly once.
28. A bipartite graph is one whose vertex set can be partitioned into two
sets A and B, such that each edge in the graph goes between a vertex
in A and a vertex in B. (No edges between nodes in the same set are
allowed.)
Give an O(|E| + |V |) algorithm that takes an input graph (represented
in adjacency list form) and decides if the graph is bipartite. If the
graph is bipartite, the algorithm should also produce the bipartition.
(Hint: use BFS.)
32
(One naive solution would be to show all movies on both days, so that
each customer can see one desired movie on each day. We would need
k channels if k is the number of movies in this solution – hence if there
is a solution in which each movie is shown at most once, we would like
to find it.)
30. Consider a rooted DAG (directed, acyclic graph with a vertex – the
root – that has a path to all other vertices). Give a linear time (O(|V |+
|E|)) algorithm to find the length of the longest simple path from the
root to each of the other vertices. (The length of a path is the number
of edges on it.)
max δ(u, v)
u,v∈V
(where δ(u, v) is the number of edges on the path from u to v). De-
scribe an efficient algorithm to compute the diameter of a tree, and
show the correctness and analyze the running time of your algorithm.
Hint: Use BFS or DFS.
32. Show that the vertices of any n-vertex binary tree can be partitioned
into two disconnected sets A and B by removing a single edge such
that |A| and |B| are at most 3n/4. Give a simple example where this
bound is met.
33
34. a Show that if (u, v) is the lightest edge on a cut, there exists a
MST containing (u, v).
b Show that if (u, v) is the heaviest edge on a cycle, there exists a
MST not containing (u, v).
37. Show that the vertices of any n-vertex binary tree can be partitioned
into two disconnected sets A and B by removing a single edge such
that |A| and |B| are at most 3n/4. Give a simple example where this
bound is met.
34
Data Structures
2. Recall that a heap allows inserts and deletes in O(log n) steps, and
FINDMAX in O(1) steps. Devise a structure that allows inserts and
deletes in O(log n) steps and both FINDMAX and FINDMIN in O(1)
steps.
35
structure should take up no more than O(N +n) space, and each query
should take O(k + log log n).
11. A Deap is a data structure that is a tree such that (a) there is nothing
at the root, (b) the left subtree is a Max-heap, (c) the right subtree
is a Min-heap, (d) every leaf in the Max-heap is bigger that its corre-
sponding leaf in the Min-heap.
Show how to implement FINDMIN, FINDMAX in O(1), INSERT,
and DELETEMIN and DELETEMAX in O(log n), and CREATE in
O(n). (My notes about the reference claim that INSERT can be done
in O(log log n) steps — but I am not sure I believe that.
36
12. Devise a Data Structure for a weighted Directed Graph on n nodes
that can do the following operations quickly.
13. Devise a Data Structure for a directed graph that can answer the
following three types of queries.
14. Consider the move-to-front heuristic where you decide whether or not
to move the item up probabilistically: the more often the item has
been accessed, the more likely to move it up. Write pseudocode to
implement this heuristic.
15. Devise a data structure to store n horizontal lines so that you can
tell, given an infinite vertical line, if it hits one of them. The data
structure should take O(n2 ) space and be able to answer the query in
O(1). Look into variants.
37
Analysis of Algorithms
1. For the following program find a function f (n) such the runtime is
Θ(f (n)).
BEGIN
Input(n)
For i := 1 to n do [TAKES 5i STEPS]
i := n
n
While i ≥ 2 do
begin
SOMETHING THAT TAKES i STEPS
i := i − 1
END
Ux = c
for the unknown vector x, where U has n rows and columns and x
and c are vectors of size n. How many multiplications are required?
Justify your answer.
38
Divide and Conquer
a. How fast (just the order, i.e. do not worry about the constants)
is a new “fancy” multiplication algorithm based on this fact?
b. How small does k have to be for this algorithm to be asymptoti-
cally faster than the “standard” (O(n2 )) algorithm?
c. How small does k have to be for this algorithm to be asymptoti-
cally faster than the O(nlg 3 ) algorithm?
(a) Exactly how many multiplications does the standard matrix mul-
tiplication algorithm use to multiply two n × n matrices? Exactly
how many additions does it use? What is the exact time it uses
to multiply two n × n matrices?
(b) What is the exact time to multiply two n × n matrices using
Strassen’s algorithm?
(c) For what values of n is Strassen’s algorithm better than the stan-
dard algorithm? Justify your answer.
39
Dynamic Programming
3. Consider the following greedy algorithm for solving the chained matrix
multiplication problem: Let the matrices A1 , ..., An , have dimensions
p1 × q1 , ..., pn × qn . Look for the smallest dimension, say qi−1 and pi ,
and parenthesize between matrices Ai−1 and Ai , so that we multiply
A1 to Ai−1 in the best possible way, Ai to An in the best possible
way, and finally multiply the two resulting matrices. In order the
determine how to multiply A1 to Ai−1 and Ai to An we apply the
algorithm recursively.
Show that this greedy algorithm does not find the optimal way to
multiply a chain of matrices.
40
4. Write an efficient algorithm to determine an order of evaluating the
matrix product M1 × M2 × M3 . . . × Mn so as to minimize the scalar
multiplications in the case where each M is of dimension 1 × 1, 1 ×
d, d × 1, d × d for some fixed d.
5. We are given a collection of n books, which must be placed in a library
book case. Let h[1..n] and w[1..n] be arrays, where h[i] denotes the
height of the i-th book and w[i] denotes the width of the i-th book.
Assume that these arrays are given by the books’ catalogue numbers
(and the books MUST be shelved in this order), and that books are
stacked in the usual upright manner (i.e., you CANNOT lay a book
on its side). All book cases have width W , but you may have any
height you like. The books are placed on shelves in the book case.
You may use as many shelves as you like, placed wherever you like (up
to the height of the book case), and you may put as many books on
each shelf as you like (up to the width of the book case). Assume for
simplicity that shelves take up no vertical height.
The book-shelver’s problem is, given h[1..n], w[1..n], and W , what is the
minimum height book case that can shelve all of these books. Below
shows an example of a shelving. The height and width of the 6th
book are shown to the right. (Notice that there is some unnecessarily
wasted height on the third shelf.)
h[6]
w[6]
H
41
6. Consider the problem of not only finding the value of the maximum
contiguous subsequence in an array, but also determining the two end-
points. Give a linear time algorithm for solving this problem. [What
happens if all entries are negative?]
10. Consider the problem of computing the probability of which team will
win a game consisting of many points. Assume team A has probability
p of winning each point and team B has probability q = 1−p of winning
each point. Let P (i, j) be the probability that team A will win the
game given that it needs i points to win and team B needs j points.
42
(a) Write down a recurrence for P (i, j).
(b) Use your recurrence to develop an efficient dynamic programming
algorithm for determining the probability that team A will win
the game given that it needs i points and team B needs j points.
(c) Analyze the running time of your algorithm.
(a) Write down a recurrence for the probability that the champion
retains the title. Assume that there are g games left to play in
the match and that the champion needs to win i games (which
may end in a 1/2).
(b) Based on your recurrence, give a dynamic programming to cal-
culate the champion’s probability of retaining the title.
(c) Analyze its running time assuming that the match has n games.
43
0 1 0 0 1 0 1 0 0 1
0 1 0 0 1 0 1 0 0 1
1 1 0 0 1 1 1 0 0 1
1 1 1 0 0 1 1 1 0 0
0 0 1 0 0 0 0 1 0 0
44
Greedy Algorithms
3. Assume that you are given a set {x1 , . . . , xn } of n points on the real
line and wish to cover them with unit length closed intervals.
(a) Describe an efficient greedy algorithm that covers the points with
as few intervals as possible.
(b) Prove that your algorithm works correctly.
4. While walking on the beach one day you find a treasure trove. Inside
there are n treasures with weights w1 , . . . , wn and values v1 , . . . , vn .
Unfortunately you have a knapsack that only holds a total weight M .
Fortunately there is a knife handy so that you can cut treasures if
necessary; a cut treasure retains its fractional value (so, for example,
a third of treasure i has weight wi /3 and value vi /3).
45
path estimate d[v] increases rather than decreases. True or False: This
modified algorithm determines the cost of the longest simple path from
the source to each vertex. If True then prove your answer, and if False
then give a counterexample.
(a) Is the Minimum cost spanning tree in G, the Maximum cost span-
ning tree in G ? Prove or disprove.
(b) Suppose that P is the shortest path from s to v in G. Is P the
longest simple path from s to v in G ? Prove, or disprove.
46
(a) He decreases the cost of an edge e that belongs to T .
(b) He increases the cost of an edge e that belongs to T .
(c) He decreases the cost of an edge f that does not belong to T .
(d) He increases the cost of an edge f that does not belong to T .
You do not need to design an algorithm, but to simply state the edges
that need to be considered.
10. Prove that if a connected undirected graph has distinct edge weights,
then the minimum cost spanning tree is unique. (Hint: Recall the
proof of Kruskal’s algorithm.)
11. Prove the following lemma:
If an edge of an undirected weighted graph is the unique most expensive
edge on a cycle, then it is not in any minimum-weight spanning tree.
12. Modify Dijkstra’s algorithm as follows. (1) Initialize all d[v] values to
−∞, (2) rather than performing an Extract-Min, perform an Extract-
Max, and (3) in relaxation, reverse all comparisons so that the shortest
path estimate d[v] increases rather than decreases. True or False: This
modified algorithm determines the cost of the longest simple path from
the source to each vertex. If True then prove your answer, and if False
then give a counterexample.
13. A coloring of an undirected graph is assignment of colors to the nodes
so that no two neighboring nodes have the same color. In the graph
coloring problem, we desire to find the minimum number of colors
needed to color an undirected graph.
One possible greedy algorithm for this problem is to start with an
arbitrary node and color it “1”. Then to consider every other node
one at a time and color it “1” if it does not neighbor a node already
colored “1”. Now do the same for all of the remaining nodes using the
color “2”. Continue in this fashion until all of the nodes in the graph
are colored.
(a) How would you implement this algorithm efficiently? Be brief but
clear. What is your running time as a function of the number of
vertices |V |, edges |E|, and colors |C|.
(b) Prove that for every graph, the algorithm might get lucky and
color it with as few colors as possible.
47
(c) Prove that the algorithm can be arbitrarily bad. In other words,
for every constant α > 0 the ratio between the number of colors
used by the greedy algorithm and by the optimal algorithm could
be at least α.
48
Sorting and Selection
1. Assume you have some kind of super-hardware that, when given two
lists of length m that are sorted, merges them into one sorted list,
√
and takes only m steps. Write down a recursive algorithm that uses
this hardware to sort lists of length n. Write down a recurrence to
√
describe the run time. (For your own enlightenment replace m with
other functions and see what happens.)
49
7. Let P be a program that has the following format.
PROCEDURE P
begin
Input(n) (this takes 0 steps)
if n = 1 then execute 4 operations and stop. (if n = 1 this takes 0
operations.)
Execute n operations
Call P on input n3
Execute n operations
Call P on input n4
end
Write a recurrence for T (n), the running time of this program. Find
a number c such that for all n ≥ 1, T (n) ≤ cn.
10. Devise a structure that allows MAX and MIN in O(1) steps, INSERT
in O(log n) steps, and DELETE in O(n). (For your own enlighten-
ment, and ONE extra point, come up with a structure that allows
MAX and MIN in O(1) steps, INSERT in O(log n), and DELETE
also in O(log n).)
11. You wish to store a set of n numbers in either a heap or a sorted array.
(Heap is MAX-heap, has MAX element on top, etc.) For each of the
following desired properties state whether you are better off using a
heap or an ordered array, or that it does not matter. Justify your
answers.
50
(a) Want to find the MAX quickly.
(b) Want to be able to delete an element quickly.
(c) Want to be able to form the structure quickly.
(d) Want to find the MIN quickly.
12. Given n elements we are going to, in the course of time, (1) do an add
or delete a times, (2) try to find MAX b times, (3) try to find MIN
c times. We can assume a, b, c are much smaller than n. For what
values of a, b, c are we better off using a heap? For what values are we
better off using a sorted array. (You must also take into consideration
the time spent forming these structures.)
13. Assume that we are going to be sorting arrays A[1..n] but that we have
a MAGIC algorithm that sorts arrays of size ≤ n3 in n steps. Rewrite
the pseudocode for QUICKSORT on page 154 so that it can use this
MAGIC algorithm. What is the worst case performance? What is the
best case performance?
14. For each of the following scenarios indicate if you would use QUICK-
SORT or HEAPSORT and why.
15. Assume that you have hardware that can sort log n elements in ONE
step. We are thinking of sorting a list of n elements using this hard-
ware. What is a LOWER BOUND on how well we can do? Explain
your result. (You can assume that the only operations allowed are
comparisons, the operation above, and swaps. Hence a variation of
the decision tree model is appropriate.)
51
17. The algorithm for bucket sort is geared to take inputs in the range
[0, 1) (fractions allowed of course). Let p be a positive natural number.
Adjust the algorithm so it takes inputs that are natural numbers in
{1, . . . , pn} and uses p buckets. Discuss informally how this algorithm
might behave if the inputs are chosen with the distribution P rob(x =
i) = Ki (Where K is the appropriate constant). How might you adjust
the algorithm so that it performs well for this distribution?
18. Consider the following two statements: (1) Sorting n elements RE-
QUIRES Ω(n log n) operations, (2) Radix sort, counting sort, and
bucket sort perform in time substantially better than O(n log n). These
statements seem to be in contradiction. Explain carefully why they
are not.
19. Assume that you can find the median of 7 elements in 12 comparisons.
Devise a linear-time MEDIAN FINDING algorithm that uses this.
Give an upper bound on the number of comparisons used (do NOT
use O-notation). Try to make the bound as tight as possible.
20. Let a, b be numbers. Assume that you can find the median of a el-
ements in b comparisons. Use this in the linear-time median finding
algorithm. Give an upper bound on the number of comparisons used
(do NOT use O-notation). Try to make the bound as tight as possible.
Find values of a, b such that you can find medians in ≤ 4n comparisons.
21. For each of the following scenarios say whether you would use MERGE-
SORT, QUICKSORT (or variants), RADIX SORT, or COUNTING
SORT. Briefly explain why you chose the one you did.
(a) The elements to be sorted are numbers from the set {1, . . . , log n}
but the only operations we are allowed to use are COMPARE and
SWAP.
(b) We have access to a VERY FAST median finding algorithm that
we can use as a subroutine.
(c) The elements to be sorted are numbers from the set {1, . . . , n}
and we have access to the actual bits of the number.
(d) No bound is known on the elements to be sorted, but we need to
guarantee fast performance.
52
22. Assume that c ≥ 0. Assume you had some kind of super-hardware
that, when given two lists of length m that are sorted, merges them
into one sorted list, and takes only mc steps. Write down a recursive
algorithm that uses this hardware to sort lists of length n Write down
a recurrence to describe the run time. For what values of c does this
algorithm perform substantially better than O(n log n)?
23. Assume that n numbers are in a MAX-heap.
(a) Show that the third largest element can be found in O(1) steps.
(b) Show that if i is any constant then the ith largest element can be
found in O(1) steps
24. Let d(n) be a function of n such that d(n) << n. Imagine that we
redesigned MAX-heaps so that instead of every internal node hav-
ing 2 children (except possibly the rightmost internal node on the
second-to-last level, which may have 1 or 2 children), they had d(n)
children (except possibly the rightmost internal node on the second-
to-last level, which may have 1, . . . , d(n) children). How fast would the
following operations be? You may use Θ-notation. (a) FIND MAX,
(b) INSERT, (c) DELETE MAX, (d) FIND (given a key, determine
whether it is in the structure).
25. Let d ≥ 0. We have special hardware labeled MAGIC that, given
an array B[1..m], places the smallest m m
2 elements into B[1.. 2 ] (not
m m
necessarily in order), places the largest 2 into B[( 2 + 1)..m] (not
necessarily in order), and returns the index of the largest element in the
first partition (i.e., MAGIC does what PARTITION does in the usual
QUICKSORT algorithm). Assume that this hardware works in md
steps. Describe a sorting algorithm similar to QUICKSORT that uses
MAGIC to sort A[1..n]. For what d, will this sorting algorithm perform
(1) much better than Θ(n log n), (2) much worse than Θ(n log n), (3)
in Θ(n log n)?
26. In answering a, b, and c no explanations are needed. Also, QUICK-
SORT refers to the randomized version.
(a) Give a scenario where RADIXSORT is better than MERGE-
SORT.
(b) Give a scenario where MERGESORT is better than QUICK-
SORT.
53
(c) Give a scenario where QUICKSORT is better than RADIXSORT.
28. We are going to be sorting numbers in the interval [0, 1) using BUCK-
ETSORT. (We use the version in the book where there are n buckets
and each individual one is sorted by INSERTION SORT.)
(a) Find an increasing function f such that the following holds: ‘If
when doing BUCKETSORT we are guaranteed that no bucket
has more than f (n) elements in it, then BUCKETSORT is guar-
anteed to do substantially better than O(n log n).’
(b) Find an infinite number of such functions with different orders of
magnitude.
29. Assume you have some kind of super-hardware that, when given two
lists of length m that are sorted, merges them into one sorted list,
√
and takes only m steps. Write down a recursive algorithm that uses
this hardware to sort lists of length n. Write down a recurrence to
√
describe the run time. (For your own enlightenment replace m with
other functions and see what happens.)
31. Assume that we have MAGIC HARDWARE that can sort k elements
in 1 step.
(a) Give an algorithm for sorting (n elements) that uses this hard-
ware. What is its running time?
(b) Give a lower bound on how well any algorithm with this hardware
can do.
32. Assume that we have MAGIC HARDWARE that can find the MAX
of three elements in 1 step.
54
(a) Give an algorithm for MAX (of n elements) that uses this hard-
ware. What is its running time?
(b) Give a lower bound on how well any algorithm with this hardware
can do. It should match the upper bound.
33. Assume that the array A[1..n] only has numbers from {1, . . . , n2 } but
that at most log log n of these numbers ever appear. Devise an algo-
rithm that sorts A in substantially less than O(n log n).
34. In some applications the lists one wants to sort are already ‘partially
sorted’ Come up with a formal way to measure ‘how sorted’ a list
is. Devise an algorithm that does very well on lists that are almost
sorted (according to your definition). (Note— there is no one ‘right
answer’ to this problem, however whatever answer you give must be
well thought out. Your definition must be rigorous.)
35. Let a, b be real numbers such that a, b ≥ 0. Assume you have some
kind of super-hardware that, when given 12 sorted lists of length m,
merges them in am mb steps.
(a) Write an algorithm that uses this hardware to, given an array
A[1..n], sort it.
(b) Let T (n) be the running time of your algorithm. Find a function
fa,b such that T (n) = θ(fa,b ). (The function fa,b might involve
several cases.)
(c) For what values of a, b does your algorithm run substantially
faster than O(n log n)?
36. Let a FEAP be just like a HEAP except that the internal nodes have
FIVE children instead of TWO (only the right most internal node
might have less than FIVE children).
(a) Give an algorithm that does the following: given a FEAP and a
new element, remove the root of the FEAP and insert the new
element. (When you are done you must have a FEAP again.)
You must describe your algorithm informally (i.e., NO CODE).
Give an example of your algorithm at work.
(b) How many comparisons does your algorithm take in the worst
case? How many comparisons does your algorithm take in the
55
best case? How many moves does it make in the worst case?
How many moves does it make in the best case? (Do not ignore
multiplicative constants.)
37. Assume that the array A[1..n] only has numbers from {1, . . . , n1.2 } but
√
that all numbers that appear are multiples of n. (You may assume
√
that n and n1.2 are integers.) Devise an O(n) algorithm to sort A.
38. Let a, b be real numbers such that a, b ≥ 0. Assume you have some
kind of super-hardware that, when given an array A[1..m] returns
the following: the ( m th largest element, the ( 2m )th largest element,
5) 5
the ( 5 ) largest element, and the ( 4m
3m th
5 )
th largest element. Assume
this hardware runs in time ma (log m)b . (Formally the numbers above
5 , etc., but you may ignore this for the purposes of this
should be m
problem.)
(a) Write an algorithm that uses this hardware to sort lists of length
n.
(b) Let T (n) be the running time of your algorithm. Find a function
fa,b such that T (n) = θ(fa,b ). (The function fa,b might involve
several cases and might depend on a and b.)
39. (a) Exhibit an O(log n) algorithm that will, given a heap on n ele-
ments and a number m with m ≤ log log n, find and delete the
mth largest element in the heap. (The end result is a heap that
has that one element deleted.)
(b) Assume m is a constant. Find functions gB (n) and gW (n) such
that your algorithm takes gB (n) + θ(1) comparisons in the Best
case, and gW (n) + θ(1) comparisons in the Worst case. (Note
that multiplicative constants are important.)
(c) Assume m is log log n. Find functions hB (n) and hW (n) such
that your algorithm takes hB (n) + θ(1) comparisons in the Best
case, and hW (n) + θ(1) comparisons in the Worst case. (Note
that multiplicative constants are important.)
56
41. Assume that we have MAGIC HARDWARE that can find the 3rd
largest of log n elements in 1 step. Give a lower bound on how well
any algorithm with this hardware can do.
42. Let A be an array of positive integers (> 0), where A(1) < A(2) <
. . . < A(n). Write an algorithm to find an i such that A(i) = i (if one
exists). What is the running time of your algorithm ?
If the integers are allowed to be both positive and negative, then what
is the running time of the algorithm (Hint: redesign a new algorithm
for this part.) ?
44. The professor claims that the Ω(n lg n) lower bound for sorting n num-
bers does not apply to his workstation, in which the control flow of the
program can split three ways after a single comparison ai : aj , accord-
ing to whether ai < aj , ai = aj , or ai > aj . Show that the professor is
wrong by proving that the number of three way comparisons required
to sort n elements is still Ω(n lg n).
45. Let hi-lo BE the problem of, given n numbers, find the smallest and
largest elements on the list. You may assume that n is even.
47. Show how to implement quicksort in worst case O(n lg n) time. Your
algorithm should be high level and very brief. (HINT: We need to
choose the partition element carefully.) Supply the analysis, which
should also be very brief.
57
48. Consider the following quicksort-like sorting algorithm. Pick two el-
ements of the list. Partition based on both of the elements. So the
elements smaller than both are to the left, the elements in between are
in the middle, and the elements larger than both are to the right.
49. The usual deterministic linear time (worst case) SELECT algorithm
begins by breaking the set into groups of 5 elements each. Assume that
we changed the algorithm to break the set into groups of 7 elements
each.
a. Write down the recurrence for the new running time. (You can
let c be the number of comparisons needed to find the median of
7 elements.)
b. Solve the recurrence.
50. A d-ary heap is like a binary heap, but instead of two children, nodes
have d children.
58
(e) What value(s) of d are optimal? Justify your answer.
51. We are going to derive the average number of moves for quicksort using
a somewhat unusual partitioning algorithm. We partition on the first
element. Take it out. Look for the right most element that is smaller
and place it in the first position (which is the newly opened position
on the left side). Look for the left most element that is larger and
place it in the newly opened position on the right side. Starting from
there look for the right most element that is smaller and place it in
the the newly opened position on the left side. Starting from there
look for the left most element that is larger and place it in the newly
opened position on the right side. Continue in this fashion until the
pointers cross. Finally, put the partition element into the hole, which
is its final position in the sorted array.
52. Devise an algorithm that finds the second largest of four elements.
59
53. Let M AXn be the problem of finding the max of n numbers. Let
SECn be the problem of finding the second largest of n numbers. We
assume the decision tree model, i.e., only comparisons are the main
operation.
(a) Show that in any Decision Tree for M AXn , every branch has
depth at least n − 1. (HINT: if only n − 2 comparisons are made
then show that the maximum cannot be determined.)
(b) Show that any Decision tree for M AXn has 2n−1 leaves.
(c) Let T be a Decision tree for SECn . Let i be such that 1 ≤ i ≤ n.
Let Ti be formed as follows: take tree T and, whenever there is
a comparison between xi and xj , eliminate the comparison by
assuming that xi wins.
i. Argue that Ti is essentially a decision tree to find MAX on
{x1 , . . . , xn } − {xi }.
ii. Show that, for all i, Ti has 2n−2 leaves.
iii. Show that, for all i and j, the leaves of Ti and the leaves of
Tj are disjoint.
iv. Find a lower bound on the number of leaves in T by using
the last two items.
v. Show that T has depth n + lg n − O(1).
54. Assume you had a special hardware component that could find the
√
minimum of n numbers in one step. Give upper and lower bounds
for sorting using this component.
√
55. Assume you had a special hardware component that could sort n
numbers in one step. Give upper and lower bounds for sorting using
this component.
56. We are going to derive a lower bound for merging two lists of sizes
m, n. (Assume all of the elements are distinct.)
60
not both). Assume it is the first element of X. How many
arrangements of the remaining elements are there? Assume
it is the first element of Y . How many arrangements of the
remaining elements are there? Based on this information,
write down a recurrence for S(m, n). Show that m+nm is a
solution to your recurrence.
(b) Give a lower bound for the number of comparisons needed to
merge two lists. Simplify your answer using Stirling’s formula.
(c) What is your answer when m = n (write it as simply as possible).
How does it compare to the lower bound of 2n − 1.
57. Consider the problem of sorting a sequence of n 0’s and 1’s using
comparisons. For each comparison of two values x and y, the algorithm
learns which of x < y, x = y, or x > y holds.
58. Assume you have a list of n elements, where each element is within k
positions of its correct sorted position (on either side). (If k = 0 then
the deck is sorted.)
61
b. Give an efficient algorithm for solving this problem.
c. Analyze its running time.
60. (a) Assume you are given a number X and two sorted lists of n
numbers each: A1 ≤ A2 ≤ · · · ≤ An and B1 ≤ B2 ≤ · · · ≤
Bn . Design a linear (i.e. O(n)) time algorithm to determine how
many pairs (i, j) there are such that Ai + Bj ≤ X. Justify the
correctness and running time of your algorithm.
(b) Show how to preprocess the lists A and B (in any way that you
like and producing any data structure that you like) so that given
X, the pair counting problem can be solved in o(n) time. How
much preprocessing time do you use? How much storage is used?
How much time is needed to answer one query (after preprocess-
ing)?
61. Illustrate the operation of radix sort on the following list of English
words: RUT, TOP, SUP, POT, TON, OPT, TOR, ROT, TOO, OUT,
SUR, PUT
62. For this problem, assume that you have access to a procedure Merge(A,
B, C, na, nb), that merges two sorted arrays A[1..na] and B[1..nb]
into a single sorted array C[1..na+nb]. This procedure runs in O(na +
nb ) time. Assume we also have access to unlimited dynamic array al-
location.
Suppose that you have a collection of n, 1-dimensional arrays each of
length n, D[1..n][1..n], such that each array is sorted (i.e. D[i][j]
<= D[i][j+1]). We want to compute a single sorted array that con-
tains all n2 elements.
The first strategy for accomplishing this is called cascading merging.
First merge D[1] with D[2] to form an array of length 2n, then merge
this array with D[3] to form an array of length 3n, etc. The final
merge is between an array of length (n − 1)n with D[n] to form the
final sorted array of length n2 . See the figure below.
62
D[1] ❍❍ D[1] ❍❍
✟✟❍❍ ✟✟❍❍❍
D[2] ✟✟❍❍ D[2] ❍
D[3] ✟✟ ✟✟✟ D[3] ❍❍✟✟✟✟
✟
D[4] ✟✟ D[4] ✟✟
For both parts (a) and (b) count only the time spent in the merging
procedure.
63
a. Write down the recursive version of Selection Sort in pseudocode.
b. Derive a recurrence for the exact number of comparisons the al-
gorithm uses.
c. Use the iteration method to solve the recurrence. Simplify as
much as possible.
d. Use mathematical induction to verify your solution.
(a) Write a recurrence for how many comparisons this merging algo-
rithm uses.
(b) Solve the recurrence using the iteration method.
(c) Prove that this algorithm merges correctly.
64
67. Consider an n × n array A containing integer elements (positive, neg-
ative, and zero). Assume that the elements in each row of A are in
strictly increasing order, and the elements if each column of A are in
strictly decreasing order. (Hence there cannot be two zeroes in the
same row or the same column.) Describe an efficient algorithm that
counts the number of occurrences of the element 0 in A. Analyze its
running time.
65
NP-Completeness
66
6. We define A ≤w B if there are 2 functions f1 , f2 such that each one
can be computed in polynomial time and
x ∈ A iff f1 (x) ∈ B and f2 (x) ∈
/ B.
Show that if B ∈ P and A ≤w B then A ∈ P. (Assume f1 takes
time p1 (n), f2 takes time p2 (n), and the algorithm for B takes time
q(n). With this, you should be able to express the running time for
the algorithm for A in terms of p1 , p2 , and q.)
8. A stable set of vertices in a graph do not have any edges between them.
The optimization version of this problem is to find a maximum stable
set in a graph.
67
Show that all four of the problems above are in NP. For all pairs of
problems P1 , P2 from {HP, DHP, HC, DHC} show that P1 ≤p P2 .
(HINT on HP ≤p HC. You will need to change the graph. Replace
some vertex v ∈ V by two new vertices, x and y, that are adjacent to
the same vertices that v was. What else do you need to do? Try to
“force” any Hamiltonian path to visit these vertices first and last.)
10. Consider the problem DENSE SUBGRAPH: Given G, does it contain
a subgraph H that has exactly K vertices and at least Y edges ?
Prove that this problem is NP-complete. You could pick any problem
to reduce from.
11. Assume that the following problem is NP-complete.
PARTITION: Given a finite set A and a “size” s(a) (a positive integer)
for each a ∈ A. Is there a subset A ⊆ A such that a∈A s(a) =
a∈A−A s(a) ?
Now prove that the following SCHEDULING problem is NP-complete:
Given a set X of “tasks”, and a “length” (x) for each task, three
processors, and a “deadline” D. Is there a way of assigning the tasks
to the three processors such that all the tasks are completed within
the deadline D? A task can be scheduled on any processor, and there
are no precedence constraints. (Hint: first prove the NP-completeness
of SCHEDULING with two processors.)
12. Prove that the following ZERO CYCLE problem is NP-complete:
Given a simple directed graph G = (V, E), with positive and negative
weights w(e) on the edges e ∈ E. Is there a simple cycle of zero weight
in G? (Hint: Reduce PARTITION to ZERO CYCLE.)
13. Let A be an algorithm that accepts strings in a language L. Let p(n)
be a polynomial. Assume that A runs in p(n) time if a given string x is
in L, but in superpolynomial time if x is not in L. Give a polynomial
time algorithm for recognizing strings in L.
14. Prove that the following problem is NP-complete.
SPANNING TREE WITH A FIXED NUMBER OF LEAVES: Given
a graph G(V, E), and an integer K, does G contain a spanning tree
with exactly K leaves ?
(a) First prove that the problem is in NP by describing a “guessing”
algorithm for it. (Write out the algorithm in detail.)
68
(b) Now prove that the problem is NP-hard, by reducing UNDI-
RECTED HAMILTONIAN PATH to it.
HAMILTONIAN PATH asks for a simple path that starts at some
vertex and goes to some other vertex, going through each remain-
ing vertex exactly once.
15. Show that both of the following two problems are NP-complete. For
each show that the problem is in NP, and show that a known NP-
complete problem is reducible to this problem.
16. Assume that you are given access to function CLIQUE(G, K) that
returns “True” if G contains a clique of size at least K, and “False”
otherwise.
17. The cycle cover problem (CC) is: given a directed graph G = (V, E)
and an integer k is there a subset of vertices V ⊆ V of size k such
that every cycle of G passes through a vertex in V ? Prove that CC
is NP-complete. (Hint: The proof that CC is in NP is tricky. Note
that a graph can have an exponential number of cycles, so the naive
verification algorithm will not work. The reduction is from vertex
cover.)
69
True if the function is satisfiable and False otherwise. (This is very
unlikely since it would imply P = NP!) This function tells you nothing
about how to set the variables to satisfy the formula. Show how to use
this function to determine the actual variable assignment that makes
the formula True.
19. If G = (V, E) is a graph then a Dominating Set is a set V ⊆ V such
that, for all vertices v, either v ∈ V or v is adjacent to an element of
V .
(a) Let DOM = {(G, k) | G has a dominating set of size k}. Show
that DOM is NP-complete (Hint: Use the Vertex Cover prob-
lem.)
(b) Let P LAN AR−DOM = {(G, k) | G is planar and has a dominating set of size k}.
Show that P LAN AR − DOM is NP-complete (Hint: Use the
DOM problem.)
(c) Let IN D−P LAN AR−DOM = {(G, k) | G is planar and has a dominating set of size k tha
Show that IN D − P LAN AR − DOM is NP-complete (Hint: Use
the P LAN AR − DOM problem.)
(d) Fix k. Let IN D−P LAN AR−DOMk = {G | G is planar and has a dominating set of size k
Show that IN D − P LAN AR − DOMk can be solved in O(6k n)
time where the constant does not depend on k. (Hint: Use that
a planar graph always has a vertex of degree ≤ 5.)
20. Let q, k be a fixed constant. Let q-CNF be CNF where each clause has
≤ q literals. A formulas is MONOTONE if it has no negated variables.
(a) Let q − k − M ON O − SAT be the set of all q-CNF monotone
formulas that are satisfiable with an assignment that has at most
k variables set to TRUE. Show that q − k − M ON O − SAT can
be solved in O(2k n) where the constant does not depend on k.
(b) q − k − SAT be the set of all q-CNF formulas that are satisfiable
with an assignment that has at most k variables set to TRUE.
Show that q−k−SAT can be solved in O(2k n) where the constant
does not depend on k.
21. (a) Show that the problem of testing if a graph is k-colorable is NP-
complete.
(b) Show that if G is a graph and v is a node of degree ≤ d then G
is d + 1-colorable iff G − {v} is d + 1-colorable.
70
(c) Fix k. Show that determining if a graph is n − k-colorable can
be done in O(2k n) time.
71