Algorithms Sariel Harpeled download

https://ebookbell.com/product/algorithms-sariel-harpeled-50099372

Explore and download more ebooks at ebookbell.com

Here are some recommended products that we believe you will be
interested in. You can click the link to download.

Algorithms And Methods In Structural Bioinformatics Nurit Haspel

https://ebookbell.com/product/algorithms-and-methods-in-structural-
bioinformatics-nurit-haspel-45210820

Algorithms And Applications For Academic Search Recommendation And

Quantitative Association Rule Mining 1st Edition Emmanouil Amolochitis

https://ebookbell.com/product/algorithms-and-applications-for-
academic-search-recommendation-and-quantitative-association-rule-
mining-1st-edition-emmanouil-amolochitis-46497506

Algorithms For Decision Making Mykel J Kochenderfer Tim A Wheeler Kyle

H Wray

https://ebookbell.com/product/algorithms-for-decision-making-mykel-j-
kochenderfer-tim-a-wheeler-kyle-h-wray-46709392

Algorithms And Autonomy The Ethics Of Automated Decision Systems Alan

Rubel Clinton Castro Adam Pham

https://ebookbell.com/product/algorithms-and-autonomy-the-ethics-of-
automated-decision-systems-alan-rubel-clinton-castro-adam-
pham-46770860
Algorithms In Structural Molecular Biology Bruce R Donald

https://ebookbell.com/product/algorithms-in-structural-molecular-
biology-bruce-r-donald-47312440

Algorithms For The People Democracy In The Age Of Ai Josh Simons

https://ebookbell.com/product/algorithms-for-the-people-democracy-in-
the-age-of-ai-josh-simons-47525932

Algorithms For Big Data Dfg Priority Program 1736 Hannah Bast

https://ebookbell.com/product/algorithms-for-big-data-dfg-priority-
program-1736-hannah-bast-47546596

Algorithms 4 11th Printing Robert Sedgewick Kevin Wayne

https://ebookbell.com/product/algorithms-4-11th-printing-robert-
sedgewick-kevin-wayne-48124128

Algorithms Humans And Interactions How Do Algorithms Interact With

People Designing Meaningful Ai Experiences 1st Edition Don Donghee
Shin

https://ebookbell.com/product/algorithms-humans-and-interactions-how-
do-algorithms-interact-with-people-designing-meaningful-ai-
experiences-1st-edition-don-donghee-shin-48299938
Algorithms With Julia Optimization Machine Learning And Differential
Equations Using The Julia Language Clemens Heitzinger

https://ebookbell.com/product/algorithms-with-julia-optimization-
machine-learning-and-differential-equations-using-the-julia-language-
clemens-heitzinger-48714106
 CS 473: Algorithms¬

Sariel Har-Peled

March 8, 2019

¬ This work is licensed under the Creative Commons Attribution-Noncommercial 3.0 License. To view a copy of this
license, visit http://creativecommons.org/licenses/by-nc/3.0/ or send a letter to Creative Commons, 171 Second
Street, Suite 300, San Francisco, California, 94105, USA.
Preface

This manuscript is a collection of class notes for the (no longer required graduate) course “473G/573/473
Graduate Algorithms” taught in the University of Illinois, Urbana-Champaign, in 1. Spring 2006, 2. Fall 07,
3. Fall 09, 4. Fall 10, 5. Fall 13, 6. Fall 14, and 7. fall 18.
Class notes for algorithms class are as common as mushrooms after a rain. I have no plan of publishing
these class notes in any form except on the web. In particular, Jeff Erickson has class notes for 374/473 which
are better written and cover some of the topics in this manuscript (but naturally, I prefer my exposition over
his).
My reasons in writing the class notes are to (i) avoid the use of a (prohibitly expensive) book in this class,
(ii) cover some topics in a way that deviates from the standard exposition, and (iii) have a clear description of
the material covered. In particular, as far as I know, no book covers all the topics discussed here. Also, this
manuscript is available (on the web) in more convenient lecture notes form, where every lecture has its own
chapter.
Most of the topics covered are core topics that I believe every graduate student in computer science should
know about. This includes NP-Completeness, dynamic programming, approximation algorithms, randomized
algorithms and linear programming. Other topics on the other hand are more optional and are nice to know
about. This includes topics like network flow, minimum-cost network flow, and union-find. Nevertheless, I
strongly believe that knowing all these topics is useful for carrying out any worthwhile research in any subfield
of computer science.
Teaching such a class always involve choosing what not to cover. Some other topics that might be worthy
of presentation include advanced data-structures, computational geometry, etc – the list goes on. Since this
course is for general consumption, more theoretical topics were left out (e.g., expanders, derandomization, etc).
In particular, these class notes cover way more than can be covered in one semester. For my own sanity, I
try to cover some new material every semester I teach this class. Furthermore, some of the notes contains more
detail than I cover in class.
In any case, these class notes should be taken for what they are. A short (and sometime dense) tour of some
key topics in algorithms. The interested reader should seek other sources to pursue them further.
If you find any typos, mistakes, errors, or lies, please email me.

Acknowledgments
(No preface is complete without them.) I would like to thank the students in the class for their input, which
helped in discovering numerous typos and errors in the manuscript. Furthermore, the content was greatly
effected by numerous insightful discussions with Chandra Chekuri, Jeff Erickson, and Edgar Ramos.
In addition, I would like to thank Qizheng He for pointing out many typos in the notes (which were fixed
more later than sooner).

Copyright
This work is licensed under the Creative Commons Attribution-Noncommercial 3.0 License. To view a copy of
this license, visit http://creativecommons.org/licenses/by-nc/3.0/ or send a letter to Creative Commons,
171 Second Street, Suite 300, San Francisco, California, 94105, USA.

— Sariel Har-Peled
March 2019, Urbana, IL

1
Contents

Preface 1

Contents 2

I NP Completeness 14

1 NP Completeness I 14
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.2 Complexity classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.2.1 Reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.3 More NP-Complete problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.3.1 3SAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.4 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2 NP Completeness II 20
2.1 Max-Clique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2 Independent Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3 Vertex Cover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
2.4 Graph Coloring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3 NP Completeness III 25
3.1 Hamiltonian Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.2 Traveling Salesman Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3 Subset Sum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.4 3 dimensional Matching (3DM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
3.5 Partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.6 Some other problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

II Dynamic programming 30

4 Dynamic programming 30
4.1 Basic Idea - Partition Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.1.1 A Short sermon on memoization . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.2 Example – Fibonacci numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.2.1 Why, where, and when? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.2.2 Computing Fibonacci numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.3 Edit Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
4.3.1 Shortest path in a DAG and dynamic programming . . . . . . . . . . . . . . . . 36

5 Dynamic programming II - The Recursion Strikes Back 37

2
 5.1 Optimal search trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.2 Optimal Triangulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3 Matrix Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.4 Longest Ascending Subsequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.5 Pattern Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
5.6 Slightly faster TSP algorithm via dynamic programming . . . . . . . . . . . . . . . . 41

III Approximation algorithms 43

6 Approximation algorithms 43
6.1 Greedy algorithms and approximation algorithms . . . . . . . . . . . . . . . . . . . . 43
6.1.1 Alternative algorithm – two for the price of one . . . . . . . . . . . . . . . . . . 45
6.2 Fixed parameter tractability, approximation, and fast exponential time algorithms
(to say nothing of the dog) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.2.1 A silly brute force algorithm for vertex cover . . . . . . . . . . . . . . . . . . . . 45
6.2.2 A fixed parameter tractable algorithm . . . . . . . . . . . . . . . . . . . . . . . 45
6.2.2.1 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.3 Approximating maximum matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.4 Graph diameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.5 Traveling Salesman Person . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
6.5.1 TSP with the triangle inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.5.1.1 A 2-approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.5.1.2 A 3/2-approximation to TSP 4, -Min . . . . . . . . . . . . . . . . . . . . . . 50
6.6 Biographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

7 Approximation algorithms II 51
7.1 Max Exact 3SAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
7.2 Approximation Algorithms for Set Cover . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.2.1 Guarding an Art Gallery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.2.2 Set Cover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
7.2.3 Lower bound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.2.4 Just for fun – weighted set cover . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.2.4.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.3 Biographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

8 Approximation algorithms III 57

8.1 Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
8.1.1 The approximation algorithm for k-center clustering . . . . . . . . . . . . . . . . 58
8.2 Subset Sum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.2.1 On the complexity of ε-approximation algorithms . . . . . . . . . . . . . . . . . 60
8.2.2 Approximating subset-sum . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
8.2.2.1 Bounding the running time of ApproxSubsetSum . . . . . . . . . . . . . . 62
8.2.2.2 The result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
8.3 Approximate Bin Packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
8.4 Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

IV Randomized algorithms 65

9 Randomized Algorithms 65

3
 9.1 Some Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
9.2 Sorting Nuts and Bolts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
9.2.1 Running time analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
9.2.1.1 Alternative incorrect solution . . . . . . . . . . . . . . . . . . . . . . . . . 68
9.2.2 What are randomized algorithms? . . . . . . . . . . . . . . . . . . . . . . . . . . 68
9.3 Analyzing QuickSort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
9.4 QuickSelect – median selection in linear time . . . . . . . . . . . . . . . . . . . . . . . 69

10 Randomized Algorithms II 71
10.1 QuickSort and Treaps with High Probability . . . . . . . . . . . . . . . . . . . . . . . 71
10.1.1 Proving that an element participates in small number of rounds . . . . . . . . . 71
10.1.2 An alternative proof of the high probability of QuickSort . . . . . . . . . . . . . 73
10.2 Chernoff inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
10.2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
10.2.2 Chernoff inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
10.2.2.1 The Chernoff Bound — General Case . . . . . . . . . . . . . . . . . . . . . 75
10.3 Treaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
10.3.1 Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
10.3.2 Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
10.3.2.1 Insertion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
10.3.2.2 Deletion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
10.3.2.3 Split . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
10.3.2.4 Meld . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
10.3.3 Summery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
10.4 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

11 Hashing 78
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
11.2 Universal Hashing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
11.2.1 How to build a 2-universal family . . . . . . . . . . . . . . . . . . . . . . . . . . 81
11.2.1.1 On working modulo prime . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
11.2.1.2 Constructing a family of 2-universal hash functions . . . . . . . . . . . . . 82
11.2.1.3 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
11.2.1.4 Explanation via pictures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
11.3 Perfect hashing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
11.3.1 Some easy calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
11.3.2 Construction of perfect hashing . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
11.3.2.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
11.4 Bloom filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
11.5 Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

12 Min Cut 87
12.1 Branching processes – Galton-Watson Process . . . . . . . . . . . . . . . . . . . . . . 87
12.1.1 The problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
12.1.2 On coloring trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
12.2 Min Cut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
12.2.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
12.2.2 Some Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
12.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
12.3.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4
 12.3.1.1 The probability of success . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
12.3.1.2 Running time analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
12.4 A faster algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
12.5 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

V Network flow 95

13 Network Flow 95
13.1 Network Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
13.2 Some properties of flows and residual networks . . . . . . . . . . . . . . . . . . . . . . 96
13.3 The Ford-Fulkerson method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
13.4 On maximum flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

14 Network Flow II - The Vengeance 100

14.1 Accountability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
14.2 The Ford-Fulkerson Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
14.3 The Edmonds-Karp algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
14.4 Applications and extensions for Network Flow . . . . . . . . . . . . . . . . . . . . . . 103
14.4.1 Maximum Bipartite Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
14.4.2 Extension: Multiple Sources and Sinks . . . . . . . . . . . . . . . . . . . . . . . 104

15 Network Flow III - Applications 105

15.1 Edge disjoint paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
15.1.1 Edge-disjoint paths in a directed graphs . . . . . . . . . . . . . . . . . . . . . . 105
15.1.2 Edge-disjoint paths in undirected graphs . . . . . . . . . . . . . . . . . . . . . . 106
15.2 Circulations with demands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
15.2.1 Circulations with demands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
15.2.1.1 The algorithm for computing a circulation . . . . . . . . . . . . . . . . . . 107
15.3 Circulations with demands and lower bounds . . . . . . . . . . . . . . . . . . . . . . . 107
15.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
15.4.1 Survey design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

16 Network Flow IV - Applications II 109

16.1 Airline Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
16.1.1 Modeling the problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
16.1.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
16.2 Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
16.3 Project Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
16.3.1 The reduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
16.4 Baseball elimination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
16.4.1 Problem definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
16.4.2 Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
16.4.3 A compact proof of a team being eliminated . . . . . . . . . . . . . . . . . . . . 116

17 Network Flow V - Min-cost flow 117

17.1 Minimum Average Cost Cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
17.2 Potentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
17.3 Minimum cost flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
17.4 Strongly Polynomial Time Algorithm for Min-Cost Flow . . . . . . . . . . . . . . . . 122
17.5 Analysis of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

5
 17.5.1 Reduced cost induced by a circulation . . . . . . . . . . . . . . . . . . . . . . . . 123
17.5.2 Bounding the number of iterations . . . . . . . . . . . . . . . . . . . . . . . . . 123
17.6 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

18 Network Flow VI - Min-Cost Flow Applications 125

18.1 Efficient Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
18.2 Efficient Flow with Lower Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
18.3 Shortest Edge-Disjoint Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
18.4 Covering by Cycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
18.5 Minimum weight bipartite matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
18.6 The transportation problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

VI Linear Programming 129

19 Linear Programming in Low Dimensions 129

19.1 Some geometry first . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
19.2 Linear programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
19.2.1 A solution and how to verify it . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
19.3 Low-dimensional linear programming . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
19.3.1 An algorithm for a restricted case . . . . . . . . . . . . . . . . . . . . . . . . . . 132
19.3.1.1 Running time analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
19.3.2 The algorithm for the general case . . . . . . . . . . . . . . . . . . . . . . . . . . 134

20 Linear Programming 135

20.1 Introduction and Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
20.1.1 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
20.1.2 Network flow via linear programming . . . . . . . . . . . . . . . . . . . . . . . . 136
20.2 The Simplex Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
20.2.1 Linear program where all the variables are positive . . . . . . . . . . . . . . . . 137
20.2.2 Standard form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
20.2.3 Slack Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
20.2.4 The Simplex algorithm by example . . . . . . . . . . . . . . . . . . . . . . . . . 138
20.2.4.1 Starting somewhere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

21 Linear Programming II 141

21.1 The Simplex Algorithm in Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
21.2 The SimplexInner Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
21.2.1 Degeneracies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
21.2.2 Correctness of linear programming . . . . . . . . . . . . . . . . . . . . . . . . . 144
21.2.3 On the ellipsoid method and interior point methods . . . . . . . . . . . . . . . . 144
21.3 Duality and Linear Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
21.3.1 Duality by Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
21.3.2 The Dual Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
21.3.3 The Weak Duality Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
21.4 The strong duality theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
21.5 Some duality examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
21.5.1 Shortest path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
21.5.2 Set Cover and Packing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
21.5.3 Network flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
21.6 Solving LPs without ever getting into a loop - symbolic perturbations . . . . . . . . . 151

6
 21.6.1 The problem and the basic idea . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
21.6.2 Pivoting as a Gauss elimination step . . . . . . . . . . . . . . . . . . . . . . . . 152
21.6.2.1 Back to the perturbation scheme . . . . . . . . . . . . . . . . . . . . . . . . 152
21.6.2.2 The overall algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

22 Approximation Algorithms using Linear Programming 153

22.1 Weighted vertex cover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
22.2 Revisiting Set Cover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
22.3 Minimizing congestion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

VII Miscellaneous topics 158

23 Fast Fourier Transform 158

23.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
23.2 Computing a polynomial quickly on n values . . . . . . . . . . . . . . . . . . . . . . . 159
23.2.1 Generating Collapsible Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
23.3 Recovering the polynomial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
23.4 The Convolution Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
23.4.1 Complex numbers – a quick reminder . . . . . . . . . . . . . . . . . . . . . . . . 164

24 Sorting Networks 164

24.1 Model of Computation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
24.2 Sorting with a circuit – a naive solution . . . . . . . . . . . . . . . . . . . . . . . . . 165
24.2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
24.2.2 Sorting network based on insertion sort . . . . . . . . . . . . . . . . . . . . . . . 166
24.3 The Zero-One Principle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
24.4 A bitonic sorting network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
24.4.1 Merging sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
24.5 Sorting Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
24.6 Faster sorting networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

25 Union Find 171

25.1 Union-Find . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
25.1.1 Requirements from the data-structure . . . . . . . . . . . . . . . . . . . . . . . . 171
25.1.2 Amortized analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
25.1.3 The data-structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
25.2 Analyzing the Union-Find Data-Structure . . . . . . . . . . . . . . . . . . . . . . . . 173

26 Approximate Max Cut 176

26.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
26.1.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
26.2 Semi-definite programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
26.3 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

27 The Perceptron Algorithm 180

27.1 The perceptron algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
27.2 Learning A Circle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
27.3 A Little Bit On VC Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184
27.3.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

7
VIII Compression, entropy, and randomness 186

28 Huffman Coding 186

28.1 Huffman coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
28.1.1 The algorithm to build Hoffman’s code . . . . . . . . . . . . . . . . . . . . . . . 188
28.1.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
28.1.3 What do we get . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
28.1.4 A formula for the average size of a code word . . . . . . . . . . . . . . . . . . . 190
28.2 Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

29 Entropy, Randomness, and Information 191

29.1 Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
29.1.1 Extracting randomness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

30 Even more on Entropy, Randomness, and Information 194

30.1 Extracting randomness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
30.1.1 Enumerating binary strings with j ones . . . . . . . . . . . . . . . . . . . . . . . 195
30.1.2 Extracting randomness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
30.2 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

31 Shannon’s theorem 196

31.1 Coding: Shannon’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
31.1.0.1 Intuition behind Shanon’s theorem . . . . . . . . . . . . . . . . . . . . . . 197
31.1.0.2 What is wrong with the above? . . . . . . . . . . . . . . . . . . . . . . . . 198
31.2 Proof of Shannon’s theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
31.2.1 How to encode and decode efficiently . . . . . . . . . . . . . . . . . . . . . . . . 198
31.2.1.1 The scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
31.2.1.2 The proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
31.2.2 Lower bound on the message size . . . . . . . . . . . . . . . . . . . . . . . . . . 202
31.3 Bibliographical Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

IX Miscellaneous topics II 203

32 Matchings 203
32.1 Definitions and basic properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
32.1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
32.1.2 Matchings and alternating paths . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
32.2 Unweighted matching in bipartite graph . . . . . . . . . . . . . . . . . . . . . . . . . 205
32.2.1 The slow algorithm; algSlowMatch . . . . . . . . . . . . . . . . . . . . . . . . . 205
32.2.2 The Hopcroft-Karp algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
32.2.2.1 Some more structural observations . . . . . . . . . . . . . . . . . . . . . . . 206
32.2.2.2 Improved algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
32.2.2.3 Extracting many augmenting paths: algExtManyPaths . . . . . . . . . . . 207
32.2.2.4 The result . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
32.3 Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210

33 Matchings II 210
33.1 Maximum weight matchings in a bipartite graph . . . . . . . . . . . . . . . . . . . . . 211
33.1.1 On the structure of the problem . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
33.1.2 Maximum Weight Matchings in a bipartite Graph . . . . . . . . . . . . . . . . . 212

8
 33.1.2.1 Building the residual graph . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
33.1.2.2 The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
33.1.3 Faster Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
33.2 The Bellman-Ford algorithm - a quick reminder . . . . . . . . . . . . . . . . . . . . . 213
33.3 Maximum size matching in a non-bipartite graph . . . . . . . . . . . . . . . . . . . . 213
33.3.1 Finding an augmenting path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
33.3.2 The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
33.3.2.1 Running time analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
33.4 Maximum weight matching in a non-bipartite graph . . . . . . . . . . . . . . . . . . . 217

34 Lower Bounds 217

34.1 Sorting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
34.1.1 Decision trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
34.1.2 An easier direct argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
34.2 Uniqueness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
34.3 Other lower bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
34.3.1 Algebraic tree model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
34.3.2 3Sum-Hard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

35 Backwards analysis 221

35.1 How many times can the minimum change? . . . . . . . . . . . . . . . . . . . . . . . 222
35.2 Yet another analysis of QuickSort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
35.3 Closest pair: Backward analysis in action . . . . . . . . . . . . . . . . . . . . . . . . . 223
35.3.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
35.3.2 Back to the problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
35.3.3 Slow algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
35.3.4 Linear time algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
35.4 Computing a good ordering of the vertices of a graph . . . . . . . . . . . . . . . . . . 226
35.4.1 The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
35.4.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
35.5 Computing nets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
35.5.1 Basic definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
35.5.1.1 Metric spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
35.5.1.2 Nets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
35.5.2 Computing nets quickly for a point set in Rd . . . . . . . . . . . . . . . . . . . . 227
35.5.3 Computing an r-net in a sparse graph . . . . . . . . . . . . . . . . . . . . . . . . 228
35.5.3.1 The algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
35.5.3.2 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
35.6 Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230

36 Linear time algorithms 230

36.1 The lowest point above a set of lines . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
36.2 Bibliographical notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

37 Streaming 233
37.1 How to sample a stream . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
37.2 Sampling and median selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
37.2.1 A median selection with few comparisons . . . . . . . . . . . . . . . . . . . . . . 235
37.3 Big data and the streaming model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
37.4 Heavy hitters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
37.5 Chernoff inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

9
X Exercises 237

38 Exercises - Prerequisites 237

38.1 Graph Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
38.2 Recurrences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
38.3 Counting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
38.4 O notation and friends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
38.5 Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
38.6 Basic data-structures and algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
38.7 General proof thingies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
38.8 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

39 Exercises - NP Completeness 248

39.1 Equivalence of optimization and decision problems . . . . . . . . . . . . . . . . . . . 248
39.2 Showing problems are NP-Complete . . . . . . . . . . . . . . . . . . . . . . . . . . 249
39.3 Solving special subcases of NP-Complete problems in polynomial time . . . . . . . 250

40 Exercises - Network Flow 258

40.1 Network Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
40.2 Min Cost Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

41 Exercises - Miscellaneous 269

41.1 Data structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
41.2 Divide and Conqueror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
41.3 Fast Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
41.4 Union-Find . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
41.5 Lower bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
41.6 Number theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
41.7 Sorting networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
41.8 Max Cut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

42 Exercises - Approximation Algorithms 275

42.1 Greedy algorithms as approximation algorithms . . . . . . . . . . . . . . . . . . . . . 275
42.2 Approximation for hard problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

43 Randomized Algorithms 278

43.1 Randomized algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

44 Exercises - Linear Programming 281

44.1 Miscellaneous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
44.2 Tedious . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

45 Exercises - Computational Geometry 285

45.1 Misc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

46 Exercises - Entropy 287

XI Homeworks/midterm/final 289

47 Fall 2001 289

47.1 Homeworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

10
 47.1.1 Homework 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
47.1.2 Homework 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295
47.1.3 Homework 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
47.1.4 Homework 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307
47.1.5 Homework 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316
47.1.6 Homework 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
47.1.7 Homework 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
47.2 Midterm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
47.3 Final . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334

48 Spring 2002 334

49 Fall 2002 334

50 Spring 2003 334

50.1 Homeworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
50.1.1 Homework 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
50.1.2 Homework 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
50.1.3 Homework 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
50.1.4 Homework 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 358
50.1.5 Homework 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
50.1.6 Homework 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369
50.1.7 Homework 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
50.2 Midterm 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376
50.3 Midterm 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381
50.4 Final . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

51 Fall 2003 385

52 Spring 2005 385

53 Spring 2006 385

54 Fall 2007 385

55 Fall 2009 385

56 Spring 2011 385

57 Fall 2011 385

58 Fall 2012 385

59 Fall 2013 385

60 Spring 2013: CS 473: Fundamental algorithms 385

60.1 Homeworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
60.1.1 Homework 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
60.1.2 Homework 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387
60.1.3 Homework 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
60.1.4 Homework 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
60.1.5 Homework 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

11
 60.1.6 Homework 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
60.1.7 Homework 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394
60.1.8 Homework 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
60.1.9 Homework 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 398
60.1.10 Homework 9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
60.1.11 Homework 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
60.2 Midterm 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
60.3 Midterm 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406
60.4 Final . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410

61 Fall 2014: CS 573 – Graduate algorithms 413

61.1 Homeworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
61.1.1 Homework 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
61.1.2 Homework 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
61.1.3 Homework 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
61.1.4 Homework 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
61.1.5 Homework 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419
61.1.6 Homework 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422
61.2 Midterm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
61.3 Final . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428

62 Fall 2015: CS 473 – Theory II 430

62.1 Homeworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
62.1.1 Homework 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430
62.1.2 Homework 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
62.1.3 Homework 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435
62.1.4 Homework 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436
62.1.5 Homework 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
62.1.6 Homework 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
62.1.7 Homework 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
62.1.8 Homework 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442
62.1.9 Homework 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
62.1.10 Homework 9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443
62.1.11 Homework 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
62.1.12 Homework 11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447
62.2 Midterm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450
62.3 Final . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

63 Fall 2018: CS 473 Algorithms 451

63.1 Homeworks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
63.1.1 Homework 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451
63.1.2 Homework 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
63.1.3 Homework 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455
63.1.4 Homework 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457
63.1.5 Homework 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459
63.1.6 Homework 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 461
63.1.7 Homework 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462
63.1.8 Homework 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464
63.1.9 Homework 9 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
63.1.10 Homework 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466

12
 63.1.11 Homework 11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470
63.2 Midterm I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 474
63.3 Midterm II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476
63.4 Final . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479

Bibliography 491

Index 494

13
 Part I
NP Completeness

Chapter 1

NP Completeness I

"Then you must begin a reading program immediately so that you man understand the crises of our age," Ignatius said
solemnly. "Begin with the late Romans, including Boethius, of course. Then you should dip rather extensively into
early Medieval. You may skip the Renaissance and the Enlightenment. That is mostly dangerous propaganda. Now,
that I think about of it, you had better skip the Romantics and the Victorians, too. For the contemporary period,
you should study some selected comic books."
"You’re fantastic."
"I recommend Batman especially, for he tends to transcend the abysmal society in which he’s found himself. His
morality is rather rigid, also. I rather respect Batman."
– A confederacy of Dunces, John Kennedy Toole.

1.1. Introduction
The question governing this course, would be the development of efficient algorithms. Hopefully, what is an
algorithm is a well understood concept. But what is an efficient algorithm? A natural answer (but not the only
one!) is an algorithm that runs quickly.
What do we mean by quickly? Well, we would like our algorithm to:
(A) Scale with input size. That is, it should be able to handle large and hopefully huge inputs.
(B) Low level implementation details should not matter, since they correspond to small improvements in
performance. Since faster CPUs keep appearing it follows that such improvements would (usually) be
taken care of by hardware.
(C) What we will really care about are asymptotic running time. Explicitly, polynomial time.
In our discussion, we will consider the input size to be n, and we would like to bound the overall running
time by a function of n which is asymptotically as small as possible. An algorithm with better asymptotic
running time would be considered to be better.

Example 1.1.1. It is illuminating to consider a concrete example. So assume we have an algorithm for a problem
that needs to perform c2n operations to handle an input of size n, where c is a small constant (say 10). Let
assume that we have a CPU that can do 109 operations a second. (A somewhat conservative assumption, as

14
 Input size n2 ops n3 ops n4 ops 2n ops n! ops
5 0 secs 0 secs 0 secs 0 secs 0 secs
20 0 secs 0 secs 0 secs 0 secs 16 mins
30 0 secs 0 secs 0 secs 0 secs 3 · 109 years
50 0 secs 0 secs 0 secs 0 secs never
60 0 secs 0 secs 0 secs 7 mins never
70 0 secs 0 secs 0 secs 5 days never
80 0 secs 0 secs 0 secs 15.3 years never
90 0 secs 0 secs 0 secs 15,701 years never
100 0 secs 0 secs 0 secs 107 years never
8000 0 secs 0 secs 1 secs never never
16000 0 secs 0 secs 26 secs never never
32000 0 secs 0 secs 6 mins never never
64000 0 secs 0 secs 111 mins never never
200,000 0 secs 3 secs 7 days never never
2,000,000 0 secs 53 mins 202.943 years never never
108 4 secs 12.6839 years 109 years never never
109 6 mins 12683.9 years 1013 years never never

Figure 1.1: Running time as function of input size. Algorithms with exponential running times can handle
only relatively small inputs. We assume here that the computer can do 2.5 · 1015 operations per second, and
the functions are the exact number of operations performed. Remember – never is a long time to wait for a
computation to be completed.

currently [Jan 2006]¬ , the blue-gene supercomputer can do about 3 · 1014 floating-point operations a second.
Since this super computer has about 131, 072 CPUs, it is not something you would have on your desktop any
time soon.) Since 210 ≈ 103 , you have that our (cheap) computer can solve in (roughly) 10 seconds a problem
of size n = 27.
But what if we increase the problem size to n = 54? This would take our computer about 3 million years to
solve. (It is better to just wait for faster computers to show up, and then try to solve the problem. Although
there are good reasons to believe that the exponential growth in computer performance we saw in the last 40
years is about to end. Thus, unless a substantial breakthrough in computing happens, it might be that solving
problems of size, say, n = 100 for this problem would forever be outside our reach.)
The situation dramatically change if we consider an algorithm with running time 10n2 . Then, in one second
our computer can handle input of size n = 104 . Problem of size n = 108 can be solved in 10n2 /109 = 1017−9 = 108
which is about 3 years of computing (but blue-gene might be able to solve it in less than 20 minutes!).
Thus, algorithms that have asymptotically a polynomial running time (i.e., the algorithms running time is
bounded by O(nc ) where c is a constant) are able to solve large instances of the input and can solve the problem
even if the problem size increases dramatically.

Can we solve all problems in polynomial time? The answer to this question is unfortunately no. There
are several synthetic examples of this, but it is believed that a large class of important problems can not be
solved in polynomial time.

Circuit Satisfiability
Instance: A circuit C with m inputs
Question: Is there an input for C such that C returns true for it.

¬ But the recently announced Super Computer that would be completed in 2012 in Urbana, is naturally way faster. It supposedly
would do 1015 operations a second (i.e., petaflop). Blue-gene probably can not sustain its theoretical speed stated above, which is
only slightly slower.

15
 As a concrete example, consider the circuit depicted on the right. x1
Currently, all solutions known to Circuit Satisfiability require checking all x2
possibilities, requiring (roughly) 2m time. Which is exponential time and too x3
slow to be useful in solving large instances of the problem. x4
This leads us to the most important open question in theoretical computer x5
science: x x∧y x x∨y x x
y y

Question 1.1.2. Can one solve Circuit Satisfiability in polynomial time? And Or Not

The common belief is that Circuit Satisfiability can NOT be solved in polynomial time. Circuit Satisfiability
has two interesting properties.
(A) Given a supposed positive solution, with a detailed assignment (i.e., proof): x1 ← 0, x2 ← 1, ..., xm ← 1
one can verify in polynomial time if this assignment really satisfies C. This is done by computing what
every gate in the circuit what its output is for this input. Thus, computing the output of C for its input.
This requires evaluating the gates of C in the right order, and there are some technicalities involved, which
we are ignoring. (But you should verify that you know how to write a program that does that efficiently.)
Intuitively, this is the difference in hardness between coming up with a proof (hard), and checking that a
proof is correct (easy).
(B) It is a decision problem. For a specific input an algorithm that solves this problem has to output either
TRUE or FALSE.

1.2. Complexity classes

Definition 1.2.1 (P: Polynomial time). Let P denote is the class of all decision problems that can be solved in
polynomial time in the size of the input.

Definition 1.2.2 (NP: Nondeterministic Polynomial time). Let NP be the class of all decision problems that can
be verified in polynomial time. Namely, for an input of size n, if the solution to the given instance is true, one
(i.e., an oracle) can provide you with a proof (of polynomial length!) that the answer is indeed TRUE for this
instance. Furthermore, you can verify this proof in polynomial time in the length of the proof.

Clearly, if a decision problem can be solved in polynomial time,

then it can be verified in polynomial time. Thus, P ⊆ NP.

Remark. The notation NP stands for Non-deterministic Polynomial.

The name come from a formal definition of this class using Turing
machines where the machines first guesses (i.e., the non-deterministic
stage) the proof that the instance is TRUE, and then the algorithm
verifies the proof.
Figure 1.2: The relation between
Definition 1.2.3 (co-NP). The class co-NP is the opposite of NP – if
the different complexity classes
the answer is FALSE, then there exists a short proof for this negative
P, NP, and co-NP.
answer, and this proof can be verified in polynomial time.

See Figure 1.2 for the currently believed relationship between these classes (of course, as mentioned above,
P ⊆ NP and P ⊆ co-NP is easy to verify). Note, that it is quite possible that P = NP = co-NP, although
this would be extremely surprising.

Definition 1.2.4. A problem Π is NP-Hard, if being able to solve Π in polynomial time implies that P = NP.

Question 1.2.5. Are there any problems which are NP-Hard?

16
 Intuitively, being NP-Hard implies that a problem is ridiculously hard. Conceptually, it would imply that
proving and verifying are equally hard - which nobody that did CS 473 believes is true.
In particular, a problem which is NP-Hard is at least as hard as ALL the problems in NP, as such it is
safe to assume, based on overwhelming evidence that it can not be solved in polynomial time.
Theorem 1.2.6 (Cook’s Theorem). Circuit Satisfiability is NP-Hard.

Definition 1.2.7. A problem Π is NP-Complete (NPC in short) if it is both NP-Hard and in NP.

Clearly, Circuit Satisfiability is NP-Complete, since we can verify a positive solution in polynomial time in
the size of the circuit,

By now, thousands of problems have been shown to be NP- NP-Hard

Complete. It is extremely unlikely that any of them can be solved
in polynomial time.
co-NP
Definition 1.2.8. In the formula satisfiability problem, (a.k.a. SAT)
we are given a formula, for example: NP
    P
a ∨ b ∨ c ∨ d ⇐⇒ (b ∧ c) ∨ (a ⇒ d) ∨ (c , a ∧ b)
NP-Complete
and the question is whether we can find an assignment to the vari-
ables a, b, c, . . . such that the formula evaluates to TRUE. Figure 1.3: The relation between the com-
plexity classes.
It seems that SAT and Circuit Satisfiability are “similar” and as such both should be NP-Hard.
Remark 1.2.9. Cook’s theorem implies something somewhat stronger than implied by the above statement.
Specifically, for any problem in NP, there is a polynomial time reduction to Circuit Satisfiability. Thus, the
reader can think about NPC problems has being equivalent under polynomial time reductions.

1.2.1. Reductions
Let A and B be two decision problems.
Given an input I for problem A, a reduction is a transformation of the input I into a new input I 0, such that

A(I) is TRUE ⇔ B(I 0) is TRUE.

Thus, one can solve A by first transforming and input I into an input I 0 of B, and solving B(I 0).
This idea of using reductions is omnipresent, and used almost in any program you write.
Let T : I → I 0 be the input transformation that maps A into B. How fast is T? Well, for our nefarious
purposes we need polynomial reductions; that is, reductions that take polynomial time.
For example, given an instance of Circuit Satisfiability, we would like to generate an equivalent formula. We
will explicitly write down what the circuit computes in a formula form. To see how to do this, consider the
following example.
x1 y1 y4
y5 y1 = x1 ∧ x4 y2 = x4 y3 = y2 ∧ x3
x2
y4 = x2 ∨ y1 y5 = x2 y6 = x5
x3 y7 y8 y7 = y3 ∨ y5 y8 = y4 ∧ y7 ∧ y6 y8
x4 y3
y2
x5 y6

We introduced a variable for each wire in the circuit, and we wrote down explicitly what each gate computes.
Namely, we wrote a formula for each gate, which holds only if the gate computes correctly the output for its
given input.
17
 The circuit is satisfiable if and only if there is an Input: boolean circuit C
assignment such that all the above formulas hold. Al- ⇓ O(size o f C)
ternatively, the circuit is satisfiable if and only if the transform C into boolean formula F
following (single) formula is satisfiable ⇓
(y1 = x1 ∧ x4 ) ∧ (y2 = x4 ) ∧ (y3 = y2 ∧ x3 ) Find SAT assign’ for F using SAT solver
∧(y4 = x2 ∨ y1 ) ∧ (y5 = x2 )
⇓
∧(y6 = x5 ) ∧ (y7 = y3 ∨ y5 )
Return TRUE if F is sat’, otherwise FALSE.
∧(y8 = y4 ∧ y7 ∧ y6 ) ∧ y8 .
Figure 1.4: Algorithm for solving CSAT using
It is easy to verify that this transformation can be done an algorithm that solves the SAT problem
in polynomial time.
The resulting reduction is depicted in Figure 1.4.
Namely, given a solver for SAT that runs in TSAT (n),
we can solve the CSAT problem in time

TCS AT (n) ≤ O(n) + TS AT (O(n)),

where n is the size of the input circuit. Namely, if we have polynomial time algorithm that solves SAT then we
can solve CSAT in polynomial time.
Another way of looking at it, is that we believe that solving CSAT requires exponential time; namely,
TCSAT (n) ≥ 2n . Which implies by the above reduction that

2n ≤ TCS AT (n) ≤ O(n) + TS AT (O(n)).

Namely, TSAT (n) ≥ 2n/c −O(n), where c is some positive constant. Namely, if we believe that we need exponential
time to solve CSAT then we need exponential time to solve SAT.
This implies that if SAT ∈ P then CSAT ∈ P.
We just proved that SAT is as hard as CSAT. Clearly, SAT ∈ NP which implies the following theorem.
Theorem 1.2.10. SAT (formula satisfiability) is NP-Complete.

1.3. More NP-Complete problems

1.3.1. 3SAT
A boolean formula is in conjunctive normal form (CNF) if it is a conjunction (AND) of several clauses, where a
clause is the disjunction (or) of several literals, and a literal is either a variable or a negation of a variable. For
example, the following is a CNF formula:
clause
z }| {
(a ∨ b ∨ c) ∧(a ∨ e) ∧ (c ∨ e).

Definition 1.3.1. 3CNF formula is a CNF formula with exactly three literals in each clause.

The problem 3SAT is formula satisfiability when the formula is restricted to be a 3CNF formula.
Theorem 1.3.2. 3SAT is NP-Complete.

Proof: First, it is easy to verify that 3SAT is in NP.

Next, we will show that 3SAT is NP-Complete by a reduction from CSAT (i.e., Circuit Satisfiability). As
such, our input is a circuit C of size n. We will transform it into a 3CNF in several steps:

18
 (A) Make sure every AND/OR gate has only two inputs. If (say) an AND gate have more inputs, we replace
it by cascaded tree of AND gates, each one of degree two.
(B) Write down the circuit as a formula by traversing the circuit, as was done for SAT. Let F be the resulting
formula.
A clause corresponding to a gate in F will be of the following forms: (i) a = b ∧ c if it corresponds to an
AND gate, (ii) a = b ∨ c if it corresponds to an OR gate, and (iii) a = b if it corresponds to a NOT gate.
Notice, that except for the single clause corresponding to the output of the circuit, all clauses are of this
form. The clause that corresponds to the output is a single variable.
(C) Change every gate clause into several CNF clauses.
(i) For example, an AND gate clause of the form a = b ∧ c will be translated into
 
a ∨ b ∨ c ∧ (a ∨ b) ∧ (a ∨ c). (1.1)

Note that Eq. (1.1) is true if and only if a = b ∧ c is true. Namely, we can replace the clause a = b ∧ c
in F by Eq. (1.1).
(ii) Similarly, an OR gate clause the form a = b ∨ c in F will be transformed into

(a ∨ b ∨ c) ∧ (a ∨ b) ∧ (a ∨ c).

(iii) Finally, a clause a = b, corresponding to a NOT gate, will be transformed into

(a ∨ b) ∧ (a ∨ b).

(D) Make sure every clause is exactly three literals. Thus, a single variable clause a would be replaced by

(a ∨ x ∨ y) ∧ (a ∨ x ∨ y) ∧ (a ∨ x ∨ y) ∧ (a ∨ x ∨ y),

by introducing two new dummy variables x and y. And a two variable clause a ∨ b would be replaced by

(a ∨ b ∨ y) ∧ (a ∨ b ∨ y),

by introducing the dummy variable y.

This completes the reduction, and results in a new 3CNF formula G which is satisfiable if and only if the original
circuit C is satisfiable. The reduction is depicted in Figure 1.5. Namely, we generated an equivalent 3CNF to
the original circuit. We conclude that if T3SAT (n) is the time required to solve 3SAT then

TCS AT (n) ≤ O(n) + T3S AT (O(n)),

which implies that if we have a polynomial time algorithm for 3SAT, we would solve CSAT is polynomial time.
Namely, 3SAT is NP-Complete.

Input: boolean circuit

⇓ O(n)
3CNF formula
⇓
Decide if given formula is satsifiable using 3SAT solver
⇓
Return TRUE or FALSE
Figure 1.5: Reduction from CSAT to 3SAT

19
1.4. Bibliographical Notes
Cook’s theorem was proved by Stephen Cook (http://en.wikipedia.org/wiki/Stephen_Cook). It was proved
independently by Leonid Levin (http://en.wikipedia.org/wiki/Leonid_Levin) more or less in the same
time. Thus, this theorem should be referred to as the Cook-Levin theorem.
The standard text on this topic is [GJ90]. Another useful book is [ACG+ 99], which is a more recent and
more updated, and contain more advanced stuff.

Chapter 2

NP Completeness II

2.1. Max-Clique

We remind the reader, that a clique is a complete graph, where every

pair of vertices are connected by an edge. The MaxClique problem asks
what is the largest clique appearing as a subgraph of G. See Figure 2.1.

MaxClique
Instance: A graph G
Question: What is the largest number of nodes in G forming
a complete subgraph? Figure 2.1: A clique of size 4
inside a graph with 8 vertices.

Note that MaxClique is an optimization problem, since the output of

the algorithm is a number and not just true/false.
The first natural question, is how to solve MaxClique. A naive algorithm
would work by enumerating all subsets S ⊆ V(G), checking for each such subset S if it induces a clique in G (i.e.,
all pairs of vertices in S are connected by an edge of G). If so, we know that GS is a clique, where GS denotes
the induced subgraph on S defined by G; that is, the graph formed by removing all the vertices are not in S
from G (in particular, only edges that have both endpoints in S appear in GS ). Finally, our algorithm would
return the largest S encountered, such that GS is a clique. The running time of this algorithm is O 2n n2 as

can be easily verified.

Suggestion 2.1.1. When solving any algorithmic problem, always try first to find a simple (or even naive) T IP
solution. You can try optimizing it later, but even a naive solution might give you useful insight into a problem
structure and behavior.
We will prove that MaxClique is NP-Hard. Before dwelling into that, the simple algorithm we devised for
MaxClique shade some light on why intuitively it should be NP-Hard: It does not seem like there is any way of
avoiding the brute force enumeration of all possible subsets of the vertices of G. Thus, a problem is NP-Hard
or NP-Complete, intuitively, if the only way we know how to solve the problem is to use naive brute force
enumeration of all relevant possibilities.

20
How to prove that a problem X is NP-Hard? Proving that a given problem X is NP-Hard is usually
done in two steps. First, we pick a known NP-Complete problem A. Next, we show how to solve any instance
of A in polynomial time, assuming that we are given a polynomial time algorithm that solves X.
Proving that a problem X is NP-Complete requires the additional burden of showing that is in NP. Note,
that only decision problems can be NP-Complete, but optimization problems can be NP-Hard; namely, the
set of NP-Hard problems is much bigger than the set of NP-Complete problems.
Theorem 2.1.2. MaxClique is NP-Hard.

Proof: We show a reduction from 3SAT. So, consider an input to 3SAT, which is a formula F defined over n
variables (and with m clauses).
We build a graph from the formula F by scanning it, as follows: a b c

(i) For every literal in the formula we generate a ver- a b

tex, and label the vertex with the literal it corre-
sponds to. b c

Note, that every clause corresponds to the three

d d
such vertices.
(ii) We connect two vertices in the graph, if they are:
a c d
(i) in different clauses, and (ii) they are not a nega-
tion of each other. Figure 2.2: The generated
Let G denote the resulting graph. See Figure 2.2 for a concrete example. graph for the formula (a∨b∨c)∧
Note, that this reduction can be easily be done in quadratic time in the size (b∨c∨ d)∧(a∨c∨ d)∧(a∨ b∨ d).
of the given formula.
We claim that F is satisfiable iff there exists a clique of size m in G.
=⇒ Let x1, . . . , xn be the variables appearing in F, and let v1, . . . , vn ∈ {0, 1} be the satisfying assignment for
F. Namely, the formula F holds if we set xi = vi , for i = 1, . . . , n.
For every clause C in F there must be at least one literal that evaluates to TRUE. Pick a vertex that
corresponds to such TRUE value from each clause. Let W be the resulting set of vertices. Clearly, W forms
a clique in G. The set W is of size m, since there are m clauses and each one contribute one vertex to the
clique.
⇐= Let U be the set of m vertices which form a clique in G.
We need to translate the clique GU into a satisfying assignment of F.
(i) set xi ← TRUE if there is a vertex in U labeled with xi .
(ii) set xi ← FALSE if there is a vertex in U labeled with xi .
This is a valid assignment as can be easily verified. Indeed, assume for the sake of contradiction, that
there is a variable xi such that there are two vertices u, v in U labeled with xi and xi ; namely, we are
trying to assign to contradictory values of xi . But then, u and v, by construction will not be connected in
G, and as such GS is not a clique. A contradiction.
Furthermore, this is a satisfying assignment as there is at least one vertex of U in each clause. Implying,
that there is a literal evaluating to TRUE in each clause. Namely, F evaluates to TRUE.
Thus, given a polytime (i.e., polynomial time) algorithm for MaxClique,
we can solve 3SAT in polytime. We conclude that MaxClique in NP-Hard.

MaxClique is an optimization problem, but it can be easily restated as a decision problem.

Clique
Instance: A graph G, integer k
Question: Is there a clique in G of size k?

21
 (a) (b) (c)

Figure 2.3: (a) A clique in a graph G, (b) the complement graph is formed by all the edges not appearing in G,
and (c) the complement graph and the independent set corresponding to the clique in G.

Theorem 2.1.3. Clique is NP-Complete.

Proof: It is NP-Hard by the reduction of Theorem 2.1.2. Thus, we only need to show that it is in NP. This
is quite easy. Indeed, given a graph G having n vertices, a parameter k, and a set W of k vertices, verifying
that every pair of vertices in W form an edge in G takes O(u + k 2 ), where u is the size of the input (i.e., number
of edges + number of vertices). Namely, verifying a positive answer to an instance of Clique can be done in
polynomial time.
Thus, Clique is NP-Complete.

2.2. Independent Set

Definition 2.2.1. A set S of nodes in a graph G = (V, E) is an independent set, if no pair of vertices in S are
connected by an edge.

Independent Set
Instance: A graph G, integer k
Question: Is there an independent set in G of size k?

Theorem 2.2.2. Independent Set is NP-Complete.

Proof: This readily follows by a reduction from Clique. Given G and k, compute the complement graph G where
we connected two vertices u, v in G iff they are independent (i.e., not connected) in G. See Figure 2.3. Clearly,
a clique in G corresponds to an independent set in G, and vice versa. Thus, Independent Set is NP-Hard, and
since it is in NP, it is NPC.

2.3. Vertex Cover

Definition 2.3.1. For a graph G, a set of vertices S ⊆ V(G) is a vertex cover if it touches every edge of G.
Namely, for every edge uv ∈ E(G) at least one of the endpoints is in S.

Vertex Cover
Instance: A graph G, integer k
Question: Is there a vertex cover in G of size k?

22
Lemma 2.3.2. A set S is a vertex cover in G iff V \ S is an independent set in G.

Proof: If S is a vertex cover, then consider two vertices u, v ∈ V \ S. If uv ∈ E(G) then the edge uv is not covered
by S. A contradiction. Thus V \ S is an independent set in G.
Similarly, if V \ S is an independent set in G, then for any edge uv ∈ E(G) it must be that either u or v are
not in V \ S. Namely, S covers all the edges of G.

Theorem 2.3.3. Vertex Cover is NP-Complete.

Proof: Vertex Cover is in NP as can be easily verified. To show that it NP-Hard we will do a reduction from
Independent Set. So, we are given an instance of Independent Set which is a graph G and parameter k, and we
want to know whether there is an independent set in G of size k. By Lemma 2.3.2, G has an independent set
of k iff it has a vertex cover of size n − k. Thus, feeding G and n − k into (the supposedly given) black box that
can solves vertex cover in polynomial time, we can decide if G has an independent set of size k in polynomial
time. Thus Vertex Cover is NP-Complete.

2.4. Graph Coloring

Definition 2.4.1. A coloring, by c colors, of a graph G = (V, E) is a mapping C : V(G) → {1, 2, . . . , c} such that
every vertex is assigned a color (i.e., an integer), such that no two vertices that share an edge are assigned the
same color.

Usually, we would like to color a graph with a minimum number of colors. Deciding if a graph can be colored
with two colors is equivalent to deciding if a graph is bipartite and can be done in linear time using DFS or
BFS¬ .
Coloring is useful for resource allocation (used in compilers for example) and scheduling type problems.
Surprisingly, moving from two colors to three colors make the problem much harder.

3Colorable
Instance: A graph G.
Question: Is there a coloring of G using three colors?

Theorem 2.4.2. 3Colorable is NP-Complete.

Proof: Clearly, 3Colorable is in NP.

We prove that it is NP-Complete by a reduction from 3SAT. Let F be the given 3SAT instance. The
basic idea of the proof is to use gadgets to transform the formula into a graph. Intuitively, a gadget is a small
component that corresponds to some feature of the input.
The first gadget will be the color generating gadget, which is formed by three special X
vertices connected to each other, where the vertices are denoted by X, F and T, respectively. We
will consider the color used to color T to correspond to the TRUE value, and the color of the F
to correspond to the FALSE value. T F
¬ If you do not know the algorithm for this, please read about it to fill this monstrous gap in your knowledge.

23
 a u
a u a u
w r
w r w r
b v
T b v b v
T T
c s
c s c s
(1) a ⇐ F, b ⇐ F, c ⇐ F (2) a ⇐ F, b ⇐ F, c ⇐ T (3) a ⇐ F, b ⇐ T, c ⇐ F
a u a u a u
w r w r w r
b v b v b v
T T T
c s c s c s
(4) a ⇐ F, b ⇐ T, c ⇐ T (5) a ⇐ T, b ⇐ F, c ⇐ F (6) a ⇐ T, b ⇐ F, c ⇐ T
a u a u
w r w r
b v b v
T T
c s c s
(7) a ⇐ F, b ⇐ T, c ⇐ T (8) a ⇐ T, b ⇐ F, c ⇐ F

Figure 2.4: The clause a ∨ b ∨ c and all the three possible colorings to its literals. If all three literals are colored
by the color of the special node F, then there is no valid coloring of this component, see case (1).

For every variable y appearing in F , we will generate a variable gadget, which is (again)
X
a triangle including two new vertices, denoted by y and y, and the third vertex is the auxiliary
vertex X from the color generating gadget. Note, that in a valid 3-coloring of the resulting
graph either y would be colored by T (i.e., it would be assigned the same color as the color as y y
the vertex T) and y would be colored by F, or the other way around. Thus, a valid coloring
could be interpreted as assigning TRUE or FALSE value to each variable y, by just inspecting the color used for
coloring the vertex y.
Finally, for every clause we introduce a clause gadget. See the figure on the a u
right – for how the gadget looks like for the clause a ∨ b ∨ c. Note, that the vertices w r
marked by a, b and c are the corresponding vertices from the corresponding variable b v
T
gadget. We introduce five new variables for every such gadget. The claim is that this
s
gadget can be colored by three colors if and only if the clause is satisfied. This can c
be done by brute force checking all 8 possibilities, and we demonstrate it only for two cases. The reader should
verify that it works also for the other cases.
Indeed, if all three vertices (i.e., three variables in a clause) on the left side of a a u
variable clause are assigned the F color (in a valid coloring of the resulting graph), w r
then the vertices u and v must be either be assigned X and T or T and X, respectively, b v
T
in any valid 3-coloring of this gadget (see figure on the left). As such, the vertex w
c s
must be assigned the color F. But then, the vertex r must be assigned the X color.
But then, the vertex s has three neighbors with all three different colors, and there
is no valid coloring for s.

24
 Figure 2.5: The formula (a ∨ b ∨ c) ∧ (b ∨ c ∨ d) ∧ (a ∨ c ∨ d) ∧ (a ∨ b ∨ d) reduces to the depicted graph.

As another example, consider the case when one of the variables on the left a u
is assigned the T color. Then the clause gadget can be colored in a valid way, w r
as demonstrated on the figure on the right. b v
T
This concludes the reduction. Clearly, the generated graph can be computed
in polynomial time. By the above argumentation, if there is a valid 3-coloring c s
of the resulting graph G, then there is a satisfying assignment for F . Similarly,
if there is a satisfying assignment for F then the G be colored in a valid way using three colors. For how the
resulting graph looks like, see Figure 2.5.
This implies that 3Colorable is NP-Complete.

Here is an interesting related problem. You are given a graph G as input, and you know that it is 3-colorable.
In polynomial time, what is the minimum number of colors you can use to
color this graph legally? Currently,
the best polynomial time algorithm for coloring such graphs, uses O n3/14 colors.

Chapter 3

NP Completeness III

3.1. Hamiltonian Cycle

Definition 3.1.1. A Hamiltonian cycle is a cycle in the graph that visits every vertex exactly once.

Definition 3.1.2. An Eulerian cycle is a cycle in a graph that uses every edge exactly once.

Finding Eulerian cycle can be done in linear time. Surprisingly, finding a Hamiltonian cycle is much harder.

25
 Hamiltonian Cycle
Instance: A graph G.
Question: Is there a Hamiltonian cycle in G?

Theorem 3.1.3. Hamiltonian Cycle is NP-Complete.

Proof: Hamiltonian Cycle is clearly in NP.

We will show a reduction from Vertex Cover. Given a
graph G and integer k we redraw G in the following way: a
b
We turn every vertex into a horizontal line segment, all of
c
the same length. Next, we turn an edge in the original graph
d
G into a gate, which is a vertical segment connecting the two
e
relevant vertices.
Note, that there is a Vertex Cover in G of size k if and only if there are k horizontal lines that stabs all the
gates in the resulting graph H (a line stabs a gate if one of the endpoints of the gate lies on the line).
a
Thus, computing a vertex cover in G is equivalent to computing k disjoints
b
paths through the graph G that visits all the gates. However, there is a technical
c
problem: a path might change venues or even go back. See figure on the right.
d
(u,v,1) (u,v,2) (u,v,3) (u,v,4) (u,v,5) (u,v,6)

u
To overcome this problem, we will replace each e gate with a component that
guarantees, that if you visit all its vertices, you have to go forward and can
NOT go back (or change “lanes”). The new component is depicted on the left.
There only three possible ways to visit all the vertices of the components by
v
(v,u,1) (v,u,2) (v,u,3) (v,u,4) (v,u,5) (v,u,6) paths that do not start/end inside the component, and they are the following:

The proof that this is the only three possibilities is by brute force. De-
picted on the right is one impossible path, that tries to backtrack by entering
on the top and leaving on the bottom. Observe, that there are vertices left
unvisited. Which means that not all the vertices in the graph are going to be
visited, because we add the constraint, that the paths start/end outside the
gate-component (this condition would be enforced naturally by our final construction).
The resulting graph H1 for the example graph we started with is depicted
on the right. There exists a Vertex Cover in the original graph iff there exists k
paths that start on the left side and end on the right side, in this weird graph.
And these k paths visits all the vertices.
The final stroke is to add connection from the left side to the right side, such that once a a

you arrive to the right side, you can go back to the left side. However, we want connection b
b
that allow you to travel exactly k times. This is done by adding to the above graph a
“routing box” component H2 depicted on the right, with k new middle vertices. The ith c c

vertex on the left of the routing component is the left most vertex of the ith horizontal line d d
in the graph, and the ith vertex on the right of the component is the right most vertex of
the ith horizontal line in the graph. e e

It is now easy (but tedious) to verify that the resulting graph H1 ∪ H2 has a Hamiltonian path iff H1 has k
paths going from left to right, which happens, iff the original graph has a Vertex Cover of size k. It is easy to
verify that this reduction can be done in polynomial time.

26
3.2. Traveling Salesman Problem
A traveling salesman tour, is a Hamiltonian cycle in a graph, which its price is the price of all the edges it uses.
TSP
Instance: G = (V, E) a complete graph - n vertices, c(e): Integer cost function over the edges of G,
and k an integer.
Question: Is there a traveling-salesman tour with cost at most k?

Theorem 3.2.1. TSP is NP-Complete.

Proof: Reduction from Hamiltonian cycle. Consider a graph G = (V, E), and let H be the complete graph
defined over V. Let (
1 e ∈ E(G)
c(e) =
2 e < E(G).
Clearly, the cheapest TSP in H with cost function equal to n iff G is Hamiltonian. Indeed, if G is not
Hamiltonian, then the TSP must use one edge that does not belong to G, and then, its price would be at least
n + 1.

3.3. Subset Sum

We would like to prove that the following problem, Subset Sum is NPC.
Subset Sum
Instance: S - set of positive integers,t: - an integer number (Target)
Question: Is there a subset X ⊆ S such that x ∈X x = t?
Í

How does one prove that a problem is NP-Complete? First, one has to choose an appropriate NPC to
reduce from. In this case, we will use 3SAT. Namely, we are given a 3CNF formula with n variables and m
clauses. The second stage, is to “play” with the problem and understand what kind of constraints can be
encoded in an instance of a given problem and understand the general structure of the problem.
The first observation is that we can use very long numbers as input to Subset Sum. The numbers can be of
polynomial length in the size of the input 3SAT formula F.
The second observation is that in fact, instead of thinking about Subset Sum as adding numbers, we can
think about it as a problem where we are given vectors with k components each, and the sum of the vectors
(coordinate by coordinate, must match. For example, the input might be the vectors (1, 2), (3, 4), (5, 6) and the
target vector might be (6, 8). Clearly, (1, 2) + (5, 6) give the required target vector. Lets refer to this new problem
as Vec Subset Sum.
Vec Subset Sum
Instance: S - set of n vectors of dimension k, each vector has non-negative numbers for its coordinates,
→
−
and a target vector t .
Question: Is there a subset X ⊆ S such that →
Í →− → −
−x ∈X x = t ?

Given an instance of Vec Subset Sum, we can covert it into an instance of Subset Sum as follows: We
compute the largest number in the given instance, multiply it by n2 · k · 100, and compute how many digits are
required to write this number down. Let U be this number of digits. Now, we take every vector in the given
instance and we write it down using U digits, padding it with zeroes as necessary. Clearly, each vector is now
converted into a huge integer number. The property is now that a sub of numbers in a specific column of the
given instance can not spill into digits allocated for a different column since there are enough zeroes separating
the digits corresponding to two different columns.

27
 Next, let us observe that we can force the solution (if it exists) for Vec Target ?? ?? 01 ???
Subset Sum to include exactly one vector out of two vectors. To this end, we a1 ?? ?? 01 ??
will introduce a new coordinate (i.e., a new column in the table on the right) a2 ?? ?? 01 ??
for all the vectors. The two vectors a1 and a2 will have 1 in this coordinate,
and all other vectors will have zero in this coordinate. Finally, we set this coordinate in the target vector to
be 1. Clearly, a solution is a subset of vectors that in this coordinate add up to 1. Namely, we have to choose
either a1 or a2 into our solution.
In particular, for each variable x appearing in F, we will introduce two rows, denoted by x and x and
introduce the above mechanism to force choosing either x or x to the optimal solution. If x (resp. x) is chosen
into the solution, we will interpret it as the solution to F assigns TRUE (resp. FALSE) to x.
Next, consider a clause C ≡ a ∨ b ∨ c.appearing in F. This clause numbers ... C ≡ a ∨ b ∨ c ...
requires that we choose at least one row from the rows corresponding a ... 01 ...
to a, b to c. This can be enforced by introducing a new coordinate a ... 00 ...
for the clauses C, and setting 1 for each row that if it is picked then b ... 01 ...
the clauses is satisfied. The question now is what do we set the target
b ... 00 ...
to be? Since a valid solution might have any number between 1 to
c ... 00 ...
3 as a sum of this coordinate. To overcome this, we introduce three
c ... 01 ...
new dummy rows, that store in this coordinate, the numbers 7, 8
C fix-up 1 000 07 000
and 9, and we set this coordinate in the target to be 10. Clearly, if
C fix-up 2 000 08 000
we pick to dummy rows into the optimal solution then sum in this
C fix-up 3 000 09 000
coordinate would exceed 10. Similarly, if we do not pick one of these
three dummy rows to the optimal solution, the maximum sum in this TARGET 10
coordinate would be 1 + 1 + 1 = 3, which is smaller than 10. Thus, the only possibility is to pick one dummy row,
and some subset of the rows such that the sum is 10. Notice, this “gadget” can accommodate any (non-empty)
subset of the three rows chosen for a, b and c.
We repeat this process for each clause of F. We end up with a set U of 2n + 3m vectors with n + m coordinate,
and the question if there is a subset of these vectors that add up to the target vector. There is such a subset
if and only if the original formula F is satisfiable, as can be easily verified. Furthermore, this reduction can be
done in polynomial time.
Finally, we convert these vectors into an instance of Subset Sum. Clearly, this instance of Subset Sum has a
solution if and only if the original instance of 3SAT had a solution. Since Subset Sum is in NP as an be easily
verified, we conclude that that Subset Sum is NP-Complete.

Theorem 3.3.1. Subset Sum is NP-Complete.

For a concrete example of the reduction, see Figure 3.1.

3.4. 3 dimensional Matching (3DM)

3DM
Instance: X,Y, Z sets of n elements, and T a set of triples, such that (a, b, c) ∈ T ⊆ X × Y × Z.
Question: Is there a subset S ⊆ T of n disjoint triples, s.t. every element of X ∪ Y ∪ Z is covered
exactly once.?

Theorem 3.4.1. 3DM is NP-Complete.

The proof is long and tedious and is omitted.

BTW, 2DM is polynomial (later in the course?).

28
 numbers
numbers a∨a b∨b c∨c d∨d D ≡ b∨c∨d C ≡ a∨b∨c
010000000001
a 1 0 0 0 00 01
010000000000
a 1 0 0 0 00 00
000100000001
b 0 1 0 0 00 01
000100000100
b 0 1 0 0 01 00 000001000100
c 0 0 1 0 01 00 000001000001
c 0 0 1 0 00 01 000000010000
d 0 0 0 1 00 00 000000010101
d 0 0 0 1 01 01 000000000007
C fix-up 1 0 0 0 0 00 07 000000000008
C fix-up 2 0 0 0 0 00 08 000000000009
C fix-up 3 0 0 0 0 00 09 000000000700
D fix-up 1 0 0 0 0 07 00 000000000800
D fix-up 2 0 0 0 0 08 00 000000000900
D fix-up 3 0 0 0 0 09 00
010101011010
TARGET 1 1 1 1 10 10
 
Figure 3.1: The Vec Subset Sum instance generated for the 3SAT formula F = b ∨ c ∨ d ∧ (a ∨ b ∨ c) is shown
on the left. On the right side is the resulting instance of Subset Sum.

3.5. Partition

Partition
Instance: A set S of n numbers.
Question: Is there a subset T ⊆ S s.t. t ∈T t = s ∈S\T s.?
Í Í

Theorem 3.5.1. Partition is NP-Complete.

Proof: Partition is in NP, as we can easily verify that such a partition is valid.
Reduction from Subset Sum. Let the given instance be n numbers a1, . . . , an and a target number t. Let
S = i= ai , and set an+1 = 3S − t and an+2 = 3S − (S − t) = 2S + t. It is easy to verify that there is a solution to
Ín
the given instance of subset sum, iff there is a solution to the following instance of partition:
a1, . . . , an, an+1, an+2 .
Clearly, Partition is in NP and thus it is NP-Complete.

3.6. Some other problems

It is not hard to show that the following problems are NP-Complete:

SET COVER
Instance: (S, F, k):
S: A set of n elements
F: A family of subsets of S, s.t. X ∈F X = S.
Ð
k: A positive integer.
Question: Are there k sets S1, . . . , Sk ∈ F that cover S. Formally, i Si = S?
Ð

29
 Part II
Dynamic programming

Chapter 4

Dynamic programming

The events of 8 September prompted Foch to draft the later legendary signal: “My centre is giving way, my right is
in retreat, situation excellent. I attack.” It was probably never sent.
– – The first world war, John Keegan..

4.1. Basic Idea - Partition Number

Definition 4.1.1. For a positive integer n, the partition number of n, denoted by p(n), is the number of different
ways to represent n as a decreasing sum of positive integers.

The different number of partitions of 6 are shown 6=6

on the right. 6=5+1
It is natural to ask how to compute p(n). The 6=4+2 6=4+1+1
“trick” is to think about a recursive solution and ob- 6=3+3 6=3+2+1 6+3+1+1+1
serve that once we decide what is the leading num- 6=2+2+2 6=2+2+1+1 6=2+1+1+1+1
ber d, we can solve the problem recursively on the 6=1+1+1+1+1+1
remaining budget n − d under the constraint that no
number exceeds d..

Suggestion 4.1.2. Recursive algorithms are one of the main tools in developing algorithms (and writing pro- T IP
grams). If you do not feel comfortable with recursive algorithms you should spend time playing with recursive
algorithms till you feel comfortable using them. Without the ability to think recursively, this class would be a
long and painful torture to you. Speak with me if you need guidance on this topic.

30
 The resulting algorithm is depicted on the right. We PartitionsI(num, d) //d-max digit
are interested in analyzing its running time. To this end, if (num ≤ 1) or (d = 1)
draw the recursion tree of Partitions and observe that return 1
the amount of work spend at each node, is proportional if d > num
to the number of children it has. Thus, the overall time d ← num
spend by the algorithm is proportional to the size of res ← 0
the recurrence tree, which is proportional (since every for i ← d down to 1
node is either a leaf or has at least two children) to the res = res + PartitionsI(num − i,i)
number of leafs in the tree, which is Θ(p(n)). return res
√ This is not very exciting, since it is easy verify that
3 n/4 ≤ p(n) ≤ nn . Partitions(n)
return PartitionsI(n, n)
Exercise 4.1.3. Prove the above bounds on p(n) (or better bounds).

Suggestion 4.1.4. Exercises in the class notes are a natural easy questions for inclusions in exams. You probably
T IP
want to spend time doing them.
√
e π 2n/3
Hardy and Ramanujan (in 1918) showed that p(n) ≈ √ (which I am sure was your first guess).
4n 3
It is natural to ask, if there is a faster algorithm. Or more specifically, why is the algorithm Partitions
so slowwwwwwwwwwwwwwwwww? The answer is that during the computation of Partitions(n) the function
PartitionsI(num, max_digit) is called a lot of times with the same parameters.
An easy way to overcome this problem is cache the PartitionsI_C(num, max_digit)
results of PartitionsI using a hash table.¬ Whenever if (num ≤ 1) or (max_digit = 1)
PartitionsI is being called, it checks in a cache table if return 1
it already computed the value of the function for this if max_digit > num
parameters, and if so it returns the result. Otherwise, it d ← num
computes the value of the function and before returning if hnum, max_digiti in cache
the value, it stores it in the cache. This simple (but return cache(hnum, max_digiti)
powerful) idea is known as memoization. res ← 0
What is the running time of PartitionS_C? Analyz- for i ← max_digit down to 1 do
ing recursive algorithm that have been transformed by res += PartitionsI_C(num − i,i)
memoization are usually analyzed as follows: (i) bound cache(hnum, max_digiti) ← res
the number of values stored in the hash table, and (ii) return res
bound the amount of work involved in storing one value
into the hash table (ignoring recursive calls). PartitionS_C(n)
Here is the argument in this case: return PartitionsI_C(n, n)
(A) If a call to PartitionsI_C takes (by itself) more than constant time, then this call performs a store in the
cache.
(B) Number of store operations in the cache is O(n2 ), since this is the number of different entries stored in the
cache. Indeed, for PartitionsI_C(num, max_digit), the parameters num and max_digit are both integers
in the range 1, . . . , n.
(C) We charge the work in the loop to the resulting store. The work in the loop is at most O(n) time (since
max_digit ≤ n).
(D) As such, the overall running time of PartitionS_C(n) is O n2 × O(n) = O n3 .

Note, that this analysis is naive but it would be sufficient

for our purposes (verify that the bound of O(n3 ) on the
running time is tight in this case).
¬ Throughout the course, we will assume that a hash table operation can be done in constant time. This is a reasonable
assumption using randomization and perfect hashing.

31
4.1.1. A Short sermon on memoization
This idea of memoization is generic and nevertheless very useful. To recap, it works by taking a recursive
function and caching the results as the computations goes on. Before trying to compute a value, check if it was
already computed and if it is already stored in the cache. If so, return result from the cache. If it is not in the
cache, compute it and store it in the cache (for the time being, you can think about the cache as being a hash
table).
• When does it work: There is a lot of inefficiency in the computation of the recursive function because
the same call is being performed repeatedly.
• When it does NOT work:
(A) The number of different recursive function calls (i.e., the different values of the parameters in the
recursive call) is “large”.
(B) When the function has side effects.

Tidbit 4.1.5. Some functional programming languages allow one to take a recursive function f (·) that you already
tidbit
implemented and give you a memorized version f 0(·) of this function without the programmer doing any extra
work. For a nice description of how to implement it in Scheme see [ASS96].

It is natural to ask if we can do better than just using caching? As usual in life – more pain, more gain.
Indeed, in a lot of cases we can analyze the recursive calls, and store them directly in an (sometime multi-
dimensional) array. This gets rid of the recursion (which used to be an important thing long time ago when
memory, used by the stack, was a truly limited resource, but it is less important nowadays) which usually yields
a slight improvement in performance in the real world.
This technique is known as dynamic programming ­ . We can sometime save space and improve running
time in dynamic programming over memoization.

Dynamic programming made easy:

(A) Solve the problem using recursion - easy (?).
(B) Modify the recursive program so that it caches the results.
(C) Dynamic programming: Modify the cache into an array.

4.2. Example – Fibonacci numbers

Let us revisit the classical problem of computing Fibonacci numbers.

4.2.1. Why, where, and when?

To remind the reader, in the Fibonacci sequence, the first two numbers F0 = 0 and F1 = 1, and Fi = Fi−1 + Fi−2 ,
for i > 1. This sequence was discovered independently in several places and times. From Wikipedia:

“The Fibonacci sequence appears in Indian mathematics, in connection with Sanskrit prosody.
In the Sanskrit oral tradition, there was much emphasis on how long (L) syllables mix with the
short (S), and counting the different patterns of L and S within a given fixed length results in the
Fibonacci numbers; the number of patterns that are m short syllables long is the Fibonacci number
Fm+1 .”

(To see that, imagine that a long syllable is equivalent in length to two short syllables.) Surprisingly, the credit
for this formalization goes back more than 2000 years (!)
­ As usual in life, it is not dynamic, it is not programming, and its hardly a technique. To overcome this, most texts find creative
ways to present this topic in the most opaque way possible.

32
 FibDP(n)
if n ≤ 1
return 1
if F[n] initialized
return F[n]
F[n] ⇐=FibDP(n − 1)+FibDP(n − 2)
return F[n]
Figure 4.1

Fibonacci was a decent mathematician (1170—1250 AD), and his most significant and lasting contribution
was spreading the Hindu-Arabic numerical system (i.e., zero) in Europe. He was the son of a rich merchant that
spend much time growing up in Algiers, where he learned the decimal notation system. He traveled throughout
the Mediterranean world to study mathematics. When he came back to Italy he published a sequence of books
(the first one “Liber Abaci” contained the description of the decimal notations system). In this book, he also
posed the following problem:
Consider a rabbit population, assuming that: A newly born pair of rabbits, one male, one female,
are put in a field; rabbits are able to mate at the age of one month so that at the end of its second
month a female can produce another pair of rabbits; rabbits never die and a mating pair always
produces one new pair (one male, one female) every month from the second month on. The puzzle
that Fibonacci posed was: how many pairs will there be in one year?
(The above is largely based on Wikipedia.)

4.2.2. Computing Fibonacci numbers

The recursive function for computing Fibonacci numbers FibR(n)

is depicted on the right. As before, the running time of if n = 0
FibR(n) is proportional to O(Fn ), where Fn is the nth Fi- return 1
bonacci number. It is known that if n = 1
" √ !n √ !n# return 1
1 1+ 5 1− 5
Fn = √ + = Θ(φn ), return FibR(n − 1) + FibR(n − 2)
5 2 2
√
where φ = 1+2 5 .
We can now use memoization, and with a bit of care, it is easy enough to come up with the dynamic
programming version of this procedure, see FibDP in Figure 4.1. Clearly, the running time of FibDP(n) is
linear (i.e., O(n)).
A careful inspection of FibDP exposes the fact that it fills the array F[...] from left to right. In particular,
it only requires the last two numbers in the array.
As such, we can get rid of the array all together, and reduce space needed FibI(n)
to O(1): This is a phenomena that is quite common in dynamic programming: prev ← 0, curr ← 1
By carefully inspecting the way the array/table is being filled, sometime one for i = 1 to n do
can save space by being careful about the implementation. next ← curr + prev
The running time of FibI is identical to the running time of FibDP. Can prev ← curr
we do better? curr ← next
Surprisingly, the answer is yes, to this end observe that return curr
x
    
y 0 1
= .
x+y 1 1 y

33
 h a r – p e l e d
s h a r p <space> e y e d
1 0 0 0 1 0 1 0 1 0 0

– l
Insert: delete: replace:
s y
e
(still) insert: ignore:
<space> e

Figure 4.2: Interpreting edit-distance as a alignment task. Aligning identical characters to each other is free of
cost. The price in the above example is 4. There are other ways to get the same edit-distance in this case.

As such,
2   n−3 
Fn−1 Fn−2 Fn−3 F2
        
0 1 0 1 0 1
= = = .
Fn 1 1 Fn−1 1 1 Fn−2 1 1 F1
Thus, computing the nth Fibonacci number can be done by computing
  n−3
0 1
.
1 1
How to this quickly? Well, we know that a∗b∗c = (a∗b)∗c = FastExp(a, n)
a∗(b∗c)® , as such one can compute a n by repeated squaring, if n = 0 then
see pseudo-code on the right. The running time of FastExp return 1
is O(log n) as can be easily verified. Thus, we can compute in if n = 1 then
Fn in O(log n) time. return a
But, something is very strange. Observe that Fn has ≈ if n is even then
log10 1.68... n = Θ(n) digits. How can we compute a number return (FastE xp(a, n/2))2
that is that large in logarithmic time? Well, we assumed else
that the time to handle a number is O(1) independent of its

2
return a ∗ FastExp a, n−1
2
size. This is not true in practice if the numbers are large.
Naturally, one has to be very careful with such assumptions.

4.3. Edit Distance

We are given two strings A and B, and we want to know how close the two strings are too each other. Namely,
how many edit operations one has to make to turn the string A into B?
We allow the following operations: (i) insert a character, (ii) delete a character, and (iii) replace a character
by a different character. Price of each operation is one unit.
For example, consider the strings A =“har-peled” and B =“sharp eyed”. Their edit distance is 4, as can be
easily seen.
® Associativity of multiplication...

34
 But how do we compute the edit-distance (min ed(A[1..m], B[1..n])
# of edit operations needed)? if m = 0 return n
The idea is to list the edit operations from left to if n = 0 return m
right. Then edit distance turns into a an alignment pinsert = ed(A[1..m], B[1..(n − 1)]) + 1
problem. See Figure 4.2. pdelete = ed(A[1..(m − 1)], B[1..n]) + 1
In particular, the idea of the recursive algorithm pr/i = ed(A[1..(m
is to inspect the last character and decide which of  − 1)], B[1..(n
 − 1)] )
+ A[m] , B[n]
the categories it falls into: insert, delete or ignore. return min pinsert , pdelete, pr eplace/ignor e


See pseudo-code on the right.
The running time of ed(...)? Clearly exponential, and roughly 2n+m , where n + m is the size of the input.
So how many different recursive calls ed performs? Only:O(m ∗ n) different calls, since the only parameters
that matter are n and m.
So the natural thing is to introduce edM(A[1..m], B[1..n])
memoization. The resulting algorithm edM if m = 0 return n
is depicted on the right. The running if n = 0 return m
time of edM(n, m) when executed on two if T[m, n] is initialized then return T[m, n]
strings of length n and m respective is pinsert = edM(A[1..m], B[1..(n − 1)]) + 1
O(nm), since there are O(nm) store opera- pdelete = edM(A[1..(m − 1)], B[1..n]) + 1 
tions in the cache, and each store requires pr/i = edM(A[1..(m − 1)], B[1..(n − 1)]) + A[m] , B[n]

O(1) time (by charging one for each recur- T[m, n] ← min pinsert , pdelete, pr eplace/ignor e


sive call). Looking on the entry T[i, j] in return T[m, n]
the table, we realize that it depends only
on T[i − 1, j], T[i, j − 1] and T[i − 1, j − 1]. Thus, instead of recursive algorithm, we can fill the table T row by
row, from left to right.
edDP(A[1..m], B[1..n]) The dynamic programming version that
for i = 1 to m do T[i, 0] ← i uses a two dimensional array is pretty sim-
for j = 1 to n do T[0, j] ← j ple now to derive and is depicted on the
for i ← 1 to m do left. Clearly, it requires O(nm) time, and
for j ← 1 to n do O(nm) space. See the pseudo-code of the
pinsert = T[i, j − 1] + 1 resulting algorithm edDP on the left.
pdelete = T[i − 1, j] + 1 It is enlightening to think about the
pr/ignor e = T[i − 1. j − 1] + A[i] , B[ j]
  algorithm as computing for each T[i, j] the
T[i, j] ← min pinsert , pdelete, pr/ignor e

cell it got the value from. What you get is
return T[m, n] a tree encoded in the table. See Figure 4.3.
It is now easy to extract from the table the
sequence of edit operations that realizes the minimum edit distance between A and B. Indeed, we start a walk
on this graph from the node corresponding to T[n, m]. Every time we walk left, it corresponds to a deletion,
every time we go up, it corresponds to an insertion, and going sideways corresponds to either replace/ignore.
Note, that when computing the ith row of T[i, j], we only need to know the value of the cell to the left of
the current cell, and two cells in the row above the current cell. It is thus easy to verify that the algorithm
needs only the remember the current and previous row to compute the edit distance. We conclude:

Theorem 4.3.1. Given two strings A and B of length n and m, respectively, one can compute their edit distance
in O(nm). This uses O(nm) space if we want to extract the sequence of edit operations, and O(n + m) space if we
only want to output the price of the edit distance.

Exercise 4.3.2. Show how to compute the sequence of edit-distance operations realizing the edit distance using
only O(n + m) space and O(nm) running time. (Hint: Use a recursive algorithm, and argue that the recursive
call is always on a matrix which is of size, roughly, half of the input matrix.)

35
 A L G O R I T H M
0 ← 1 ← 2 ← 3 ← 4 ← 5 ← 6 ← 7 ← 8 ← 9
↑ v
A 1 0 ← 1 ← 2 ← 3 ← 4 ← 5 ← 6 ← 7 ← 8
↑ ↑ v
L 2 1 0 ← 1 ← 2 ← 3 ← 4 ← 5 ← 6 ← 7
↑ ↑ ↑ v -
T 3 2 1 1 ← 2 ← 3 ← 4 4 ← 5 ← 6
↑ ↑ ↑ ↑ v -
R 4 3 2 2 2 2 ← 3 ← 4 ← 5 ← 6
↑ ↑ ↑ - - v -
U 5 4 3 3 3 3 3 ← 4 ← 5 ← 6
↑ ↑ ↑ - - - v
I 6 5 4 4 4 4 3 ← 4 ← 5 ← 6
↑ ↑ ↑ ↑ ↑ ↑ ⇑
S 7 6 5 5 5 5 4 ← 4 ← 5 ← 6
↑ ↑ ↑ ↑ ↑ ↑ ↑ v
T 8 7 6 6 6 6 5 4 ← 5 ← 6
↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ v
I 9 8 7 7 7 7 6 5 5 ← 6
↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑ v
C 10 9 8 8 8 8 7 6 6 6

Figure 4.3: Extracting the edit operations from the table.

4.3.1. Shortest path in a DAG and dynamic programming

Given a dynamic programming problem and its associated recursive program, one can consider all the different
possible recursive calls, as configurations. We can create graph, every configuration is a node, and an edge is
introduced between two configurations if one configuration is computed from another configuration, and we put
the additional price that might be involved in moving between the two configurations on the edge connecting
them. As such, for the edit distance, we have directed edges from the vertex (i, j) to (i, j − 1) and (i − 1, j) both
with weight 1 on them. Also, we have an edge between (i, j) to (i − 1, j − 1) which is of weight 0 if A[i] = B[ j]
and 1 otherwise. Clearly, in the resulting graph, we are asking for the shortest path between (n, m) and (0, 0).
And here are where things gets interesting. The resulting graph G is a DAG (directed acyclic graph ¯ ).
DAG can be interpreted as a partial ordering of the vertices, and by topological sort on the graph (which takes
linear time), one can get a full ordering of the vertices which agrees with the DAG. Using this ordering, one can
compute the shortest path in a DAG in linear time (in the size of the DAG). For edit-distance the DAG size is
O(nm), and as such this algorithm takes O(nm) time.
This interpretation of dynamic programming as a shortest path problem in a DAG is a useful way of thinking
about it, and works for many dynamic programming problems.
More surprisingly, one can also compute the longest path in a DAG in linear time. Even for negative weighted
edges. This is also sometime a problem that solving it is equivalent to dynamic programming.
¯ No cycles in the graph – its a miracle!

36
Chapter 5

Dynamic programming II - The Recursion

Strikes Back

“No, mademoiselle, I don’t capture elephants. I content myself with living among them. I like them. I like looking at
them, listening to them, watching them on the horizon. To tell you the truth, I’d give anything to become an elephant
myself. That’ll convince you that I’ve nothing against the Germans in particular: they’re just men to me, and that’s
enough.”
– – The roots of heaven, Romain Gary.

5.1. Optimal search trees

Given a binary search tree T, the time to search for an element x, that is stored in T, is O(1 + depth(T, x)),
where depth(T, x) denotes the depth of x in T (i.e., this is the length of the path connecting x with the root of
T).

Problem 5.1.1. Given a set of n (sorted) keys A[1 . . . n], build the 12 32
best binary search tree for the elements of A.
4 32 12 45
Note, that we store the values in the internal node of the binary
trees. The figure on the right shows two possible search trees for the
21 45 4 21
same set of numbers. Clearly, if we are accessing the number 12 all
the time, the tree on the left would be better to use than the tree on Two possible search trees for the set
the right. A = [4, 12, 21, 32, 45].
Usually, we just build a balanced binary tree, and this is good
enough. But assume that we have additional information about what is the frequency in which we access the
element A[i], for i = 1, . . . , n. Namely, we know that A[i] is going be accessed f [i] times, for i = 1, . . . , n.
Õn
In this case, we know that the total search time for a tree T is S(T) = (depth(T, i) + 1)f [i], where depth(T, i)
i=1
is the depth of the node in T storing the value A[i]. Assume that A[r] is the value stored in the root of the tree
T. Clearly, all the values smaller than A[r] are in the subtree leftT , and all values larger than r are in rightT .
Thus, the total search time, in this case, is
price of access to root
z }| {
r−1
Õ Õn n
Õ
S(T) = (depth(leftT , i) + 1)f [i] + f [i] + (depth(rightT , i) + 1)f [i].
i=1 i=1 i=r+1
Observe, that if T is the optimal search tree for the access frequencies f [1], . . . , f [n], then the subtree leftT
must be optimal for the elements accessing it (i.e., A[1 . . . r − 1] where r is the root).
Thus, the price of T is
Õn
S(T) = S(leftT ) + S(rightT ) + f [i],
i=1
where S(Q) is the price of searching in Q for the frequency of elements stored in Q.

37
 Figure 5.1: A polygon and two possible triangulations of the polygon.

This recursive formula naturally gives rise

CompBestTreeI (A[i . . . j], f [i . . . j] )
to a recursive algorithm, which is depicted on
for r = i . . . j do
the right. The naive implementation requires
Tle f t ← CompBestTreeI(A[i . . . r − 1], f [i . . . r − 1])
O(n2 ) time (ignoring the recursive call). But
Tright ← CompBestTreeI(A[r +
1 . . . j], f [r + 1 . . . j])
in fact, by a more careful implementation, to-
Tr ← Tree Tle f t , A[r] , Tright
gether with the tree T, we can also return the
Pr ← S(Tr )
price of searching on this tree with the given fre-
quencies. Thus, this modified algorithm. Thus,
return cheapest tree out of Ti , . . . ,Tj .
the running time for this function takes O(n)
time (ignoring recursive calls). The running
CompBestTree (A[1 . . . n], f [1 . . . n] )
time of the resulting algorithm is
return CompBestTreeI( A[1 . . . n], f [1 . . . n])
n−1
Õ
α(n) = O(n) + (α(i) + α(n − i − 1)),
i=0

and the solution of this recurrence is O(n3n ).

We can, of course, improve the running time using memoization. There are only O(n2 ) different recursive
calls, and as such, the running time of CompBestTreeMemoize is O(n2 ) · O(n) = O(n3 ).
Theorem 5.1.2. One can compute the optimal binary search tree in O n3 time using O n2 space.

A further improvement raises from the fact that the root location is “monotone”. Formally, if R[i, j] denotes
the location of the element stored in the root for the elements A[i . . . j] then it holds that R[i, j − 1]
≤ R[i, j] ≤
R[i, j +1]. This limits the search space, and we can be more efficient in the search. This leads to O n2 algorithm.
Details are in Jeff Erickson class notes.

5.2. Optimal Triangulations

Given a convex polygon P in the plane, we would like to find the triangulation of P of minimum total length.
Namely, the total length of the diagonals of the triangulation of P, plus the (length of the) perimeter of P are
minimized. See Figure 5.1.
Definition 5.2.1. A set S ⊆ Rd is convex if for any to x, y ∈ S, the segment x y is contained in S.
A convex polygon is a closed cycle of segments, with no vertex pointing inward. Formally, it is a simple
closed polygonal curve which encloses a convex set.
A diagonal is a line segment connecting two vertices of a polygon which are not adjacent. A triangulation
is a partition of a convex polygon into (interior) disjoint triangles using diagonals.

Observation 5.2.2. Any triangulation of a convex polygon with n vertices is made out of exactly n−2 triangles.

38
 Our purpose is to find the triangulation of P that has the minimum
total length. Namely, the total length of diagonals used in the triangu-
lation is minimized. We would like to compute the optimal triangulation
using divide and conquer. As the figure on the right demonstrate, there
is always a triangle in the triangulation, that breaks the polygon into
two polygons. Thus, we can try and guess such a triangle in the optimal
triangulation, and recurse on the two polygons such created. The only
difficulty, is to do this in such a way that the recursive subproblems can
be described in succinct way.
4
5 To this end, we assume that the polygon is specified as list of vertices 1 . . . n
in a clockwise ordering. Namely, the input is a list of the vertices of the polygon,
3
for every vertex, the two coordinates are specified. The key observation, is that
in any triangulation of P, there exist a triangle that uses the edge between
6 vertex 1 and n (red edge in figure on the left).
2
In particular, removing the triangle using the edge 1 − n leaves us with two
7
polygons which their vertices are consecutive along the original polygon.

8 Let M[i, j] denote the price of triangulating a polygon starting at vertex i

1
and ending at vertex j, where every diagonal used contributes its length twice
to this quantity, and the perimeter edges contribute their length exactly once. We have the following “natural”
recurrence:


 0 j ≤i


M[i, j] = 0 j =i+1 .

 mini<k< j (∆(i, j, k) + M[i, k] + M[k, j]) Otherwise


q
Where Dist(i, j) = (x[i] − x[ j])2 + (y[i] − y[ j])2 and ∆(i, j, k) = Dist(i, j) + Dist( j, k) + Dist(i, k), where the ith
point has coordinates (x[i], y[i]), for i = 1, . . . , n. Note, that the quantity we are interested in is M[1, n], since it
the triangulation of P with minimum total weight.
Using dynamic programming (or just memoization), we get an algorithm that computes optimal triangula-
tion in O(n3 ) time using O(n2 ) space.

5.3. Matrix Multiplication

We are given two matrix: (i) A is a matrix with dimensions p × q (i.e., p rows and q columns) and (ii) B is a
matrix of size q × r. The product matrix AB, with dimensions p × r, can be computed in O(pqr) time using the
standard algorithm.
A 1000 × 2 Things becomes considerably more interesting when we have to multiply a chain for matri-
B 2 × 1000ces. Consider for example the three matrices A, B and C with dimensions as listed on the left.
C 1000 × 2Computing the matrix ABC = A(BC) requires 2·1000·2+1000·2·2 = 8, 000 operations. On the
other hand, computing the same matrix using (AB)C requires 1000 · 2 · 1000 + 1000 · 1000 · 2 =
4, 000, 000. Note, that matrix multiplication is associative, and as such (AB)C = A(BC).
Thus, given a chain of matrices that we need to multiply, the exact ordering in which we do the multiplication
matters as far to multiply the order is important as far as efficiency.

Problem 5.3.1. The input is n matrices M1, . . . , Mn such that Mi is of size D[i − 1] × D[i] (i.e., Mi has D[i − 1]
rows and D[i] columns), where D[0 . . . n] is array specifying the sizes. Find the ordering of multiplications to
compute M1 · M2 · · · Mn−1 · Mn most efficiently.

39
 Again, let us define a recurrence for this problem, where M[i, j] is the amount of work involved in computing
the product of the matrices Mi · · · M j . We have


 0 j =i


M[i, j] = D[i − 1] · D[i] · D[i + 1] j =i+1

 mini ≤k< j (M[i, k] + M[k + 1, j] + D[i − 1] · D[k] · D[ j])

j > i + 1.

Again, using memoization (or dynamic programming), one can compute M[1, n], in O(n3 ) time, using O(n2 )
space.

5.4. Longest Ascending Subsequence

Given an array of numbers A[1 . . . n] we are interested in finding the longest ascending subsequence. For
example, if A = [6, 3, 2, 5, 1, 12] the longest ascending subsequence is 2, 5, 12. To this end, let M[i] denote longest
increasing subsequence having A[i] as the last element in the subsequence. The recurrence on the maximum
possible length, is
1 n=1



M[n] =
 1 + 1≤k<n,max
 M[k] otherwise.
 A[k]< A[n]

The length of the longest increasing subsequence is maxi=1 n M[i]. Again, using dynamic programming, we

get an algorithm with running time O(n2 ) for this problem. It is also not hard to modify the algorithm so that
it outputs this sequence (you should figure out the details of this modification). A better O(n log n) solution is
possible using some data-structure magic.

5.5. Pattern Matching

Tidbit 5.5.1. Magna Carta or Magna Charta - the great charter that King John of England was forced by the
English barons to grant at Runnymede, June 15, 1215, traditionally interpreted as guaranteeing certain civil tidbit
and political liberties.

Assume you have a string S = ”Magna Carta” and a pattern P = ”?ag ∗ at ∗ a ∗ ” where “?” can match a
single character, and “*” can match any substring. You would like to decide if the pattern matches the string.
We are interested in solving this problem using dynamic programming. This is not too hard since this is
similar to the edit-distance problem that was already covered.

IsMatch(S[1 . . . n], P[1 . . . m])

if m = 0 and n = 0 then return TRUE. The resulting code is depicted on the left,
if m = 0 then return FALSE. and as you can see this is pretty tedious.
if n = 0 then Now, use memoization together with this
if P[1 . . . m] is all stars then return TRUE recursive code, and you
get an algorithm
else return FALSE with running time O mn2 and space O(nm),
if (P[m] = ’?’) then where the input string of length n, and m is
return IsMatch(S[1 . . . n − 1], P[1 . . . m − 1]) the length of the pattern.
if (P[m] , ’*’) then Being slightly more clever, one can get a
if P[m] , S[n] then return FALSE faster algorithm with running time O(nm).
else return IsMatch(S[1 . . . n − 1], P[1 . . . m − 1]) BTW, one can do even better. A O(m + n)
for i = 0 to n do time is possible but it requires Knuth-Morris-
if IsMatch(S[1 . . . i], P[1 . . . m − 1]) then Pratt algorithm, which is a fast string match-
return TRUE ing algorithm.
return FALSE

40
Figure 5.2: A drawing of the Mona Lisa by solving a TSP instance. The figure on the right is the TSP in the
eyes region.

Figure 5.3: A certain country and its optimal TSP tour.

5.6. Slightly faster TSP algorithm via dynamic programming

TSP: Traveling Salesperson Problem

Instance: A graph G = (V, E) with non-negative edge costs/lengths. Cost c(e) for each edge e ∈ E.
Question: Find a tour of minimum cost that visits each node.

No polynomial time algorithm known for TSP– the problem is NP-Hard.

Even an exponential Time algorithm requires some work. Indeed, there are n! potential TSP tours. Clearly,
√
n! ≤ nn = exp(n ln n) and n! ≥ (n/2)n/2 = exp((n/2) ln(n/2)). Using Stirling’s formula, we have n! ' n(n/e)n ,
which gives us a somewhat tighter estimate n! = Θ(2cn log n ) for some constant c > 1.
So, naively, any running time algorithm would have running time (at least) Ω(n!). Can we do better? Can
we get a ≈ 2O(n) running time algorithm in this case?

Towards a Recursive Solution.

(A) Order the vertices of V in some arbitrary order: v1, v2, . . . , vn .
(B) opt(S): optimum TSP tour for the vertices S ⊆ V in the graph restricted to S. We would like to compute
opt(V).

41
 Can we compute opt(S) recursively?
(A) Say v ∈ S. What are the two neighbors of v in optimum tour in S?
(B) If u, w are neighbors of v in an optimum tour of S then removing v gives an optimum path from u to w
visiting all nodes in S − {v}.
Path from u to w is not a recursive subproblem! Need to find a more general problem to allow recursion.

We start with a more general problem: TSP Path.

TSP Path
Instance: A graph G = (V, E) with non-negative edge costs/lengths(c(e) for edge e) and two nodes
s, t.
Question: Find a path from s to t of minimum cost that visits each node exactly once.

We can solve the regular TSP problem using this problem.

We define a recursive problem for the optimum TSP Path problem, as follows:

opt(u, v, S) : optimum TSP Path from u to v in the graph restricted to S s.t. u, v ∈ S.

(A) What is the next node in the optimum path from u to v?

(B) Suppose it is w. Then what is opt(u, v, S)?
(C) opt(u, v, S) = c(u, w) + opt(w, v, S − {u})
(D) We do not know w! So try all possibilities for w.

A Recursive Solution.
(A) opt(u, v, S) = minw ∈S,w,u,v (c(u, w) + opt(w, v, S − {u}))
(B) What are the subproblems for the original problem opt(s, t, V)? For every subset S ⊆ V, we have the
subproblem opt(u, v, S) for u, v ∈ S.
As usual, we need to bound the number subproblems in the recursion:
(A) number of distinct subsets S of V is at most 2n
(B) number of pairs of nodes in a set S is at most n2
(C) hence number of subproblems is O(n2 2n )

Exercise 5.6.1. Show that one can compute TSP using above dynamic program in O(n3 2n ) time and O(n2 2n )
space.

Lemma 5.6.2. Given a graph G with n vertices, one can solve TSP in O(n3 2n ) time.

The disadvantage of dynamic programming solution is that it uses a lot of memory.

42
 Part III
Approximation algorithms

Chapter 6

Approximation algorithms

6.1. Greedy algorithms and approximation algorithms

A natural tendency in solving algorithmic problems is to locally do whats seems to be the right thing. This is
usually referred to as greedy algorithms. The problem is that usually these kind of algorithms do not really
work. For example, consider the following optimization version of Vertex Cover:

VertexCoverMin
Instance: A graph G, and integer k.
Question: Return the smallest subset S ⊆ V(G), s.t. S touches all the edges of G.

For this problem, the greedy algorithm will always take the vertex with the high-
est degree (i.e., the one covering the largest number of edges), add it to the cover set,
remove it from the graph, and repeat. We will refer to this algorithm as GreedyVer-
texCover.
It is not too hard to see that this algorithm does not output the optimal vertex-
cover. Indeed, consider the graph depicted on the right. Clearly, the optimal solution
Figure 6.1: Example.
is the black vertices, but the greedy algorithm would pick the four yellow vertices.
This of course still leaves open the possibility that, while we do not get the optimal
vertex cover, what we get is a vertex cover which is “relatively good” (or “good enough”).
Definition 6.1.1. A minimization problem is an optimization problem, where we look for a valid solution that
minimizes a certain target function.

Example 6.1.2. In the VertexCoverMin problem the (minimization) target function is the size of the cover. For-
mally Opt(G) = minS ⊆V (G),S cover of G |S|.
The VertexCover(G) is just the set S realizing this minimum.

Definition 6.1.3. Let Opt(G) denote the value of the target function for the optimal solution.

43
 Intuitively, a vertex-cover of size “close” to the optimal solution would be considered to be good.
Definition 6.1.4. Algorithm Alg for a minimization problem Min achieves an approximation factor α ≥ 1 if for
all inputs G, we have:
Alg(G)
≤ α.
Opt(G)
We will refer to Alg as an α-approximation algorithm for Min.

L R L R L R L R L R L R

Figure 6.2: Lower bound for greedy vertex cover.

As a concrete example, an algorithm is a 2-approximation for VertexCoverMin, if it outputs a vertex-cover

which is at most twice the size of the optimal solution for vertex cover.
So, how good (or bad) is the GreedyVertexCover algorithm described above? Well, the graph in Figure 6.1
shows that the approximation factor of GreedyVertexCover is at least 4/3.
It turns out that GreedyVertexCover performance is considerably worse. To this end, consider the following
bipartite graph: G n = (L ∪ R, E), where L is a set of n vertices. Next, for i = 2, . . . , n, we add a set Ri of bn/ic
vertices, to R, each one of them of degree i, such that all of them (i.e., all vertices of degree i in R) are connected
to distinct vertices in L. The execution of GreedyVertexCover on such a graph is shown in Figure 6.2.
Clearly, in G n all the vertices in L have degree at most n − 1, since they are connected to (at most) one
vertex of Ri , for i = 2, . . . , n. On the other hand, there is a vertex of degree n at R (i.e., the single vertex of Rn ).
Thus, GreedyVertexCover will first remove this vertex. We claim, that GreedyVertexCover will remove all the
vertices of R2, . . . , Rn and put them into the vertex-cover. To see that, observe that if R2, . . . , Ri are still active,
then all the nodes of Ri have degree i, all the vertices of L have degree at most i − 1, and all the vertices of
R2, . . . , Ri−1 have degree strictly smaller than i. As such, the greedy algorithms will use the vertices of Ri . Easy
induction now implies that all the vertices of R are going to be picked by GreedyVertexCover. This implies the
following lemma.
Lemma 6.1.5. The algorithm GreedyVertexCover is Ω(log n) approximation to the optimal solution to Vertex-
CoverMin.

Proof: Consider the graph G n above. The optimal solution is to pick all the vertices of L to the vertex cover,
which results in a cover of size n. On the other hand, the greedy algorithm picks the set R. We have that
n n j k n  n
Õ Õ n Õ n  Õ 1
|R| = |Ri | = ≥ −1 ≥ n − 2n = n(Hn − 2).
i=2 i=2
i i=2
i i=1
i

Here, Hn = = lg n + Θ(1) is the nth harmonic number. As such, the approximation ratio for GreedyVer-
Ín
i=1 1/i
|R| n(Hn − 2)
texCover is ≥ = = Ω(log n).
|L| n

44
Theorem 6.1.6. The greedy algorithm for VertexCover achieves Θ(log n) approximation, where n is the number
of vertices in the graph. Its running time is O(mn2 ).

Proof: The lower bound follows from Lemma 6.1.5. The upper bound follows from the analysis of the greedy
of Set Cover, which will be done shortly.
As for the running time, each iteration of the algorithm takes O(mn) time, and there are at most n iterations.

6.1.1. Alternative algorithm – two for the price of one

One can still do much better than the greedy algorithm in this case. In particular, let ApproxVertexCover
be the algorithm that chooses an edge from G, add both endpoints to the vertex cover, and removes the two
vertices (and all the edges adjacent to these two vertices) from G. This process is repeated till G has no edges.
Clearly, the resulting set of vertices is a vertex-cover, since the algorithm removes an edge only if it is being
covered by the generated cover.

Theorem 6.1.7. ApproxVertexCover is a 2-approximation algorithm for VertexCoverMin that runs in O(n2 )
time.

Proof: Every edge picked by the algorithm contains at least one vertex of the optimal solution. As such, the
cover generated is at most twice larger than the optimal.

6.2. Fixed parameter tractability, approximation, and fast exponential time

algorithms (to say nothing of the dog)
6.2.1. A silly brute force algorithm for vertex cover
So given a graph G = (V, E) with n vertices, we can approximate VertexCoverMin up to a factor of two in
polynomial time. Let K be this approximation – we know that any vertex cover in G must be of size at least
K/2, and we have a cover of size K. Imagine the case that K is truly small – can we compute the optimal
vertex-cover in this case quickly? Well, of course, we could just try all possible subsets of vertices size at most
K, and check for each one whether it is a cover or not. Checking if a specific set of vertices is a cover takes
O(m) = O(n2 ) time, where m = |E|. So, the running time of this algorithm is
K     K
Õ n Õ    
O n ≤
2
O n ·n =O n
i 2 K+2
,
i=1
i i=1

where ni is the number of subsets of the vertices of G of size exactly i. Observe that we do not require to know

K – the algorithm can just try all sizes of subsets, till it finds a solution. We thus get the following (not very
interesting result).

Lemma 6.2.1. Given a graph G = (V, E) with n vertices, one can solve VertexCoverMin in O nα+2 time, where

α is the size the minimum vertex cover.

6.2.2. A fixed parameter tractable algorithm

As before, our input is a graph G = (V, E), for which we want to compute a vertex-cover of minimum size. We
need the following definition:

Definition 6.2.2. Let G = (V, E) be a graph. For a subset S ⊆ V, let GS be the induced subgraph over S.
Namely, it is a graph with the set of vertices being S. For any pair of vertices x, y ∈ V, we have that the edge
xy ∈ E(GS ) if and only if x y ∈ E(G), and x, y ∈ S.

45
 fpVertexCoverInner (X, β)
// Computes minimum vertex cover for the induced graph GX
// β: size of VC computed so far.
if X = ∅ or GX has no edges then return β
e ← any edge uv of GX .
β1 = f pV ertexCover Inner(X \ {u, v} , β + 2)
// Only take u to the cover, but then we must also take
// all the vertices that are neighbors of v,
// to cover their edges with v
β2 = f pV ertexCover Inner X \ {u} ∪ NG X (v) , β + NG X (v)

// Only take v to the cover...

β3 = f pV ertexCover Inner X \ {v} ∪ NG X (u) , β + NG X (u)

return min(β1, β2, β3 ).

algFPVertexCover (G = (V, E))

return fpVertexCoverInner (V, 0)

Figure 6.3: Fixed parameter tractable algorithm for VertexCoverMin.

Also, in the following, for a vertex v, let NG (v) denote the set of vertices of G that are adjacent to v.
Consider an edge e = uv in G. We know that either u or v (or both) must be in any vertex cover of G, so
consider the brute force algorithm for VertexCoverMin that tries all these possibilities. The resulting algorithm
algFPVertexCover is depicted in Figure 6.3.

Lemma 6.2.3. The algorithm algFPVertexCover (depicted in Figure 6.3) returns the optimal solution to the
given instance of VertexCoverMin.

Proof: It is easy to verify, that if the algorithm returns β then it found a vertex cover of size β. Since the depth
of the recursion is at most n, it follows that this algorithm always terminates.
Consider the optimal solution Y ⊆ V, and run the algorithm, where every stage of the recursion always pick
the option that complies with the optimal solution. Clearly, since in every level of the recursion at least one
vertex of Y is being found, then after O(|Y |) recursive calls, the remaining graph would have no edges, and it
would return |Y | as one of the candidate solution. Furthermore, since the algorithm always returns the minimum
solution encountered, it follows that it would return the optimal solution.

Lemma 6.2.4. The depth of the recursion of algFPVertexCover(G) is at most α, where α is the minimum size
vertex cover in G.

Proof: The idea is to consider all the vertices that can be added to the vertex cover being computed without
covering any new edge. In particular, in the case the algorithm takes both u and v to the cover, then one of
these vertices must be in the optimal solution, and this can happen at most α times.
The more interesting case, is when the algorithm picks NG X (v) (i.e., β2 ) to the vertex cover. We can add
v to the vertex cover in this case without getting any new edges being covered (again, we are doing this only
conceptually – the vertex cover computed by the algorithm would not contain v [only its neighbors]). We do
the same thing for the case of β3 .
Now, observe that in any of these cases, the hypothetical set cover being constructed (which has more vertices
than what the algorithm computes, but covers exactly the same set of edges in the original graph) contains one
vertex of the optimal solution picked into itself in each level of the recursion. Clearly, the algorithm is done
once we pick all the vertices of the optimal solution into the hypothetical vertex cover. It follows that the depth
the recursion is ≤ α.

46
Theorem 6.2.5. Let G be a graph with n vertices, and with the minimal vertex cover being of size α. Then,
the algorithm algFPVertexCover
(depicted in Figure 6.3) returns the optimal vertex cover for G and the running
time of this algorithm is O 3α n2 .

Proof: By Lemma 6.2.4, the recursion tree has depth α. As such, it contains at most 2 · 3α nodes. Each node
in the recursion requires O n work (ignoring the recursive calls), if implemented naively. Thus, the bound on
2

the running time follows.

Algorithms where the running time is of the form O(nc f (α)), where α is some parameter that depends on
the problem are fixed parameter tractable algorithms for the given problem.

6.2.2.1. Remarks
Currently, the fastest algorithm known for this problem has running time O(1.2738α + αn) [CKX10]. This
algorithm uses similar ideas, but is considerably more complicated.
 is possible for VertexCoverMin, unless P = NP. The
It is known that no better approximationthan 1.3606
currently best approximation known is 2 − Θ 1/ log n . If the Unique Games Conjecture is true, then no
p

better constant approximation is possible in polynomial time.

6.3. Approximating maximum matching

Definition 6.3.1. Consider an undirected graph G = (V, E). The graph might have a weight function ω(e),
specifying a positive value on the edges of G (if no weights are specified, treat every edge as having weight 1).
• A subset M ⊆ E is a matching if no pair of edges of M share endpoints.
• A perfect matching is a matching that covers all the vertices of G.
• A min-weight perfect matching, is the minimum weight matching among all perfect matching, where
the weight of a matching is Õ
ω(M) = ω(e).
e∈M

• The maximum-weight matching (or just maximum matching is the matching with maximum weight
among all matchings.
• A matching M is maximal if no edge can be added to it. That is, for every edge e ∈ E, we have that the
edges of M contains at least one endpoint of e.

Note the subtle difference between maximal and maximum – the first, is a local maximum, while the other one
is the global maximum.

Lemma 6.3.2. Given an undirected unweighted graph G with n vertices and m edges, one can compute a
matching M in G, such that |M | ≥ |opt| /2, where opt is the maximum size (i.e., cardinality) matching in G.
The running time is O(n + m).

Proof: The algorithm is shockingly simple – repeatedly pick an edge of G, remove it and the edges adjacent to
it, and repeat till there are no edges left in the graph. Let M be the resulting matching.
To see why this is a two approximation (i.e., 2|M | ≥ |opt|, observe that every edge of M is adjacent to at
most two edges of opt. As such, each edge of M pays for two edges of opt, which implies the claim.
One way to see that is to imagine that we start with the matching opt and let M = {m1, . . . , mt } – at each
iteration, we insert mi into the current matching, and remove any old edges that intersect it. As such, we moved
from the matching of M to the matching of opt. In each step, we deleted at most two edges, and inserted one
edges. As such, |opt| ≤ 2|M |.

47
Lemma 6.3.3. Given an undirected weighted graph G with n vertices and m edges, one can compute a matching
M in G, such that ω(M) ≥ ω(opt)/2, where opt is the maximum weight matching in G. The running time is
O(n log n + m).

Proof: We run the algorithm for the unweighted case, with the modification that we always pick the heaviest
edge still available. The same argument as in Lemma 6.3.2 implies that that this is a two approximation. As
for the running time – we need a min-heap for m elements, that performs at most n deletions, and as such, the
running time is O(n log n + m) by using a Fibonacci heap.

Remark 6.3.4. Note, that maximum matching (and all the variants mentioned above) are solvable in polynomial
time. The main thing is that the above algorithm is both simple and give us a decent starting point which can
be used in the exact algorithm.

6.4. Graph diameter

FILL IN.

6.5. Traveling Salesman Person

We remind the reader that the optimization variant of the TSP problem is the following.

TSP-Min
Instance: G = (V, E) a complete graph, and ω(e) a cost function on edges of G.
Question: The cheapest tour that visits all the vertices of G exactly once.

Theorem 6.5.1. TSP-Min can not be approximated within any factor unless NP = P.

Proof: Consider the reduction from Hamiltonian Cycle into TSP. Given a graph G, which is the input for the
Hamiltonian cycle, we transform it into an instance of TSP-Min. Specifically, we set the weight of every edge to
1 if it was present in the instance of the Hamiltonian cycle, and 2 otherwise. In the resulting complete graph,
if there is a tour price n then there is a Hamiltonian cycle in the original graph. If on the other hand, there
was no cycle in G then the cheapest TSP is of price n + 1.
Instead of 2, let us assign the missing edges in H a weight of cn, for c an arbitrary number. Let H denote
the resulting graph. Clearly, if G does not contain any Hamiltonian cycle in the original graph, then the price
of the TSP-Min in H is at least cn + 1.
Note, that the prices of tours of H are either (i) equal to n if there is a Hamiltonian cycle in G, or (ii) larger
than cn + 1 if there is no Hamiltonian cycle in G. As such, if one can do a c-approximation, in polynomial time,
to TSP-Min, then using it on H would yield a tour of price ≤ cn if a tour of price n exists. But a tour of price
≤ cn exists if and only if G has a Hamiltonian cycle.
Namely, such an approximation algorithm would solve a NP-Complete problem (i.e., Hamiltonian Cycle)
in polynomial time.

Note, that Theorem 6.5.1 implies that TSP-Min can not be approximated to within any factor. However, once
we add some assumptions to the problem, it becomes much more manageable (at least as far as approximation).
What the above reduction did, was to take a problem and reduce it into an instance where this is a huge gap,
between the optimal solution, and the second cheapest solution. Next, we argued that if had an approximation
algorithm that has ratio better than the ratio between the two endpoints of this empty interval, then the
approximation algorithm, would in polynomial time would be able to decide if there is an optimal solution.

48
6.5.1. TSP with the triangle inequality
6.5.1.1. A 2-approximation
Consider the following special case of TSP:

TSP4, -Min
Instance: G = (V, E) is a complete graph. There is also a cost function ω(·) defined over the edges of
G, that complies with the triangle inequality.
Question: The cheapest tour that visits all the vertices of G exactly once.

We remind the reader that the triangle inequality holds for ω(·) if

∀u, v, w ∈ V(G), ω(u, v) ≤ ω(u, w) + ω(w, v).

The triangle inequality implies that if we have a path σ in G, that starts at s and ends at t, then ω(st) ≤ ω(σ).
Namely, shortcutting, that is going directly from s to t, is always beneficial if the triangle inequality holds
(assuming that we do not have any reason to visit the other vertices of σ).
Definition 6.5.2. A cycle in a graph G is Eulerian if it visits every edge of G exactly once.

Unlike Hamiltonian cycle, which has to visit every vertex exactly once, an Eulerian cycle might visit a vertex
an arbitrary number of times. We need the following classical result:
Lemma 6.5.3. A graph G has a cycle that visits every edge of G exactly once (i.e., an Eulerian cycle) if and
only if G is connected, and all the vertices have even degree. Such a cycle can be computed in O(n + m) time,
where n and m are the number of vertices and edges of G, respectively.

Our purpose is to come up with a 2-approximation algorithm for TSP 4, -Min. To this end, let Copt denote
the optimal TSP tour in G. Observe that Copt is a spanning graph of G, and as such we have that
 
ω Copt ≥ weight cheapest spanning graph of G .

But the cheapest spanning graph of G, is the minimum spanning tree (MST) of G, and as such ω Copt ≥

ω(MST(G)).
The MST can be computed in O(n log n + m) = O(n2 ) time, where n is the number of vertices of G,
and m = n2 is the number of edges (since G is the complete graph). Let T denote the MST of G, and covert
T into a tour by duplicating every edge twice. Let H denote the new graph. We have that H is a connected
graph, every vertex of H has even degree, and as such H has an Eulerian tour (i.e., a tour that visits every edge
of H exactly once).
As such, let C denote the Eulerian cycle in H. Observe that

ω(C) = ω(H) = 2ω(T) = 2ω(MST(G)) ≤ 2ω Copt .

Next, we traverse C starting from any vertex v ∈ V(C). As we traverse C, we skip vertices that we already
visited, and in particular, the new tour we extract from C will visit the vertices of V(G) in the order they first
appear in C. Let π denote the new tour of G. Clearly, since we are performing shortcutting, and the triangle
inequality holds, we have that ω(π) ≤ ω(C). The resulting algorithm is depicted in Figure 6.4.
It is easy to verify, that all the steps of our algorithm can be done in polynomial time. As such, we have
the following result.
Theorem 6.5.4.
Given an instance of TSP with the triangle inequality (TSP 4, -Min) (namely, a graph G with
n vertices and n2 edges, and a cost function
ω(·) on the edges that comply with the triangle inequality), one
can compute a tour of G of length ≤ 2ω Copt , where Copt is the minimum cost TSP tour of G. The running
time of the algorithm is O n .
2

49
Random documents with unrelated
content Scribd suggests to you:
 Kathin ja Peterin tullessa oli talon isäntä poissa. Kathi pyysi silloin
heti päästä levolle, ja heti kun hän oli pannut maata oman pienen
huoneensa valkoiseen sänkyyn, vaipui hän uneen, niin syvään
uneen, ettei rajuilmakaan, joka kohta sen jälkeen puhkesi, voinut
herättää häntä.

Täti ja Peter istuivat huoneeseen, joka oli hänen huoneensa

vieressä, avonaisen oven luokse, ollaksensa siinä jos hän heräisi
jyrinästä.

Rajuilman yltyessä tuli hän hetkiseksi levottomaksi ja nousi

puoleksi istualleen. Mutta samassa alkoivat kirkonkellot soida — ja
hän vaipui takaisin tyynylleen syvään ja rauhalliseen uneen,
hengittäen levollisesti kuin lapsi.

"Voin lyödä vetoa ettei hän ole nukkunut tällä tavoin ainoanakaan
yönä siellä vuorilla", sanoi Peter. "Ei ollut mitään järkeä siinä, että
hänen piti lähteä kotoaan."

Räiskyvä, paukkuva ukkosen jyrinä, joka seurasi häikäisevää

salamaa, tuntui seuraavassa silmänräpäyksessä tärisyttävän taloa
perustuksiaan myöten.

"Jumalan pyhä äiti!" kuiskasi täti.

"Se iski johonkin", sanoi Peter.

Samassa soitto vaikeni — ja pahin rajuilma oli ohitse.

Yhdeksän ja kymmenen tienoissa, kun kaikki oli ohitse, antoi täti

väen mennä levolle. Mutta hän ja Peter jäivät odottamaan Kassiania,
jonka viipyminen teki hänet levottomaksi.
 Kun hän saapui — niin kalpeana ja väsyneenä kauheasta
jumalanilmasta, että hän oli oikein vieraan näköinen — ja hyvin
lyhyesti oli heille ilmoittanut järisyttävän tapahtuman, kesti kauan,
ennenkuin kukaan heistä lähti levolle.

Kassian ei hämmästynyt nähdessään Peterin täällä, eikä

kuullessaan, että Kathi nukkui huoneessaan. Oli olikeastaan kuin ei
hän olisi nähnyt eikä kuullut mitään. Ja kauan vielä sen jälkeeen kun
hän oli paneutunut vuoteeseensa, kuuli täti hänen astuntansa hänen
omasta huoneestaan. Edestakaisin — edestakaisin…

— Aina aikaisesta aamusta kokoontuivat kaikki goldrainilaiset

kirkkomaan takana olevan matalan talon edustalle, jonka seinustella
kasvoi kukkivia viiniköynnöksiä. Ja hyvin hiljaa he kaikki astuivat
sisään, toinen toisensa perässä, ja näkivät hänen siinä makaavan
valkoisella vuoteella mustassa papinpuvussaan — hiuksien
muodostaessa kuin sädekehän kalpeitten kasvojen ympärille, jotka
olivat niin lempeät ja kirkkaat…

"Hän on sellaisen näköinen, jonka sydämen toivo on toteutunut",

sanoi itkevä Marianne. Hän oli valvonut yön kuolleen luona parin
seurakunnan miehen kanssa ja tahtoi olla siinä koko päivän, niin
kauan kun hän jaksoi pysyä pystyssä. Keskiyön tienoilla tuli sitten
pari luostariveljeä suorittamaan kuolinvalvomisen. Ja sen huoneen
ovea pieneen puutarhaan ei lukittaisikaan koko yönä. Hän rakasti
pienten kappelien öistä valoa; nyt tuli valon loistaa hänen omasta
huoneestaan, ja jos matkustavainen kulki ohitse ja katsoi sisään,
kohtasivat häntä hiljaiset, valoisat kasvot, jotka ikäänkuin
toivottaisivat "Jumalan rauhaa."

Kaikki toivat parasta mitä tiesivät. Toiset toivat tuoreita kukkia,

toiset suuria helmiseppeleitä, muutamat ruukkukasvinsa
akkunoiltaan, punaisia neilikoita ja pelargonioita, jotka he asettivat
hänen huoneeseensa. Toisilla oli vahakynttilöitä, toiset ripottelivat
oksia ja lehtiä.
 — Kassian Gamperilla kulki täti paremmassa puvussaan leikellen
kukkia kaikista ruukkukasveista. Hän menisi Peterin kanssa, joka sen
jälkeen taas aikoi lähteä vuorille. Vanhukset eivät voineet olla niin
kauan ilman häntä. Hän oli puhunut hiukan Kassianin kanssa, joka
suostui kaikkiin hänen toivomuksiinsa — ja istuessaan katsoi
järkähtämättä hänen ohitseen kauas etäisyyteen — eikä hän voinut
odottaa saavansa tavata Kathia, koska hän yhä nukkui.

"Ei, hän on juuri herännyt", sanoi täti, jonka poskia pitkin valuivat
kyyneleet, "mutta luulen olevan parasta, ettet mene hänen luokseen.
Hän nukkui niin rauhallisesti käsi pään ympäri taivutettuna… Mutta
sitten luulen hänen heränneen kirkon kellojen soitosta. Hän makaa
silmät raollaan kuin sellainen, joka ei tahdo ajatella, vaan on vain
iloinen — niinkuin lapsi jouluaamuna. — Onko pyhä, täti? kysyi hän.
— Ei, sanon minä, etkö voi kuulla, että joku on kuollut! — Hän
kohoutuu vuoteessaan ja hänen silmänsä sumenevat. — Kuka? kysyy
hän. Ja minä sanon: eräs, josta kaikki olemme pitäneet! Silloin hän
katselee minua kuin se, joka rukoilee elämänsä puolesta. — Ei, ei
isä, sanon minä, vaikka hänkin oli lähellä kuolemaa; vaan pappi, joka
soitti vanhan Matthiaksen puolesta, ja häneen löi salama. Isä löysi
hänet sieltä. — Sitten hän laskeutuu makuulleen ja kääntyy
seinäänpäin. Luulen, että hänen tulee olla ihan yksinään. Pastori
neuvoi häntä niin kauniisti aikoinaan. Ja nyt kaikki herää hänessä
uudelleen — se voi olla kylliksi vaikeata hänelle, pikku-raukalle!"

— Kun täti saapui kotiin taas tunnin kuluttua, istuutui hän Kathin
luokse, joka yhä makasi kasvot seinäänpäin, ja alkoi ihan hiljaa
kertoa hänelle miten kaunista siellä oli ollut ja miten kaikki olivat
tuoneet, mikä mitäkin muassaan tehdäkseen kaiken hänen
ympärillään juhlalliseksi. Kirkkoherra oli itse lukenut kuolinrukoukset
— itku kurkussa… Eikö Kathi tahtonut nousta ja lähteä sinne? Hän oli
näöltään kuin pyhimys!

Mutta Kathi vain viittasi kädellään kieltävästi ja jäi makaamaan

tahtomatta syödä tai juoda.

Silloin kysyi läti Kassianilta, eikö hän tahtonut lähteä sinne? Mutta
hänhän oli siellä eilen illalla, vastasi hän. Oli mukana kantamassa
hänet kotiin — — Ja hänen kyllä tuli pysyä levossa tänään, sillä joka
hetki hän tunsi pyörrytystä.

*****

Mutta kun kaikki talossa olivat menneet levolle, alkoi Kassian

Gamper kävellä edestakaisin niinkuin viime yönäkin — ei
huoneessaan, vaan käytävässä ulko-oven ja yläkertaan vieväin
portaitten välissä. Hän kulki edestakaisin kädet selkänsä takana
pienen lampun valossa, joka riippui katossa — hänen ajatuksiensa
liikkuessa vain yhdessä asiassa.

"Ennen keskiyötä, mutta myöhään, kun kaikki on hiljaista.

Saamme nyt nähdä — —"

Hän oli varma, ihan varma siitä. Nyt tulisi suuri tilinteko.

Viimeisinä kahtenakymmenenäneljänä tuntina, jotka olivat olleet

pitkät kuin vuodet, oli hän yhä uudestaan ajatellut sitä mitä hän
tunsi, kun kuolema oli tarttunut häneen — ja hän kyllä tiesi, että
suurimman tilinteon pitää ihminen itsensä kanssa. Ainoastaan siinä
voi hän oikein tuomita syyllisen.

Mutta tässähän hänellä oli isän-oikeus — niin, isän velvollisuus. Ja

oikeuden tulee vallita. Siitä ei kukaan saa uskaltaa poiketa. Sen tulee
vallita.

Kirkontornin kello löi yksitoista, raskaasti ja voimakkaasti — niin

kuin se olisi myöntänyt sen oikeaksi.

Niin, sen tornin kello — — sekin voisi kertoa oikeuden voimasta. —

—

Syntyi taas yöllinen hiljaisuus.

Sitten aukaistiin ylhäällä ovi — ihan hiljaa. Kassian pysähtyi ja

asettui portaitten varjoon.

Portaita alas kulkivat hiipivät askeleet. Porras narahti — ja ne

pysähtyivät.

Hän ihmetteli että niin pieni kolina saattoi pelästyttää kulkijaa ja

hänen sydämensä takoi, takoi aivan kuuluvasti.

Askeleet hiipivät eteenpäin — saapuivat käytävään… ja silloin —

seisoi hän siinä.

Huusiko hän? Oliko joku huutanut? Ei, ei ainakaan hän! — Sillä

hän vain seisoi — niinkuin lapsi, joka tietää ettei se voi puolustautua,
liikkumattomana, kietoutuneena suureen, mustaan huiviin, joka oli
heitetty pään ylt ja jota hän piti tiukasti ympärilleen käärittynä. Hän
seisoi niin liikkumattomana, että silmäluomet, jotka raskaasti olivat
painuneet silmien yli, keltaisenvalkoisina ja vähäsen turvonneina,
näyttivät kuin vahaan leikatuilta.

Miksei hän mitään puhunut? Hän olisi voinut puhua — sanoa

jotain, joka selittäisi kaiken… Sitä kai ei ollut niin vaikea keksiä! Miksi
hän seisoi vain kuin lapsi, joka ei voi puolustautua. Silloin piti
Kassianin puhua — hänenhän täytyi…

Mutta hänen tuli alottaa varovasti… muuten salpaisi hänen

henkensä se suuri sana, joka nyt oli sanottava.

"Oletko se sinä? Sinäkö siinä olet, Kathi?" Hänen täytyi keskeyttää

kunnes sydän hiukan rauhoittuisi.

"Ja tahdot lähteä ulos, niinkö? — — sillä — olet ajatellut että voisit
mennä sinne?"

Hänen huulensa liikkuivat kuiskaten myöntäväsi —

kuulumattomasti.

"Ajattelit heti, että sinäkin tahdot mennä — niikuin kaikki

muutkin."

Taas kuulumaton myönnytys.

"Mutta — kun kuulit, miten kaikki sinne kokoontuivat — tuntui

sinusta niin oudolta tavata heitä kaikkia yhdellä kertaa — — Eikö
totta? Nyt — kun olet ollut niin kauan poissa! Silloin odotit iltaa — ja
tiesit, ettei siellä ollut enää ketään."

Hänen huulensa liikkuivat taas myönnytykseksi:

"Mutta nyt tahdot sinäkin mennä? Sehän on ihan luonnollista!"

Mitä tämä olikaan? Eikö hän sanonut kaikkea mitä toinen olisi
voinut, mitä hänen olisi pitänyt keksiä sanoakseen! Mitä tämä
olikaan? Tahtoiko hän ensin rauhoittaa häntä — ja sitten vasta sanoa
sanottavansa?
 Äkkiä hänen kasvonsa kalpenivat. Hänen katseensa oli kohdannut
paikan, jossa hän piteli käsiään — huivin alla.

"Hän vie ne muassaan", ajatteli hän, "hän vie ne todellakin

muassaan! Hän pitää niitä käsissään — hän tahtoo panna ne sinne,
jotta ne pantaisiin mukaan — — ei, ei"…

Raskaana ja suurena laskeutui hänen kätensä tytön kädelle, työnsi

huivin syrjään — ja veti esiin pitkät, kokoonkäärityt palmikot, jotka
olivat hänen käsissään.

Ei, hän ei huutanut… mutta silmäluomet kohoutuivat, tuskallisesti

väristen — ja hänen silmänsä kohtasivat hänen silmänsä, silmät
jotka olivat kuin Aloisian…

Hän tarttui raskaisiin palmikkoihin, punnitsi — — niitä

tahtomattaan samalla kädellään — vanhan tapansa mukaan.

Mutta siinä liikkeessä oli jotain, joka kääntyi häntä itseään vastaan
— tarttui hänen sydämeensä…

"No niin — — olet myös kuullut, miten kaikki sydämensä

kiitollisuudesta ovat vieneet yksi yhtä ja toinen taas toista muassaan
sinne… parasta mitä saattoivat ajatella… Niin et sinäkään tahtonut
mennä tyhjin käsin? Sinäkin tahdoit — koko sydämestäsi — — Ja niin
sinä otit — sillä sinullahan ei ole muuta, Kathi"…

"Ei, minulla ei ole muuta."

Kassian päästi palmikot kädestään.

"Mutta älä nyt ole liian kauan poissa, Kathi. On myöhäistä"…

 Tuntui kuin olisi tytär tahtonut rauhoittaa isäänsä sanoessaan:
"Olen kyllä taas pian täällä."

"Niin, sillä Peter käski minun pitää sinusta hyvää huolta. Hän sanoi
nimenomaan että pitäisin hyvää huolta sinusta… Ja hän uskoo
varmasti että sinusta tulee hänelle hyvä vaimo, reipas ja uskollinen
ja hyvä vaimo."

Kathi ei nostanut silmiänsä, mutta sanoi äkkiä äänekkäästi — kuin

se joka puhuu unissaan eikä voi hillitä ääntään — ja lujasti kuin olisi
hän antanut lupauksen: "Kyllä."

Kassian hengitti syvään — hän tiesi voivansa siihen luottaa, ja

hetkisen seisoivat molemmat äänettöminä.

Sitten Kassian sanoi vain: "Niin — mene siis. Jumala kanssasi,

Kathi… Jumala kanssasi! — Annan lampun palaa niin kauan."

Kathi meni hänen ohitsensa — ulos ovesta ja sulki sen jälkeensä.

Hän oli yksin käytävässä.

Miten oli? oliko hän sallinut hänen lähteä? Lähteä sinne… Oi, ei, ei!
Ei ikinä! Eikä hän näin pääsisi asiasta! Nyt oli hän hänen käsissään —
tänä iltana vihdoinkin tuli puhua! Oli tapahtuva suuri, suuri tilinteko…

Hän kiskaisi oven selälleen, juoksi ulos pimeään ja riensi hänen

jälkeensä.

Tyttö kuuli hänen tulevan ja pysähtyi, kääntyi ja nosti kasvonsa

häntä kohti niinkuin äsken — käytävän ovesta valuvan valon
liekuttavassa loisteessa… Ja hänen silmänsä olivat kuin Aloisian…

Kassian viittasi hänelle kädellään.

 "Niin — mene — Jumala kanssasi, Kathi", toisti hän
soinnuttomasti. —
"Jumala kanssasi!" — —

Mutta Kathi jäi paikoilleen. Ja hänestä tyttö äkkiä kasvoi — tuli niin
suureksi öisessä pimeydessä.

Ja hän katseli häntä kuin sellainen, jonka suru on tehnyt

täyskasvuiseksi, kuin sellainen, joka on luopunut kaikesta — ja jolla
senvuoksi ei ole enää mitään peljättävää.

Sitten nosti hän kätensä, jossa hänellä oli pitkät palmikot. "Vien ne
jokeen", sanoi hän. "Sitten ei ole enää mitään jäljellä Goldrainin
Kathista" — —

Hän oli vaiti hetkisen.

"Ja sitten tahdon mennä sinne — rukoilemaan… Sillä silloinhan

olen vain eräs, joka on kuollut kaukana!"

Hän viittasi hänelle kädellään — mutta tällä kertaa ei hän voinut

sanoa mitään.

— Hetki sen jälkeen hävisivät hänen askeleensa pimeään

hijaisuuteen…

End of Project Gutenberg's Kaspar Zinglerin sydän, by Ingeborg

Maria Sick
*** END OF THE PROJECT GUTENBERG EBOOK KASPAR ZINGLERIN
SYDÄN ***

Updated editions will replace the previous one—the old editions will
be renamed.

Creating the works from print editions not protected by U.S.

copyright law means that no one owns a United States copyright in
these works, so the Foundation (and you!) can copy and distribute it
in the United States without permission and without paying
copyright royalties. Special rules, set forth in the General Terms of
Use part of this license, apply to copying and distributing Project
Gutenberg™ electronic works to protect the PROJECT GUTENBERG™
concept and trademark. Project Gutenberg is a registered trademark,
and may not be used if you charge for an eBook, except by following
the terms of the trademark license, including paying royalties for use
of the Project Gutenberg trademark. If you do not charge anything
for copies of this eBook, complying with the trademark license is
very easy. You may use this eBook for nearly any purpose such as
creation of derivative works, reports, performances and research.
Project Gutenberg eBooks may be modified and printed and given
away—you may do practically ANYTHING in the United States with
eBooks not protected by U.S. copyright law. Redistribution is subject
to the trademark license, especially commercial redistribution.

START: FULL LICENSE

THE FULL PROJECT GUTENBERG LICENSE
 PLEASE READ THIS BEFORE YOU DISTRIBUTE OR USE THIS WORK

To protect the Project Gutenberg™ mission of promoting the free

distribution of electronic works, by using or distributing this work (or
any other work associated in any way with the phrase “Project
Gutenberg”), you agree to comply with all the terms of the Full
Project Gutenberg™ License available with this file or online at
www.gutenberg.org/license.

Section 1. General Terms of Use and

Redistributing Project Gutenberg™
electronic works
1.A. By reading or using any part of this Project Gutenberg™
electronic work, you indicate that you have read, understand, agree
to and accept all the terms of this license and intellectual property
(trademark/copyright) agreement. If you do not agree to abide by all
the terms of this agreement, you must cease using and return or
destroy all copies of Project Gutenberg™ electronic works in your
possession. If you paid a fee for obtaining a copy of or access to a
Project Gutenberg™ electronic work and you do not agree to be
bound by the terms of this agreement, you may obtain a refund
from the person or entity to whom you paid the fee as set forth in
paragraph 1.E.8.

1.B. “Project Gutenberg” is a registered trademark. It may only be

used on or associated in any way with an electronic work by people
who agree to be bound by the terms of this agreement. There are a
few things that you can do with most Project Gutenberg™ electronic
works even without complying with the full terms of this agreement.
See paragraph 1.C below. There are a lot of things you can do with
Project Gutenberg™ electronic works if you follow the terms of this
agreement and help preserve free future access to Project
Gutenberg™ electronic works. See paragraph 1.E below.
1.C. The Project Gutenberg Literary Archive Foundation (“the
Foundation” or PGLAF), owns a compilation copyright in the
collection of Project Gutenberg™ electronic works. Nearly all the
individual works in the collection are in the public domain in the
United States. If an individual work is unprotected by copyright law
in the United States and you are located in the United States, we do
not claim a right to prevent you from copying, distributing,
performing, displaying or creating derivative works based on the
work as long as all references to Project Gutenberg are removed. Of
course, we hope that you will support the Project Gutenberg™
mission of promoting free access to electronic works by freely
sharing Project Gutenberg™ works in compliance with the terms of
this agreement for keeping the Project Gutenberg™ name associated
with the work. You can easily comply with the terms of this
agreement by keeping this work in the same format with its attached
full Project Gutenberg™ License when you share it without charge
with others.

1.D. The copyright laws of the place where you are located also
govern what you can do with this work. Copyright laws in most
countries are in a constant state of change. If you are outside the
United States, check the laws of your country in addition to the
terms of this agreement before downloading, copying, displaying,
performing, distributing or creating derivative works based on this
work or any other Project Gutenberg™ work. The Foundation makes
no representations concerning the copyright status of any work in
any country other than the United States.

1.E. Unless you have removed all references to Project Gutenberg:

1.E.1. The following sentence, with active links to, or other

immediate access to, the full Project Gutenberg™ License must
appear prominently whenever any copy of a Project Gutenberg™
work (any work on which the phrase “Project Gutenberg” appears,
or with which the phrase “Project Gutenberg” is associated) is
accessed, displayed, performed, viewed, copied or distributed:
 This eBook is for the use of anyone anywhere in the United
States and most other parts of the world at no cost and with
almost no restrictions whatsoever. You may copy it, give it away
or re-use it under the terms of the Project Gutenberg License
included with this eBook or online at www.gutenberg.org. If you
are not located in the United States, you will have to check the
laws of the country where you are located before using this
eBook.

1.E.2. If an individual Project Gutenberg™ electronic work is derived

from texts not protected by U.S. copyright law (does not contain a
notice indicating that it is posted with permission of the copyright
holder), the work can be copied and distributed to anyone in the
United States without paying any fees or charges. If you are
redistributing or providing access to a work with the phrase “Project
Gutenberg” associated with or appearing on the work, you must
comply either with the requirements of paragraphs 1.E.1 through
1.E.7 or obtain permission for the use of the work and the Project
Gutenberg™ trademark as set forth in paragraphs 1.E.8 or 1.E.9.

1.E.3. If an individual Project Gutenberg™ electronic work is posted

with the permission of the copyright holder, your use and distribution
must comply with both paragraphs 1.E.1 through 1.E.7 and any
additional terms imposed by the copyright holder. Additional terms
will be linked to the Project Gutenberg™ License for all works posted
with the permission of the copyright holder found at the beginning
of this work.

1.E.4. Do not unlink or detach or remove the full Project

Gutenberg™ License terms from this work, or any files containing a
part of this work or any other work associated with Project
Gutenberg™.

1.E.5. Do not copy, display, perform, distribute or redistribute this

electronic work, or any part of this electronic work, without
prominently displaying the sentence set forth in paragraph 1.E.1
with active links or immediate access to the full terms of the Project
Gutenberg™ License.

1.E.6. You may convert to and distribute this work in any binary,
compressed, marked up, nonproprietary or proprietary form,
including any word processing or hypertext form. However, if you
provide access to or distribute copies of a Project Gutenberg™ work
in a format other than “Plain Vanilla ASCII” or other format used in
the official version posted on the official Project Gutenberg™ website
(www.gutenberg.org), you must, at no additional cost, fee or
expense to the user, provide a copy, a means of exporting a copy, or
a means of obtaining a copy upon request, of the work in its original
“Plain Vanilla ASCII” or other form. Any alternate format must
include the full Project Gutenberg™ License as specified in
paragraph 1.E.1.

1.E.7. Do not charge a fee for access to, viewing, displaying,

performing, copying or distributing any Project Gutenberg™ works
unless you comply with paragraph 1.E.8 or 1.E.9.

1.E.8. You may charge a reasonable fee for copies of or providing

access to or distributing Project Gutenberg™ electronic works
provided that:

• You pay a royalty fee of 20% of the gross profits you derive
from the use of Project Gutenberg™ works calculated using the
method you already use to calculate your applicable taxes. The
fee is owed to the owner of the Project Gutenberg™ trademark,
but he has agreed to donate royalties under this paragraph to
the Project Gutenberg Literary Archive Foundation. Royalty
payments must be paid within 60 days following each date on
which you prepare (or are legally required to prepare) your
periodic tax returns. Royalty payments should be clearly marked
as such and sent to the Project Gutenberg Literary Archive
Foundation at the address specified in Section 4, “Information
 about donations to the Project Gutenberg Literary Archive
Foundation.”

• You provide a full refund of any money paid by a user who

notifies you in writing (or by e-mail) within 30 days of receipt
that s/he does not agree to the terms of the full Project
Gutenberg™ License. You must require such a user to return or
destroy all copies of the works possessed in a physical medium
and discontinue all use of and all access to other copies of
Project Gutenberg™ works.

• You provide, in accordance with paragraph 1.F.3, a full refund of

any money paid for a work or a replacement copy, if a defect in
the electronic work is discovered and reported to you within 90
days of receipt of the work.

• You comply with all other terms of this agreement for free
distribution of Project Gutenberg™ works.

1.E.9. If you wish to charge a fee or distribute a Project Gutenberg™

electronic work or group of works on different terms than are set
forth in this agreement, you must obtain permission in writing from
the Project Gutenberg Literary Archive Foundation, the manager of
the Project Gutenberg™ trademark. Contact the Foundation as set
forth in Section 3 below.

1.F.

1.F.1. Project Gutenberg volunteers and employees expend

considerable effort to identify, do copyright research on, transcribe
and proofread works not protected by U.S. copyright law in creating
the Project Gutenberg™ collection. Despite these efforts, Project
Gutenberg™ electronic works, and the medium on which they may
be stored, may contain “Defects,” such as, but not limited to,
incomplete, inaccurate or corrupt data, transcription errors, a
copyright or other intellectual property infringement, a defective or
damaged disk or other medium, a computer virus, or computer
codes that damage or cannot be read by your equipment.

1.F.2. LIMITED WARRANTY, DISCLAIMER OF DAMAGES - Except for

the “Right of Replacement or Refund” described in paragraph 1.F.3,
the Project Gutenberg Literary Archive Foundation, the owner of the
Project Gutenberg™ trademark, and any other party distributing a
Project Gutenberg™ electronic work under this agreement, disclaim
all liability to you for damages, costs and expenses, including legal
fees. YOU AGREE THAT YOU HAVE NO REMEDIES FOR
NEGLIGENCE, STRICT LIABILITY, BREACH OF WARRANTY OR
BREACH OF CONTRACT EXCEPT THOSE PROVIDED IN PARAGRAPH
1.F.3. YOU AGREE THAT THE FOUNDATION, THE TRADEMARK
OWNER, AND ANY DISTRIBUTOR UNDER THIS AGREEMENT WILL
NOT BE LIABLE TO YOU FOR ACTUAL, DIRECT, INDIRECT,
CONSEQUENTIAL, PUNITIVE OR INCIDENTAL DAMAGES EVEN IF
YOU GIVE NOTICE OF THE POSSIBILITY OF SUCH DAMAGE.

1.F.3. LIMITED RIGHT OF REPLACEMENT OR REFUND - If you

discover a defect in this electronic work within 90 days of receiving
it, you can receive a refund of the money (if any) you paid for it by
sending a written explanation to the person you received the work
from. If you received the work on a physical medium, you must
return the medium with your written explanation. The person or
entity that provided you with the defective work may elect to provide
a replacement copy in lieu of a refund. If you received the work
electronically, the person or entity providing it to you may choose to
give you a second opportunity to receive the work electronically in
lieu of a refund. If the second copy is also defective, you may
demand a refund in writing without further opportunities to fix the
problem.

1.F.4. Except for the limited right of replacement or refund set forth
in paragraph 1.F.3, this work is provided to you ‘AS-IS’, WITH NO
OTHER WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO WARRANTIES OF
MERCHANTABILITY OR FITNESS FOR ANY PURPOSE.

1.F.5. Some states do not allow disclaimers of certain implied

warranties or the exclusion or limitation of certain types of damages.
If any disclaimer or limitation set forth in this agreement violates the
law of the state applicable to this agreement, the agreement shall be
interpreted to make the maximum disclaimer or limitation permitted
by the applicable state law. The invalidity or unenforceability of any
provision of this agreement shall not void the remaining provisions.

1.F.6. INDEMNITY - You agree to indemnify and hold the Foundation,

the trademark owner, any agent or employee of the Foundation,
anyone providing copies of Project Gutenberg™ electronic works in
accordance with this agreement, and any volunteers associated with
the production, promotion and distribution of Project Gutenberg™
electronic works, harmless from all liability, costs and expenses,
including legal fees, that arise directly or indirectly from any of the
following which you do or cause to occur: (a) distribution of this or
any Project Gutenberg™ work, (b) alteration, modification, or
additions or deletions to any Project Gutenberg™ work, and (c) any
Defect you cause.

Section 2. Information about the Mission

of Project Gutenberg™
Project Gutenberg™ is synonymous with the free distribution of
electronic works in formats readable by the widest variety of
computers including obsolete, old, middle-aged and new computers.
It exists because of the efforts of hundreds of volunteers and
donations from people in all walks of life.

Volunteers and financial support to provide volunteers with the

assistance they need are critical to reaching Project Gutenberg™’s
goals and ensuring that the Project Gutenberg™ collection will
remain freely available for generations to come. In 2001, the Project
Gutenberg Literary Archive Foundation was created to provide a
secure and permanent future for Project Gutenberg™ and future
generations. To learn more about the Project Gutenberg Literary
Archive Foundation and how your efforts and donations can help,
see Sections 3 and 4 and the Foundation information page at
www.gutenberg.org.

Section 3. Information about the Project

Gutenberg Literary Archive Foundation
The Project Gutenberg Literary Archive Foundation is a non-profit
501(c)(3) educational corporation organized under the laws of the
state of Mississippi and granted tax exempt status by the Internal
Revenue Service. The Foundation’s EIN or federal tax identification
number is 64-6221541. Contributions to the Project Gutenberg
Literary Archive Foundation are tax deductible to the full extent
permitted by U.S. federal laws and your state’s laws.

The Foundation’s business office is located at 809 North 1500 West,

Salt Lake City, UT 84116, (801) 596-1887. Email contact links and up
to date contact information can be found at the Foundation’s website
and official page at www.gutenberg.org/contact

Section 4. Information about Donations to

the Project Gutenberg Literary Archive
Foundation
Project Gutenberg™ depends upon and cannot survive without
widespread public support and donations to carry out its mission of
increasing the number of public domain and licensed works that can
be freely distributed in machine-readable form accessible by the
widest array of equipment including outdated equipment. Many
small donations ($1 to $5,000) are particularly important to
maintaining tax exempt status with the IRS.

The Foundation is committed to complying with the laws regulating

charities and charitable donations in all 50 states of the United
States. Compliance requirements are not uniform and it takes a
considerable effort, much paperwork and many fees to meet and
keep up with these requirements. We do not solicit donations in
locations where we have not received written confirmation of
compliance. To SEND DONATIONS or determine the status of
compliance for any particular state visit www.gutenberg.org/donate.

While we cannot and do not solicit contributions from states where

we have not met the solicitation requirements, we know of no
prohibition against accepting unsolicited donations from donors in
such states who approach us with offers to donate.

International donations are gratefully accepted, but we cannot make

any statements concerning tax treatment of donations received from
outside the United States. U.S. laws alone swamp our small staff.

Please check the Project Gutenberg web pages for current donation
methods and addresses. Donations are accepted in a number of
other ways including checks, online payments and credit card
donations. To donate, please visit: www.gutenberg.org/donate.

Section 5. General Information About

Project Gutenberg™ electronic works
Professor Michael S. Hart was the originator of the Project
Gutenberg™ concept of a library of electronic works that could be
freely shared with anyone. For forty years, he produced and
distributed Project Gutenberg™ eBooks with only a loose network of
volunteer support.
Project Gutenberg™ eBooks are often created from several printed
editions, all of which are confirmed as not protected by copyright in
the U.S. unless a copyright notice is included. Thus, we do not
necessarily keep eBooks in compliance with any particular paper
edition.

Most people start at our website which has the main PG search
facility: www.gutenberg.org.

This website includes information about Project Gutenberg™,

including how to make donations to the Project Gutenberg Literary
Archive Foundation, how to help produce our new eBooks, and how
to subscribe to our email newsletter to hear about new eBooks.
Welcome to our website – the perfect destination for book lovers and
knowledge seekers. We believe that every book holds a new world,
offering opportunities for learning, discovery, and personal growth.
That’s why we are dedicated to bringing you a diverse collection of
books, ranging from classic literature and specialized publications to
self-development guides and children's books.

More than just a book-buying platform, we strive to be a bridge

connecting you with timeless cultural and intellectual values. With an
elegant, user-friendly interface and a smart search system, you can
quickly find the books that best suit your interests. Additionally,
our special promotions and home delivery services help you save time
and fully enjoy the joy of reading.

Join us on a journey of knowledge exploration, passion nurturing, and

personal growth every day!

ebookbell.com