1. RAM Model..............................................................

18
Introduction.......................................................... 1
8
Pseudo-Code........................................................... 2
1
Example: n! - Pseudo-Code Versus Machine Code...................
.. 22
Motivation for Asymptotic Time Complexity............................. 2
3
Analyzing Algorithms with Asymptotic Notation......................... 2
4
The 0 Notation..................................................
.. 24
Example: Bubble Sort............................................
.. 25
Example: Run-Length Codes.......................................
.. 26
Example: Horner's Method for Polynomial Evaluation..............
.. 27
Example: Matrix Multiplication..................................
.. 28
Example: Pascal's Triangle of Binomial Coefficients.............
.. 29
(*) Example: Solving Sets of Linear Equations...................
.. 31
(*) Example: Lagrange Interpolation of Polynomials..............
.. 33
Logarithms and Exponentials........................................... 3
6
(*) Non-Integer Logarithms and Exponentials.....................
.. 37
Logarithms and Exponentials Versus Polynomials..................
.. 38
Example: Binary Search..........................................
.. 39
Example: Binary Numbers.........................................
.. 40
(*) Representing Arrays............................................... 4
1
The Significance of Asymptotic Complexity............................. 4
2
Basic Approach to Algorithm Design.................................... 4
3
Sample Exercises...................................................... 4
4
Exercises............................................................. 5
9
Chapter Notes......................................................... 7
1
2. Lists.................................................................. 75
Introduction.......................................................... 7
5
Array Implementation of Stacks........................................ 7
7
Example: Using a Stack to Reverse an Array......................
.. 78
(*) Example: Evaluating a Postfix Expression with a Stack.......
.. 79
Array Implementation of Queues ("Circular Queues").................... 8
0

Example: Using a Queue to Separate an Array.....................

.. 81
General Purpose Lists................................................. 8
2
Example: Using the Basic List Operations........................
.. 83
Representing Lists With Pointers...................................... 8
4
Pointer Variables...............................................
.. 85
(*) Implementing Pointers.......................................
.. 86
Implementation of the Basic List Operations in 0(1) Time.............. 8
7
Example: Details of the INSERT Operation........................
.. 88
Example: Details ofthe DELETE Operation.........................
.. 89
Example: Reversing a Singly Linked List.........................
.. 90
(*) Example: The POP(i) Stack Operation.........................
.. 91
Sample Exercises...................................................... 9
2
Exercises............................................................. 9
7
Chapter Notes.........................................................10
1
3. Induction and Recursion................................................102
Introduction..........................................................10
2
Example: Binary Search..........................................
..104
Example: Traversing Linked Lists................................
..105
Example: Fast Computation of Integer Exponents..................
..106
(*) Example: Converting a String to a Number....................
..107
(*) Example: Evaluating a Prefix Expression.....................
..108
(*) Example: Converting Infix to Prefix or Postfix..............
..109
Proof by Induction....................................................11
0
Example: Summing Powers of 2....................................
..110
Example: Summing Odd Integers...................................
..111
Example: Correctness of Binary Search...........................
..112
Example: Towers of Hanoi Puzzle.................................
..113
Elimination of Recursion..............................................11
4
Example: Eliminating Recursion from n...........................
..115
Example: Complexity of Recursive n..............................
..116
Example: Eliminating Recursion from Towers of Hanoi.............

..117
Example: Complexity of TOWER....................................
..118
Example: Non-Recursive Towers of Hanoi Algorithm................
..119
Sample Exercises......................................................12
0
Exercises.............................................................13
2
Chapter Notes.........................................................14
0
4. Trees..................................................................141
Introduction..........................................................14
1
Tree Terms............................................................14
2
Representing Trees....................................................14
3
Pre-Order Traversal...................................................14
4
Example: Height of a Vertex v...................................
..144
Level-Order Traversal.................................................14
5
Example: MIN-HEIGHT of a Vertex v...............................
..145
Binary Search Trees...................................................14
6
Basic Binary Search Tree Operations.............................
..147
Details of the DELETEMIN Operation..............................
..148
Example: Some Sample Binary Search Tree Manipulations...........
..149
In-Order Traversal of a Binary Search Tree......................
..150
Example: Evaluating An Arithmetic Expression....................
..151
(*) Joining and Splitting Binary Search Trees...................
..152
Indexing a Binary Search Tree...................................
..154
(*) Binary Search Tree Ordering Lemma...........................
..155
(*) Average Time to Build a Binary Search Tree..................
..156
The Rotation Operation for Binary Search Trees........................15
8
Self-Adjusting Binary Search Trees....................................16
0
(*) Amortized Complexity of Self-Adjusting Binary Search Trees..
..162
Example: Tree Sort..............................................
..163
Joining and Splitting Self-Adjusting Binary Search Trees........
..164
Sample Exercises......................................................16
5
Exercises.............................................................17
0

Chapter Notes.........................................................17
4
5. Algorithm Design.......................................................175
Introduction..........................................................17
5
Divide and Conquer....................................................17
7
Example: Merge Sort.............................................
..178
Example: Quick Sort.............................................
..180
Example: Finding the /(h Largest Element........................
..181
Example: Polynomial Multiplication..............................
..182
Example: Strassen's Algorithm for Matrix Multiplication.........
..183
Divide and Conquer Recurrence Relations...............................18
4
Dynamic Programming...................................................18
5
Example: The Knapsack Problem...................................
..185
Example: The Paragraphing Problem...............................
..186
Example: Optimal Ordering of Matrix Multiplications.............
..187
Example: Context-Free Language Recognition......................
..188
Dynamic Programming Sums..............................................18
9
Randomized Algorithms.................................................19
0
Example: Statistical Sampling...................................
..190
Example: Randomized Quick Sort..................................
..191
Example: Randomized kth Largest Element.........................
..192
Greedy Algorithms.....................................................19
4
Example: Bin Packing............................................
..194
Example: HuffInan Trees.........................................
..195
Example: Shortest Common Superstrings...........................
..196
Simulated Annealing...................................................19
7
Exercises.............................................................19
8
Chapter Notes.........................................................21
1
6. Hashing................................................................216
Introduction..........................................................21
6
Basic Hashing Algorithm...............................................21
7
Hash Functions for Data Items with Many Bits..........................21
8

Complexity of Hashing.................................................21
9
The Constant e........................................................22
0
Expected Number of Empty Buckets......................................22
1
Chernoff Bound........................................................22
2
Size of the Largest Bucket............................................22
3
Overfilling a Hash Table..............................................22
5
Resizing a Hash Table.................................................22
6
Universal Hashing.....................................................22
7
Twin Hashing..........................................................22
8
Bloom Filters.........................................................22
9
Exercises.............................................................23
0
Chapter Notes.........................................................23
4
7. Heaps..................................................................236
Introduction..........................................................23
6
Complete k-ary Trees..................................................23
7
Full k-ary Trees......................................................23
8
Heap Implementation with Full Trees...................................23
9
Building a Heap in Linear Time........................................24
0
Heap Sort.............................................................24
1
Implementing Heaps with Pointers......................................24
2
Lower Bounds on Heap Operations and Sorting...........................24
3
Exercises.............................................................24
5
Chapter Notes.........................................................24
9
8. Balanced Trees.........................................................250
Introduction..........................................................25
0
2-3 Trees.............................................................25
1
Inserting into a 2-3 Tree.......................................
..252
Deleting from a 2-3 Tree........................................
..254
Joining 2-3 Trees...............................................
..257
Splitting 2-3 Trees.............................................
..259
Red-Black Trees.......................................................26
1

Properties of Red-Black Trees...................................

..262
Equivalence of Red-Black and 2-3 Trees..........................
..263
Example: Red-Black Tree Insertion Algorithm.....................
..264
Example: Inserting into a Red-Black Tree in Sorted Order........
..265
Height of a Red-Black Tree......................................
..266
A VL Trees............................................................26
7
The AVL Algorithm...............................................
..268
Height of an A VL Tree..........................................
..269
Storing Data Only in the Leaves.......................................27
1
Exercises.............................................................27
2
Chapter Notes.........................................................28
0
9. Sets Over a Small Universe.............................................282
Introduction..........................................................28
2
On the Fly Array Initialization.......................................28
4
In-Place Permutation..................................................28
5
Bucket Sorting........................................................28
6
Bit-Vector Representation of Sets.....................................28
7
Union-Find Problem....................................................28
8
Linked List Implementation of Union-Find........................
..289
Tree Implementation of Union-Find...............................
..290
Tree Implementation of Union-Find with Path Compression.........
..291
Example: Off-Line Processing of Heap Operations.................
..293
Other Operations that can be Added to Union-Find................
..294
Exercises.............................................................29
5
Chapter Notes.........................................................30
1
10. Graphs................................................................302
Introduction..........................................................30
2
Graph Terms...........................................................30
3
Representing Graphs...................................................30
4
Depth-First Search....................................................30
5
Breadth-First Search..................................................30
6

Depth-First Spanning Trees............................................30

7
Bi-Connected and Strongly-Connected Components........................30
8
Bi-Connected Components of an Undirected Graph..................
..309
Strongly-Connected Components of a Directed Graph...............
..310
Minimum Weight Spanning Trees.........................................31
1
Proof of Correctness of Prim and Kruskal Algorithms.............
..311
Implementation of Prim's Algorithm..............................
..312
Implementation of Kruskal's Algorithm...........................
..313
Topological Sort of a Directed Graph..................................31
4
Euler Paths...........................................................31
5
Single-Source Minimum Cost Paths......................................31
6
Dijkstra's Algorithm............................................
..316
Adjacency Matrix Implementation of Dijkstra's Algorithm.........
..317
Adjacency List Implementation of Dijkstra's Algorithm...........
..317
All Pairs Minimum Cost Paths..........................................31
8
Floyd's Algorithm for Shortest Paths............................
..318
Warshall's Algorithm for Transitive Closure.....................
..319
Generic Path Finding Framework..................................
..320
Maximum Flow..........................................................32
1
Undirected Paths................................................
..322
Augmenting Paths................................................
..322
Augmenting Path Theorem for Flow................................
..322
Max-Flow = Min-Cut Theorem......................................
..322
Generic Approach to Computing Maximum Flow......................
..323
Edmonds-Karp Algorithm..........................................
..324
The Residual and Level Graphs...................................
..326
Blocking Flows..................................................
..327
Dinic's Algorithm...............................................
..327
MKM Algorithm...................................................
..328
Bounded Flow....................................................
..329

Maximum Matching......................................................33
1
Augmenting Path Theorem for Matching............................
..331
Matching in Bipartite Graphs....................................
..332
Matching in Undirected Graphs...................................
..334
Stable Marriage.......................................................33
7
NP-Complete Graph Problems............................................33
8
Polynomial-Time Reductions......................................
..339
NP-Complete Problems............................................
..339
The Class NP....................................................
..339
The "first" NP-Complete Problem.................................
..340
The "second" NP-Complete Problem................................
..340
Dealing with NP-Complete Graph Problems.........................
..341
Exercises.............................................................34
2
Chapter Notes.........................................................36
1
11. Strings...............................................................367
Introduction..........................................................36
7
Lexicographic Sorting of Strings......................................36
9
Knuth-Morris-Pratt (KMP) String Matching..............................37
0
KMP Algorithm Using Back-Up Links...............................
..370
Back-Up Diagrams................................................
..371
Efficient Computation ofthe KMP Back-Up Array...................
..372
Converting the KMP Back-Up Array to Direct Links................
..373
KMP Algorithm Using Direct Links................................
..373
Reducing the Space for the KMP Direct Array.....................
..374
Boyer-Moore String Matching...........................................37
5
Karp-Rabin Randomized "Finger Print" String Matching..................37
6
Shift-And String Matching.............................................37
7
Shift-and with don't-care positions.............................
..377
Shift-And with Anything-But Positions...........................
..378
Shift-And with Minimum Mis-Matches..............................
..380
Shift-And with Character Mapping................................

..381
Shift-And with Character Bit-Vectors............................
..382
Comparison of String Matching Methods.................................38
5
Pattern Matching......................................................38
6
Pattern Diagrams................................................
..387
McNaughton-Yamada Algorithm.....................................
..388
Matching with Pattern Diagrams..................................
..390
Tries.................................................................39
1
Example: Sorting Strings with Tries.............................
..391
Example: Aho-Corasick Multiple String Matching..................
..392
Example: Prefix and Huffman Codes...............................
..393
Example: Data Compression using a Dynamic Dictionary............
..394
Compact Tries.........................................................39
6
Suffix Tries..........................................................39
7
Example Applications of Suffix Tries............................
..397
Simple Suffix Trie Construction Algorithm.......................
..398
McCreight's Linear Time Suffix Trie Construction................
..399
Example: Brute-Force Versus McCreight on an$....................
..401
Sliding Suffix Tries..................................................40
2
Sliding Window With Two McCreight Tries.........................
..403
Fiala-Greene Sliding Suffix Trie Algorithm......................
..404
Example: Data Compression using a Sliding Window................
..406
Edit Distance: A Measure of String Similarity.........................40
7
Example: Longest Common Sub-Sequence............................
..408
Arithmetic Codes......................................................40
9
Conceptual Arithmetic Encoding / Decoding Algorithm.............
..410
Defining The End of a Finite String.............................
..411
Example.........................................................
..412
On-Line Encoding and Decoding...................................
..414
Practical Considerations........................................
..414
The Burrows-Wheeler Transform.........................................41

5
Inverse BWT Using Only Two Passes...............................
..416
Example: MTF Data Compression...................................
..417
Exercises.............................................................41
9
Chapter Notes.........................................................42
6
12. Discrete Fourier Transform............................................433
Introduction..........................................................43
3
Complex Numbers.......................................................43
4
Complex Exponentials..................................................43
5
Principal nth Roots of Unity..........................................43
6
Definition of the DFT.................................................43
7
F and G are Inverse Functions.........................................43
8
Similarity of F and G.................................................43
8
Examples..............................................................43
9
How to Split F into Two Computations of Half the Size.................44
3
Divide and Conquer "Butterfly.........................................44
4
Recursive FFT Algorithm...............................................44
5
In-Place Bit Reversal.................................................44
6
Recursive In-Place FFT Algorithm......................................44
7
Non-Recursive In-Place FFT Algorithm..................................44
8
Simplified Non-Recursive In-Place FFT Algorithm.......................44
9
DFT over Finite Fields on an Array of Integers........................45
1
Example: Fast Convolutions with the DFT...............................45
2
DFT On Two Arrays of Reals............................................45
3
DFT On A Single Array of Reals........................................45
4
Inverse DFT for Reals.................................................45
5
Discrete Cosine Transform.............................................45
6
C and D are Inverse Functions.........................................45
7
DCT Basis Functions...................................................45
8
Relationship of the DCT to the DFT....................................45
9
Computing the DCT in O(nlog(n)) Time..................................46
0

Computing the Inverse DCT in O(nlog(n)) Time..........................46

1
Two Dimensional DFT and DCT...........................................46
3
Example: JPEG Image Compression.......................................46
4
Example: MPEG Video Compression.......................................46
5
Exercises.............................................................46
6
Chapter Notes.........................................................47
9
13. Parallel Computation..................................................482
Introduction..........................................................48
2
Example: Summing an Array.......................................
..489
Example: List Prefix-Sum / List Ranking.........................
..490
Example: List Prefix-Sum on a Binary Tree.......................
..491
Example: 0(1) CRCW Array Max with 0(n2) Processors..............
..493
Example: O(loglog(n)) CRCW Array Max with O(n) Processors.......
..494
Example: Matrix Multiplication..................................
..495
Example: Merge Sort.............................................
..496
Example: Quick Sort.............................................
..497
Brent's Lemma.........................................................49
8
PRAM Simulation.......................................................49
9
EREW PRAM MODEL.......................................................50
0
Example: Broadcast on an EREW PRAM..............................
..500
Example: Sum on an EREW PRAM....................................
..500
Example: Matrix Multiplication on an EREW PRAM..................
..501
Data Distribution on an EREW PRAM...............................
..502
Sorting on an EREW PRAM.........................................
..503
Hypercube / CCC / Butterfly Networks..................................50
7
Hypercube (HC)..................................................
..507
Cube Connected Cycles (CCC).....................................
..507
Butterfly (BF)..................................................
..508
Equivalence ofthe CCC and Butterfly Networks....................
..509
Example: Broadcast and Sum on a Butterfly.......................
..510
Example: Prefix-Sum on a Butterfly..............................

..511
Example: Matrix Multiplication on a Butterfly...................
..512
Data Distribution on a Butterfly................................
..513
Sorting on a Butterfly..........................................
..515
1-1 Packet Routing on a Butterfly...............................
..517
Mesh Network..........................................................51
8
Example: Broadcast on a Mesh....................................
..519
Example: Sum on a Mesh..........................................
..519
Example: Prefix-Sum on a Mesh...................................
..520
Example: Matrix Multiplication on a Mesh........................
..521
Data Distribution on a Mesh.....................................
..523
Sorting on a Mesh...............................................
..524
1-1 Packet Routing on a Mesh....................................
..525
Area-Time Tradeoffs...................................................52
6
Computer Chips..................................................
..526
Boolean Functions...............................................
..526
Example: cMOS...................................................
..527
Example: nMOS...................................................
..528
Constructing Memory With Chips..................................
..529
Computing with Chips............................................
..531
Parallel Hardware Layout........................................
..532
Area-Time Tradeoff for Sorting..................................
..533
Sorting Area-Time Tradeoff vs. PRAM Simulation..................
..535
Generalizations ofthe Sorting Area-Time Tradeoff................
..535
Exercises.............................................................53
6
Chapter Notes.........................................................55
1
Appendix: Common SumS.....................................................553
A. Approximating Sums with Integrals..................................55
3
B. Arithmetic Sum.....................................................55
5
C. Simple Geometric Sum (unweighted, k=O).............................55
6
D. Linear Weighted Geometric Sum (k= 1)...............................55
7

E. Quadratic Weighted Geometric Sum (k=2).............................55

8
F. Cubic Weighted Geometric Sum (k=3).................................55
9
G. Weighted Geometric Sum (for any non-negative integer k)............56
0
I. Harmonic Sum.......................................................56
3
J. Sums of Inverse Powers.............................................56
4
Bibliography..............................................................565
Notation..................................................................593
Index.....................................................................595