0 ratings0% found this document useful (0 votes) 757 views992 pagesAlgorithms Sequential, Parallel, and Distributed
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content,
claim it here.
Available Formats
Download as PDF or read online on Scribd
ALGORITHMS
SEQUENTIAL, PARALLEL,
AND DISTRIBUTED
Kenneth a pecan and Jerome L. Paul
sity of Cincinnati
THOMSONTHOMSON
Algorithms: Sequential, Parallel, and Distributed
by Kenneth A. Berman and Jerome L. Paul
Senior Acquisitions Editor: Production Editor:
Amy Yarnevich Jennifer Harvey
Senior Product Manager: Designer:
‘Alyssa Pratt Pre-Press Company, Inc
oot pec Cover Design:
ieaitrien Alec Laura Rickenbach
Jenny Smith
Cover Image:,
‘Senior Marketing Manager: RayPaul
Karen Seitz
COPYRIGHT © 2005 Course Technol- ALL RIGHTS RESERVED. No part
ogy, adivision of Thomson Learn- _of this work covered by the copy-
ing, Inc. Thomson Learning" is a _right hereon may be reproduced
trademark used herein under fr used in any form or by any
license, ‘means—graphic, electronic, or
‘mechanical, including photacopy-
ing, recording, taping, Web distrib-
ution, or information storage and
retrieval systems—without the writ:
ten permission of the publisher.
Printed in the United States of
America
23456789 aM 0605,
For permission to use material from
For more information, contact thistext or produc, contact us by
Course Technology,
25 Thomson Place, ‘Tel (800) 730-2214
Boston, Massachusetts, 02210, fax (200) 730-2015
wwewthomsonrights.com
(Or find us on the World Wide Web
‘at: www course.com
Compositor:
Pre-Press Company, Inc.
Printer:
Quebecor World Taunton
Disclaimer
Course Technology reserves the
right to revise this publication and
‘make changes from time to time in
its content without notice
ISBN 0-534-42057-5,TABLE OF CONTENTS
Preface
PART! INTRODUCTION TO ALGORITHMS
Chapter 1 Introduction and Preliminaries
1.1. Algorithms from Ancient to Modern Times.
1.1.1 Evaluating Powers
1.1.2 The Euclidean Algorithm
1.1.3 Babylonian Square Roots
1.1.4 Evaluating Polynomials
xxiii1M Table of Contents
Chapter 2
1.2
13
1.4
24
22
23
24
25
26
Toward a Modern Theory of Algorithms
1.2.1 Early Computation Devices
1.2.2.A Formal Theory of Algorithms
1.2.3 Advent of Electronic Computers
Computing in the Third Millennium
1.3.1 Serial Computing
1.3.2 Parallel Computing
1.33 The internet
1.3.4 Distributed Computing
1.3.5 Quantum Computing
Closing Remarks
References and Suggestions for Further Reading
Exercises
Design and Analysis Fundamentals
Guidelines for Algorithm Design
Recursion
Data Structures and Algorithm Design
Major Design Strategies
2.4.1 Divide and Conquer
2.4.2 Dynamic Programming
2.4.3 The Greedy Method
2.4.4 Backtracking and Branch-and-Bound
Analyzing Algorithm Performance
2.5.1 Multiple-Variable Input Size
Designing and Analyzing Some Basic
Comparison-Based List Algorithms
2.6.1 Linear Search
2.6.2 Binary Search
2.6.3 Interpolation Search
2.6.4 Finding the Maximum and Minimum Elements in a List
2.6.5 Insertionsort
2.6.6 Mergesort
2.6.7 Quicksort
13
1B
14
14
15
15
16
16
7
18
19
20
20
23
24
24
26
27
28
28
28
29
29
33
33
34
35
37
38
40
45
49Table of Contents Mv
2.7 Closing Remarks 54
References and Suggestions for Further Reading 55
Exercises 56
Chapter 3 Mathematical Tools for Algorithm Analysis 63
3.1. Asymptotic Behavior of Functions 64
3.1.1 Asymptotic Order 65
3.1.2 Illustrations of Orders of Complexity 68
3.1.3 The Asymptotic Relations O and © 70
3.1.4 Hierarchy of Orders B
3.1.5 Establishing Order Relationships 75
3.1.6 LHopital’s Rule 79
3.1.7 Asymptotic Notation for Functions of Several Variables 80
3.2 Asymptotic Order Formulae for Three Important Series. 81
3.2.1 Sums of Powers a1
3.2.2 Sums of Logarithms 83
3.2.3 The Harmonic Series 84
3.3. Recurrence Relations for Complexity 85
3.3.1 Solving Linear Recurrence Relations Relating the nth 88
Term to the (n/b)th Term
3.3.2 Solving Linear Recurrence Relations Relating the nth 90
Term to the (n-1)st Term
3.4 Mathematical Induction and Proving the Correctness 1
of Algorithms
3.5 Establishing Lower Bounds for Problems 95
3.5.1 Lower Bound for Finding the Maximum 96
3.5.2 Lower Bounds for Adjacent-Key Comparison Sorting 7
3.5.3 Lower Bound for Comparison-Based Sorting in General 99
3.6 Hard Problems 100
3.7 NP-Complete Problems 102
3.8 Closing Remarks 104
References and Suggestions for Further Reading 105
Exercises 105Chapter 4
44
4.2
43
44
45
46
47
Table of Contents
Trees and Applications to Algorithms
Definitions
4.1.1 Binary Trees
4.1.2 General Trees
Mathematical Properties of Binary Trees
4.2.1 Complete and Full Binary Trees
4.2.2 Logarithmic Lower Bound for the Depth of Binary Trees
4.2.3 2-Trees
4.2.4 Internal and Leaf Path Lengths of Binary Trees
4.2.5 Number of Binary Trees
Implementation of Trees and Forests
4.3.1 Array Implementation of Complete Trees
4.3.2 Implementing General Binary Trees Using Dynamic
Variables
4.3.3 Child-Sibling Representation of a General Tree
4.3.4 Parent Representation of Trees and Forests
Tree Traversal
4.4.1 Traversing Binary Trees
4.4.2 Traversing General Trees
Binary Search Trees
45.1 Definition and Examples of Binary Search Trees
4.2 Searching a Binary Search Tree
4.5.3 Inserting Nodes into Binary Search Trees
4.5.4 Deleting Nodes from Binary Search Trees
4.5.5 Multiway Search Trees
4.5.6 Treesort
Priority Queues and Heaps
4.6.1 Heap Implementation of a Priority Queue
4.6.2 Inserting into a Heap
4.6.3 Deleting from a Heap
4.6.4 Creating a Heap
4.6.5 Sorting by Selection: Selectionsort versus Heapsort
Implementing Disjoint Sets
4.7.1 Union and Find Algorithms
4.7.2 Improved Union and Find Using the Collapsing Rule
113
115
115
116
17
118
120
120
122
125
126
126
126
126
128
130
130
133
134
135,
136
137
138
141
142
144
145,
146
147
150
152
153
154
156Table of Contents vii
48 Closing Remarks 160
References and Suggestions for Further Reading 160
Exercises 161
Chapter 5 More on Sorting Algorithms 167
5.1. Shellsort 168
5.2. Bingosort 172
5.3 Radixsort 174
5.4 External Sorting 179
5.5 Closing Remarks 184
References and Suggestions for Further Reading 184
Exercises 185
Chapter6 Probability and Average Complexity 189
of Algorithms
6.1 Expectation and Average Complexity 190
6.2 Techniques for Computing Average Complexity 191
6.3 Average Complexity of LinearSearch 193
6.3.1 Average Complexity of LinearSearch with Repeated 194
Elements.
6.4 Average Complexity of insertionSort 195
65 Average Complexity of QuickSort 196
6.6 Average Complexity of MaxMin2 198
6.7 Average Complexity of BinarySearch and SearchBinSrchTree 200
68 Searching a Link-Ordered List 203
6.9. Closing Remarks 208
References and Suggestions for Further Reading 209
Exercises 209iim Table of Contents
ART II_ MAJOR DESIGN STRATEGIES _
chapter 7
7A
72
73
74
75
Chapter 8
8.1
82
83
8.4
85
86
87
The Greedy Method
General Description
Optimal Sequential Storage Order
The Knapsack Problem
Huffman Codes
7.4.1 Binary Prefix Codes
7.4.2 The Huffman Algorithm
Closing Remarks
References and Suggestions for Further Reading
Exercises
Divide-and-Conquer:
The Divide-and-Conquer Paradigm
Symbolic Algebraic Operations on Polynomials
8.2.1 Multiplication of Polynomials of the Same Input Size
8.2.2 Multiplication of Polynomials of Different input Sizes
Multiplication of Large Integers
Multiplication of Matrices
Selecting the kth Smallest Value in a List
85.1 The Algorithm Select
8.5.2 A Version of Select with Linear Worst-Case Complexity
Two Classical Problems in Computational Geometry
8.6.1 The Closest-Pair Problem
8.6.2 The Convex Hull Problem
Closing Remarks
References and Suggestions for Further Reading
Exercises
213
215
215
218
219
224
225
226
231
232
232
237
237
239
240
242
243
244
246
247
250
253
254
257
259
260
261Chapter 9
91
9.2
93
94
95
Chapter 10
10.1
10.2
10.3
10.4
Table of Contents
Dynamic Programming
Optimization Problems and the Principle of Optimality
Optimal Parenthesization for Computing a Chained
Matrix Product
Optimal Binary Search Trees
Longest Common Subsequence
Closing Remarks
References and Suggestions for Further Reading
Exercises
Backtracking and Branch-and-Bound
State-Space Trees
10.1.1 An Example
10.1.2 Searching State-Space Trees
Backtracking
10.2.1 A Backtracking Algorithm for the Sum of Subsets Problem
10.2.2 The General Backtracking Paradigm
10.2.3 Tic-Tac-Toe
10.2.5 Solving Optimization Problems Using Backtracking
Branch-and-Bound
10.3.1 General Branch-and-Bound Paradigm
Closing Remarks
References and Suggestions for Further Reading
Exercises
265
266
267
272
280
284
285
285
289
291
292
293
295
296
300
302
308
314
316
319
320
320
ix1M Table of Contents
PART II|_ GRAPH AND NETWORK ALGORITHMS
Chapter 11
14
11.3
114
Chapter 12
12.4
12.2
123
Graphs and Digraphs
Graphs and Digraphs
11.1.1 Graphs
11.1.2 Graphs as Interconnection Networks
11.1.3 Digraphs
11.1.4 Implementing Graphs and Digraphs
Search and Traversal of Graphs
11.2.1 Depth-First Search
11.2.2 Depth-First Search Tree
11.2.3 Depth-First Traversal
11.2.4 Breadth-First Search
11.2.5 Breadth-First Search Tree and Shortest Paths
11.2.6 Searching and Traversing Digraphs
Topological Sorting
Closing Remarks
References and Suggestions for Further Reading
Exercises
Minimum Spanning Tree and
Shortest-Path Algorithms
Minimum Spanning Tree
12.1.1 Kruskal’s Algorithm
12.1.2 Prim’s Algorithm,
Shortest Paths In Graphs and Digraphs
12.2.1 The Bellman-Ford Algorithm
12.2.2 Dijkstra’s Algorithm
12.2.3 Floyd's Algorithm
Closing Remarks
References and Suggestions for Further Reading
Exercises
325
327
328
329
335
336
338
342
343
346
347
348
350
350
354
356
357
357
365
366
367
370
375
376
381
386
389
390
391Chapter 13
13.4
13.2
13.3
13.4
Chapter 14
14.1
14.2
14.3
Table of Contents
Graph Connectivity and Fault-Tolerance
of Networks
Strongly Connected Components
Articulation Points and Biconnected Components
Fault-Tolerant Routing Schemes
13.3.1 Open Ear Decompositions
13.3.2 s-t Orderings and Independent Spanning Trees
Closing Remarks
References and Suggestions for Further Reading
Exercises
Matching and Network Flow Algorithms
Perfect Matchings in Bipartite Graphs
14.1.1 The Marriage Problem
14.1.2 The Hungarian Algorithm
14.1.3 Maximum Perfect Matching in a Weighted
Bipartite Graph
Maximum Flows in Capacitated Networks
14.2.1 Flows in Digraphs
14.2.2 Flows in Capacitated Networks
14.2.3 Finding an Augmenting Semipath
14.2.4 Bounding Flow Values by Capacities of Cuts
14.2.5 The Ford-Fulkerson Algorithm
14.2.6 Maximum Flows in O/1 Networks: Menger’s Theorem
14.2.7 Maximum Size Matching
Closing Remarks
References and Suggestions for Further Reading
Exercises
395
396
400
408
409
4a
413
a3
a4
417
418
418
420
424
428
428
430
433
435
438
a2
443
445
445
446
m xidi ML Table of Contents
PART IV PARALLEL AND DISTRIBUTED ALGORITHMS
Chapter 15
15.1
15.2
15.3
15.4
15.5
Chapter 16
16.1
Introduction to Parallel Algorithms and
Architectures
Approaches to the Design of Parallel Algorithms
Architectural Constraints and the Design of Parallel
Algorithms
15.2.1 Single Instruction Versus Multiple instruction
15.2.2 The Number and Type of Processors Available
15.2.3 Shared Memory: PRAMS
15.2.4 Example: Searching on a PRAM
15.2.5 Distributed Memory: Interconnection Networks
15.2.6 Three Basic Goodness Measures for
Interconnection Networks
15.2.7 Example: Searching on Meshes
15.2.8 VO Constraints
Performance Measures of Parallel Algorithms
15.3.1 Speedup and Amdahl’s Law:
Parallel Sorting
15.4.1 Sorting on the CRCW and CREW PRAMs
15.4.2 Even-Odd Mergesort on the EREW PRAM
15.4.3 The 0/1 Sorting Lemma
15.4.4 Sorting on the One-Dimensional Mesh
15.4.5 Sorting on the Two-Dimensional Mesh
Closing Remarks
References and Suggestions for Further Reading
Exercises
Parallel Design Strategies
Parallel Prefix Computations
16.1.1 Parallel Prefix on a PRAM
16.1.2 Parallel Prefix on the Complete Binary Tree
16.1.3 Parallel Prefix on the Two-Dimensional Mesh
449
451
453
453
454
454
455
457
462
465
467
472
472
477
478
478
480
487
491
494
498
499
500
505
506
506
508
51016.2
16.3
16.4
Chapter 17
vA
17.2
173
174
175
17.6
Table of Contents Mx
16.1.4 Knapsack in Parallel
16.1.5 Carry-Lookahead Addition
Pointer Jumping
16.2.1 Finding Roots in a Forest
16.2.2 Minimum Spanning Tree in Parallel
Matrix Operations in Parallel
16.3.1 Matrix Multiplication on the CREW PRAM
16.3.2 All-Pairs Shortest Paths in Parallel
Closing Remarks
References and Suggestions for Further Reading
Exercises
Internet Algorithms
Search Engines
Ranking Web Pages
17.2.1 PageRank
17.2.2 HITS Algorithm
Hashing
17.3.1 Hash Functions
17.3.2 Collision Resolution by Chaining
17.3.3 Collision Resolution with Open Addressing
Caching, Content Delivery, and Consistent Hashing
17.4.1 Web Caching
17.4.2 Consistent Hashing
Message Security Algorithms: RSA
17.5.1 Modular Arithmetic
17.5.2 RSA Public-Key Cryptosystem
17.5.3 Digital Signatures and Authentication
Closing Remarks
References and Suggestions for Further Reading
Exercises
511
513
515
515,
518
522
523
523
524
525
525
531
532
532
533
537
540
541
543
545
548,
548
555
556
560
561
561
562
562iv. ML Table of Contents
‘chapter 18
18.1
18.2
18.3
18.4
18.5
18.6
18.7
Chapter 19
19.1
19.2
Distributed Computation Algorithms
SPMD Distributed-Computing Model
Message Passing
18.2.1 Synchronous (Blocking) Message Passing
18.2.2 Avoiding Deadlock
18.2.3 Compare-Exchange Operations
18.2.4 Broadcast, Scatter, and Gather
18.2.5 Asynchronous (Nonblocking) Message Passing
18.2.6 Computation/Communication Alteration
Distributed Even-Odd Transposition Sort
Embarrassingly Parallel Master-Worker Paradigm
18.4.1 Embarrassingly Parallel Master-Worker Code Skeleton,
18.4.2 Matrix Multiplication
Shared-Task Master-Worker Paradigm
18.5.1 Generic Shared-Task Master-Worker Code Skeleton
Distributed Depth-First Search (Backtracking)
18.6.1 Embarrassingly Parallel Backtracking
18.6.2 Shared-Task Parallel Backtracking
18.6.3 Sum of Subsets Problem (Optimization Version)
Closing Remarks
References and Suggestions for Further Reading
Exercises
Distributed Network Algorithms
Leader Election
19.1.1 Leader Election in a Ring
19.1.2 Leader Election in a General Network
Broadcasting and Breadth-First Search
19.2.1 Broadcasting Using Flooding
19.2.2 Broadcasting in a Tree
19.2.3 Performing Global Computations Using Fan-In
19.2.4 Distributed Algorithm for Constructing a
Breadth-First Search Tree
19.2.5 Leader Election in a General Distributed Network
567
569
570
572
574
575
577
580
582
583
584
584
586
588
589
594
595
596
597
601
602
602
607
610
610
612
613
613
614
614
617
618193
19.4
19.5
19.6
197
Table of Contents
Shortest Paths
All-Pairs Shortest Paths
Minimum Spanning Tree
‘Asynchronous Model
19.6.1 Asynchronous Leader Election in a Ring
19.6.2 Asynchronous Leader Election in a Tree
19.6.3 Asynchronous Leader Election in a General Network
19.6.4 Asynchronous Broadcasting
Closing Remarks
References and Suggestions for Further Reading
Exercises
PARTV SPECIAL TOPICS
Chapter 20
20.1
20.2
20.3
20.4
205
206
20.7
Chapter 21
214
21.2
String Matching and Document Processing
The Naive Algorithm
The Knuth-Morris-Pratt Algorithm
The Boyer-Moore String-Matching Algorithm
The Karp-Rabin String-Matching Algorithm
Approximate String Matching
Tries and Suffix Trees
20.6.1 Standard Tries
20.6.2 Compressed Tries
20.6.3 Suffix Trees
Closing Remarks
References and Suggestions for Further Reading
Exercises
Balanced Search Trees
The Dictionary Problem
Rotations in Binary Search Trees
618
619
621
623
624
625
625
625
626
626
627
629
631
632
634
638
640
643
645
645,
647
648,
649
649
650
653
653
654
awvi ME Table of Contents
213
214
21.5
Chapter 22
22.4
22.2
22.3
22.4
22.5
22.6
22.7
Chapter 23
23.4
23.2
Red-Black Trees
21.3.1 Definition and Depth of Red-Black Trees
21.3.2 Implementing Red-Black Trees
21.3.3 Inserting a Node into a Red-Black Tree
21.3.4 Deleting a Node from a Red-Black Tree
B-Trees
21.4.1 Definition and Properties of B-Trees
21.4.2 Searching a B-Tree
21.4.3 Inserting a Key into a B-Tree
21.4.4 Deleting a Key from a B-Tree
Closing Remarks
References and Suggestions for Further Reading
Exercises
The Fast Fourier Transform
The Discrete Fourier Trafsform
The Fast Fourier Transform
An Iterative Version of the Fast Fourier Transform
Transforming the Problem Domain
The Inverse Discrete Fourier Transform and Fast.
Polynomial Multiplication
The Fast Fourier Transform in Parallel
22.6.1 The FFT on a PRAM
22.6.2 The FFT on the Butterfly Network
Closing Remarks
References and Suggestions for Further Reading
Exercises
Heuristic Search Strategies: A*-Search
and Game Trees
Artificial Intelligence: Production Systems
8-Puzzle Game
657
657
660
661
665
675
676
678
678
684
686
686
687
693
694
604
699
704
706
709
709
710
m1
m1
72
715
716
7Toble of Contents Ml xvii
23.3 A*Search 720
23.3.1 Heuristics, 722
23.3.2 A*-Search and the 8-Puzzle Game 77
23.3.3 Shortest Paths in the Freeway System 728
23.4 Least-Cost Branch-and-Bound 730
23.5 Game Trees 732
23.6 Closing Remarks 742
References and Suggestions for Further Reading 743
Exercises 744
Chapter 24 Probabilistic and Randomized Algorithms 751
24.1. Probabilistic Algorithms 752
24,2. Randomizing Deterministic Algorithms 753
24.2.1 Randomizing LinearSearch 754
24.2.2 Randomizing ProbeLinSrch 755
24.2.3 Randomizing QuickSort 756
24,3. Monte Carlo and Las Vegas Algorithms 757
24.3.1 Biased Monte Carlo Algorithms 787
24.3.2 A Monte Carlo Algorithm of Testing Polynomial Equality 759
24.3.3 Primality Testing 760
24.3.4 Minimum Cut in Graphs 763
24.3.5 Las Vegas Algorithms 769
24.4 Probabilistic Numerical Algorithms ™m™
24.4.1 Numerical Integration ™m
24.4.2 Estimating the Size of Sets 774
24.5 Probabilistic Parallel Algorithms 77
24.5.1 Exploiting Obvious Parallelism 77
24.5.2 The Marriage Problem 778
24.5.3 Randomized Protocols for Breaking Deterministic 781
Deadlock
24,6 Closing Remarks 783
References and Suggestions for Further Reading 784
Exercises 785Table of Contents
hapter 25
25.1
25.2
25.3
25.4
25.5
Lower-Bound Theory
Basic Terminology and Techniques
25.1.1 Simple Counting Arguments
25.1.2 Enumeration Arguments
25.1.3 Decision Trees
25.1.4 Adversary Arguments
25.1.5 Information Theoretic Arguments
25.1.6 Graph Theoretic Arguments
Decision Trees and Comparison Trees
25.2.1 Examples
25.2.2 Lower Bound for the Worst-Case Complexity of
CComparison-Based Searching of Lists
25.2.3 Lower Bound for the Average Complexity of
Comparison-Based Searching of Ordered Lists
25.2.4 Lower Bound for the Worst-Case Complexity of
Comparison-Based Sorting Algorithms
25.2.5 Lower Bound for the Average Complexity of
Comparison-Based Softing Algorithms
Adversary Arguments
25.3.1 Lower Bound for Finding Maximum and Minimum
Elements in a List
25.3.2 Lower Bound for Finding the Largest and Second-Largest,
Elements in a List
25.3.3 Lower Bound for Finding the Median Element in a List
Lower Bounds for Parallel Algorithms
25.4.1 Lower Bound for Finding the Maximum on the
EREW PRAM
25.4.2 Lower Bound for Comparison-Based Sorting on the
EREW PRAM
25.4.3 Lower Bound for Finding the Maximum on the
RCW PRAM
Closing Remarks
References and Suggestions for Further Reading
Exercises
789
789
790
790
790
791
791
792
792
793
796
797
798
799
800
800
803
806
808
809
810
810
812
813
813Table of Contents xix
Chapter 26 NP-Complete Problems 817
26.1 The Classes P and NP 818
26.1.1 Input Size 818
26.1.2 Optimization Problems and Their Decision Versions: B19
26.1.3 The Class NP 820
26.1.4 Some Sample Problems in NP 821
26.2 Reducibility 822
26.3 NP-Complete Problems: Cook's Theorem 824
26.4 Some Sample NP-Complete Problems 826
26.5 The Class co-NP 833,
26.6 The Classes NC and P-Complete 834
26.6.1 Straight-Line Code Problem for Boolean Expressions 835
26.6.2 Straight-Line Code Without Disjunction 836
26.6.3 NOR Straight-Line Code 837
26.6.4 Lexicographically First Maximal Independent Set 838
26.7 Closing Remarks 839
References and Suggestions for Further Reading 839
Exercises 840
Chapter 27 Approximation Algorithms 843
27.1 The Traveling Salesman Problem 844
27.2 Bin Packing 846
27.2.1 Next Fit 847
27.2.2 First Fit 848
27.3 The Steiner Tree Problem 849
27.4 The Facility Location Problem 852
27.5 Closing Remarks 856
References and Suggestions for Further Reading 856
Exercises 8561 ME Table of Contents
Appendix
Appendix
Appendix
Appendix
A
Ad
A2
A3
Aa
AS
AG
AT
Bl
B.2
B3
Ba
BS
c
D
DA
D.2
D3
Mathematical Notation and Background
Basic Mathematical and Built-in Functions
Modular Arithmetic
Some Summation Formulas
Binomial Coefficients
Sets
Complex Numbers
Mathematical Induction
‘A.7.1 Principle of Mathematical Induction
‘A.7.2 Variations of the Principle of Mathematical Induction
Linear Data Structures
Implementing the List ADT: Arrays Versus Linked Lists
Stack ADT
Queue ADT
Removing Recursion
B.4.1 Removing Tail Recursion
Removing Explicit Recursion by Simulation
Interpolating Asympotic Behavior
Random Walks in Digraphs
Enumerating Walks
Markov Chains and Random Walks
Eigenvalues and Eigenvectors
861
861
863
864
866
869
870
875
876
877
879
881
882
883
883
885
889
893
893
894
895Appendix E
Ea
£2
£3
E4
Appendix F
FA
F2
F3
Appendix G
G1
G2
Bibliography
Index
Table of Contents ML xxi
Elementary Probability Theory
Sample Spaces and Probability Distributions
Conditional Probability
Random Variables and Expectation
Conditional Expectation: Partitioning the Sample Space
Examples of Message-Passing Interface Code
Some Basic MPI Function Descriptions (C Version)
MPI Version of EvenOddSort
MPI Version of MatrixProduct
Pseudocode Conven
ns
Pseudocode Conventions for Sequential Algorithms
Pseudocode Conventions for Parallel Algorithms
G.2.1 Pseudocode Conventions for PRAMS
G.2.2 Pseudocode Conventions for Interconnection Network Models
897
897
900
902
906
911
912
914
917
919
919
924
924
927
933
947PREFACE
The objectives of this book are to provide a solid foundation for the classical the.
ory of sequential algorithms and to cover some of the most important recent al
gorithmic developments, including the rapidly advancing theory of parallel and
distributed algorithms, The book is intended to serve as a text for a core upper-
division undergraduate course or first-year graduate course in the design and
analysis of algorithms.
While parts of the book are revised and updated versions of text taken from
the authors’ previous book, Fundamentals of Sequential and Parallel Algorithms, we
have added a significant amount of new material, including an introduction to
Internet and distributed algorithms. Also, the material on parallel algorithms
that was integrated throughout the previous book has now been moved to a sep-
arate part on parallel and distributed algorithms. This separation enables the in-
structor to cover as little or as much of the parallel material as desired. Another
new feature is the inclusion of a number of appendices containing ancillary ma-
terial, such as a review of mathematical prerequisites. Placing this material in ap-
pendices keeps the text more sharply focused on algorithmic issues.
One of the major goals of this book is to provide the reader with a large
toolkit of fundamental sequential, parallel, and distributed algorithmic solutions
to frequently encountered problems, such as searching and sorting: matrixxiv ML Preface
manipulations; constructing minimum spanning trees and shortest paths in net-
works; scientific computations, including the Fast Fourier Transform; and data
compression and security, among others. Another major goal is to provide the
reader with the mathematical tools necessary to analyze the correctness and ef-
ficiency of algorithms and to be able to judge how close an algorithm is to being
optimal for a given problem. With these tools, the reader can choose the best
algorithm to use in a given circumstance from among several algorithms that
might be available for the problem. Another goal of this book is to enable the
reader to recognize problems that might be NP-hard and to search for algo-
rithms that work well on average or for an approximate solution that works well
even in the worst case. Perhaps the most important goal, however, is to enhance
the reader's ability to create new algorithms or modify existing algorithms for
solving new problems,
Fueled by the rapid expansion of parallel computers and distributed net-
works such as the Internet, ad hoc mobile computing networks, wireless com-
munication networks, peer-to-peer networks, and computation grids, the theory
of parallel and distributed algorithms has now taken a central position in com-
puter science and engineering. Many of the classical problems solved in the se~
quential environment now appear in the parallel and distributed environment.
In addition, new problems arise, such as packet-routing and message-passing
problems in distributed networks, searching and relevancy ranking of pages on
the World Wide Web, and reliability and security issues relating to communica-
tion in large-scale networks. In view of the recent advances in computer and
network technology, all students of computer science and computer engineering
should be exposed to the theory of parallel and distributed algorithms as part of
their undergraduate training.
‘When designing a parallel or distributed algorithm, many issues arise that
‘were not present in the sequential setting. For example, the type of architecture,
the communication model, the number of processors available, synchronization
issues, and so forth must be considered when designing a parallel or distributed
algorithm. Paradigms and tools developed for sequential algorithms can often be
applied in parallel and distributed settings but usually need to be modified and
combined with new tools to be implemented and to efficiently solve problems in
those settings. For example, exploiting the obvious parallelism implicit in the
divide-and-conquer paradigm is usually just the first step in designing a parallel
algorithm for a problem. Acceptable speedup usually occurs only when addi-
tional tools and strategies are developed to achieve further parallelism. New
tools in the distributed setting include flooding and broadcasting, gathering,
leader election, termination detection, and so forth. Although the primary focus
of this text remains the core material in a traditional undergraduate sequential
algorithms course, parallel and distributed algorithms are covered in sufficient
detail to prepare the student for independent study or for taking a more
advanced graduate course devoted entirely to these topics.Preface Maw
Organization of the Book
‘The book is divided into five parts, with appendices that review the prerequisite
material and supplemental material. We feel that this division allows for a nat-
ural flow of the topics and gives the instructor the flexibility to adapt the mater-
ial easily to fit a variety of course objectives. Part I provides the foundation for
the theory of algorithms. Part II introduces the student to the various classical se-
quential design strategies. Because of the importance of graph and network al-
gorithms in parallel and distributed computing, we devote Part III to this topic.
Part IV introduces the student to parallel and distributed algorithms, as well as
searching and the Internet. Finally, Part V contains a number of special topics,
each of which constitutes a major application area in the theory of algorithms.
Part I begins with a brief history of the subject of algorithms, including a dis-
cussion of some algorithms that have been around for quite a long time. We in-
troduce some guidelines for algorithm design, including recursion, and provide
some familiar examples as illustrations. The fundamental measures of sequential
algorithms are defined, including best-case, worst-case, and average complexity
of an algorithm. The notion of the asymptotic order of these measures is also in-
troduced. We illustrate these measures by analyzing some of the most familiar
algorithms, such as exponentiation, searching, and sorting. We also discuss the
important issues of establishing the correctness of an algorithm and of deciding
whether more-efficient algorithms should be sought. Some straightforward
lower bounds are established to show the student how to determine how close
an algorithm is to exhibiting optimal behavior. Because trees are so important in
the design and analysis of algorithms, we devote a chapter to their mathematical
properties, how they are implemented, and some of their fundamental uses,
such as in maintaining search trees and priority queues. We close Part I with a
discussion of the average behavior of algorithms.
Fortunately, the subject of algorithms does not consist of a large collection of,
ad hoc methods, each tailored to specific problems, without any unifying princi-
ples. Although ad hoc methods may have characterized the early development
of algorithm design, as the study of algorithms progressed, certain major design
strategies emerged. Part Il is devoted to classical major design strategies for algo-
rithms, including the greedy method, divide-and-conquer, dynamic program-
ming, and backtracking and branch-and-bound, These strategies are also used
for algorithm design throughout the remainder of the book. Other major design
strategies of more recent origin, such as heuristic search and probabilistic meth-
ods, are covered in Part V.
Although graph and network algorithms have been part of the core material
in sequential algorithms for some time, they have taken on special significance
with the emergence of parallel computers and various distributed networks
such as the Internet. To set the stage for the study of parallel and distributedPreface
algorithms, we devote Part III to the basic theory of graph and network algo-
rithms, The minimum spanning tree and shortest-path algorithms discussed here
use strategies developed in Part Il. Placing these algorithms in one place not only
serves to unify the treatment of graph and network algorithms but also reinforces
the importance of recognizing when one of the major design strategies is applica-
ble for a given problem. In addition to minimum spanning trees and shortest-path
algorithms, we discuss graph connectivity, fault-tolerant routing schemes, and
network flow algorithms, topics of major significance in distributed networks.
In Part IV, we introduce the student to the theory of parallel and distributed
algorithms, The parallel treatment includes a discussion of both the shared mem-
ory (PRAM) model and the distributed memory model as implemented by vari-
ous interconnection network models (meshes, hypercubes, and so forth). We
introduce pseudocode for parallel algorithms in both the PRAM and intercon-
nection network models, and we discuss how basic searching and sorting algo-
rithms are implemented on these models. We stress how to recognize when
sequential strategies for solving these problems might be adapted to the parallel
setting and what modifications and new tools might be necessary. We then in-
troduce some of these new tools for parallel design strategies, including parallel
prefix computations, pointer jumping, and parallel matrix operations.
In the distributed treatment of Part IV, we consider three main areas: search-
ing and ranking algorithms for the Internet, cluster computation using message
passing, and algorithms suitable for distributed networks. Our model for message
passing in the single-program, multiple-data (SPMD) model of distributed com-
puting closely follows the Message Passing Interface (MPI) model that has be-
come the standard for cluster-based computing. (In Appendix F, we show how
our pseudocode for message passing easily translates into actual MPI programs
implementing some of our algorithms.) We discuss synchronous and asynchro-
nous message passing, including how to avoid deadlock. The computation/com-
munication alternation model is introduced, and its implementation using the
master-worker paradigm is discussed.
In the chapter on distributed network algorithms, which is typified by the
lack of a central control mechanism (except for lock-step synchronization of
rounds in the synchronized setting), we also develop pseudocode for synchro-
nous and asynchronous distributed algorithms. We solve some fundamental dis-
tributed network problems such as flooding and broadcasting, leader election,
performing global operations, and computing shortest paths and minimum span-
ning trees. Here again, solutions to these problems are often facilitated by our
previous discussions of sequential algorithms, as well as by solving the problems
in the parallel environment.
In Part V, we cover a collection of special topics, all of which are active re-
search areas today. We give enough of an introduction to these topics to allow
the student to investigate them in further detail, either independently or in
courses entirely devoted to one or more of the topics.Preface Ml xxvii
Appendix A reviews some mathematical prerequisites that are needed for al-
gorithm analysis. Appendix B reviews linear data structures and includes a dis-
cussion of removing recursion. Appendix C discusses interpolating asymptotic
behavior. Appendix D discusses random walks on digraphs, Markov chains, and
eigenvalues. Appendix E includes a review of the results from elementary prob-
ability theory necessary to analyze the average behavior of algorithms, as well as
to discuss probabilistic algorithms. Appendix F contains a brief introduction to
‘MPI programming, including a translation into MPI code of the pseudocode for
some of the distributed algorithms discussed in Chapter 18. Finally, Appendix G
gives pseudocode conventions that we use for sequential and parallel algorithms.
In addition to the selected references and suggestions for further reading at
the end of each chapter, we include a bibliography at the end of the book. An In-
structor’s Manual containing a full set of solutions is accessible only to instructors
adopting the text. This supplement can be downloaded from www.course.com.
Possible Course Coverage
‘The inclusion of an extended treatment of parallel and distributed algorithms,
together with a number of special topics, provides enough material to serve as
the basis for a two-semester course in algorithms, However, the material in the
first two parts, together with a selection of topics from the remaining parts, gives
the instructor sufficient flexibility to readily adapt the material to a one-semester
course.
In Part I, much of the material in Chapters 1 through 4 can be quickly
reviewed or made a reading assignment for those students already familiar with
elementary data structures and algorithms. However, some important material
on exponentiation, the mathematical properties of binary trees, union and find
algorithms, and correctness proofs included in the first four chapters should be
covered, as these concepts are used often in the text and may not be included in
the student’s background. Chapter 5 covers some special topics in sorting, such,
as radix and external sorting, which are not needed in the remainder of the text.
‘The first three sections of Chapter 6 should be covered, as material there forms
the basis for the analysis of the average behavior of algorithms.
Most of Part II on major design strategies should be covered, as the examples
have been chosen to be important representatives of the strategies. These design.
strategies are the basis of many algorithms throughout the text, and the instruc-
tor may wish to include some additional examples when covering Part Il. For
example, standard greedy algorithms for minimum spanning trees and shortest-
path trees in graphs are placed in Part III on graph and network algorithms,xvii
Preface
but these topics certainly could be covered as part of Chapter 7 on the greedy
method. Also, the Fast Fourier Transform is discussed in Chapter 22 but could
also be included in the study of the divide-and-conquer strategy in Chapter 8.
In Part III, we would expect that Chapters 11 and 12 would be covered in
their entirety, whereas topics could be selected from the remaining chapters de-
pending on time and preference.
Part IV contains a number of topics in parallel and distributed algorithms. It
is entirely possible to study these chapters independently. For some examples in
distributed algorithms, itis useful, but not necessary, to have seen a parallel ver-
sion of the algorithm. However, our descriptions of the distributed algorithms,
such as even-odd sorting, are complete and mostly self-contained. Also, our dis-
cussion of Internet algorithms is largely independent of the rest of the material in
Part IV.
Finally, Part V contains a collection of special topics that are also indepen-
dent of one another. We would expect that most instructors would cover at least
part of Chapter 26 on NP-complete problems. The instructor can then pick and
choose from the remaining topics as time allows. For example, string-matching
algorithms, covered in Chapter 20, have taken on new significance with the
emergence of the Internet. Chapter 24 gives an introduction to probabilistic al-
gorithms, which is an important area of research today with many applications
‘The Fast Fourier Transform, certainly one of the most important algorithms for
scientific applications, is the topic of Chapter 22. All these topics are covered in
sufficient depth to prepare the student for further investigations.
Acknowledgments
‘We wish to thank the many students whose suggestions and corrections over the
years contributed to our first book and the current book. Special thanks go to
Michal Kouril for his suggestions and careful reading of Chapter 18 and his test-
ing of the MPI code in Appendix F. We also wish to thank our colleagues for their
suggestions, including Fred Annexstein, Yizong Cheng, John Franco, Ken Meyer,
John Schlipf, and Dieter Schmidt.
The material in this and the previous book have undergone a number of ex-
ternal reviews, which have been invaluable. The reviewers of the first book in-
clude Andrew Astromoff, Theodore Brown, Donald Burlingame, Adair Dingle,
Barry Donahue, Rex Dwyer, Keith Harrow, T. C. Hu, David John, Arkady
Kanevsky, Sampath Kannan, Sheau-Doug Lang, Udi Manber, Harold Martin,
Tim McGuire, Jerry Potter, Michael Quinn, Robert Roos, Richard Salter, Cliff
Shaffer, Greg Starling, Robert Tarjan, David Teague, Roger Wainright, and Lynn
Ziegler.‘The reviewers of the second book include:
Stefano Lonardi, University of California, Riverside
Steven Janke, Colorado College
Roy P. Pargas, Clemson Colllege
Lorin Schwiebert, Wayne State University
We would like to thank Kallie Swanson of Brooks/Cole who originally
signed us up for this project. We would also like to thank Alyssa Pratt and Amy
Yamevich of Course Technology, Jennifer Harvey of Pre-Press Company, and
Barbara McGowran, our copyeditor, for their highly professional and invaluable
technical support throughout the production of this book. It has been a pleasure
to work with them,
‘We cannot overstate our debt of gratitude to our wives, Peiyan and Ruta, for
their constant encouragement, love, and support. Without their considerable
sacrifices, this book would not have been possible. We dedicate this book to them
and to our children, Rachel and Sarah, and Ray and Renee.
K.A.B.andJ.L.P
xxixuu ONE
INTRODUCTION TO ALGORITHMS
A . * a.
core ae ;DEFINITION 1.1
(iP. Caun tt
+S cee
INTRODUCTION AND
PRELIMINARIES
The use of algorithms did not begin with the introduction of computers. In fact,
people have been using algorithms as long as they have been solving problems
systematically. Informally, we can describe an algorithm as a finite sequence of
rules that solves a problem. Although a completely rigorous definition of an al-
gorithm uses the notion of a Turing Machine, the following definition will sulfice
for our purposes.
An algorithm is a complete, step-by-step procedure for solving a specific problem.
Each step must be unambiguously expressed in terms of a finite number of rules
and guaranteed to terminate in a finite number of applications of the rules. Typ-
ically, a rule calls for the execution of one or more operations.
A sequential algorithm performs operations one at a time, in sequence,
whereas a parallel ot distributed algorithm can perform many operations
simultaneously.4M PARTI: Introduction to Algorithms
1.1
In this text, the operations allowed in a sequential algorithm are restricted to
the instructions found in a typical high-level procedural computer language.
These instructions are, for example, arithmetical operations, logical comparisons,
and transfers of control. A parallel or distributed algorithm allows the same op-
erations as a sequential algorithm (in addition to communication operations.
among processors), but a given operation can be performed on multiple data in-
stances simultaneously. For instance, a parallel algorithm might add 1 to each el-
ement in an array of numbers in a single parallel step.
‘An important consideration in algorithm design is what models and archi-
tectures will implement the algorithm, We discuss the various models later in
this chapter. In addition, we provide some historical background for the study of
sequential, parallel, and distributed algorithms and give a brief trace of how al-
gorithms have developed from ancient times to the present.
Algorithms from Ancient to Modern Times
‘The algorithms discussed in this section solve some classical problems in arith-
metic. Some algorithms commonly executed on today’s computers were origi-
nally developed more than three thousand years ago. The examples we present
illustrate the fact that the most straightforward algorithm for solving a given
problem often is not the most efficient.
1
-1 Evaluating Powers
An ancient problem in arithmetic is the efficient evaluation of integer powers of
a number x. The naive approach to evaluating x" is to repeatedly multiply xby it-
self n ~ 1 times, yielding the following algorithm
function NaivePowers(x, 1)
Input: x (a real number), (a positive integer)
Output: x
Product 1 do
i if odd(n) then AccumPow « AccumPowsPow endl
i in floor(n/2)CHAPTER 1: Introduction and Preliminaries I 7
Pow « PowePow
‘endwhile
Pow —AccumPow+Pow
return(Pow)
end Powers
Because it does not explicitly require the binary expansion of n, right-to-left
binary exponentiation is somewhat easier to program than the left-to-right bi-
nary method. We leave it as an exercise to show that both methods require the
same number of multiplications.
It turns out that Powers is the iterative version of a recursive method based
on the formula
_ {ey meven
ve (yoni nodd
which is simply the previous formula for x" with the order of exponenti
terchanged. Both formitlas lead to immediate implementations as recursive
functions for computing x", We discuss this further in Chapter 2.
Both right-to-left and left-to-right binary exponentiation illustrate that the
simplest algorithm for solving a problem is often much less efficient than a more
clever but perhaps more complicated algorithm. We state this as an important
key fact for algorithm design.
The simplest algorithm for solving a problem is often not the most efficient. Therefore,
when designing an algorithm, do not settle for just any algorithm that works.
‘The binary methods for exponentiation do not always yield the minimum number of multiplications.
For example, computing x by either of these methods requires six multiplications. However, it can
be done using only five multiplications (see Exercise 13).
The binary methods allow computers to perform the multiplications required to compute x’,
where 1 is an integer with hundreds of binary digits. However, for such values of n, the successive
Powers of x being computed grow exponentially, quickly exceeding the storage capacity of any
computer that could ever be built. In practice—for example, in Internet security communication
protocols that involve computing x" for n having hundreds of digits—the exponential growth of the
successive powers is avoided by always reducing the powers modulo some fixed integer p at each
stage (called modular exponentiation).BME PARTI: Introduction to Algorithms
1.1.2 The Euclidean Algorithm
One of the oldest problems in number theory is determining the greatest common
divisor of two positive integers a and b, or gcd(a,b), which is the largest positive
integer k that divides both a and 6 with no remainder. The problem of calculat-
ing ged(a.b) was already known to the mathematicians of ancient Greece. A
naive algorithm computes the prime factorization of a and b and collects com-
mon prime powers whose product is then equal to gcd(a.b). However, for large a
and b, computing the prime factorizations is very time consuming, even on
today’s fastest computers. A more efficient algorithm was published in Euclid’s
Elements (circa 300 8.C.) and was a refinement of an algorithm known 200 years
earlier. The earlier algorithm was based on the observation that for a > 6, an in-
teger divides both @ and b if, and only if, it divides a ~ b and b. Thus,
ged(a,b) = ged(a— bb), a=b, ged(aa)=a qa)
Formula (1.1.1) yields the following algorithm for computing gcd (a,b):
function NoiveGCD(o, b)
Input: 7, b (two positive integers)
Output: ged(a, b) (the greatest common disor of @ and b)
while o #b do
ifa > b then
aeo-b
else
beb-o
endif
endwhile
return(o)
fend NaiveGCD
After each iteration of the while loop in NaiveGCD, the larger of the previous
values of a and b is replaced by a strictly smaller positive number. Hence, GCD
eventually terminates, having calculated the greatest common divisor of the
original a and b,
Euclid’s gcd algorithm refines the algorithm NaiveGCD by utilizing the fact
that ifa > band a ~ bis still greater than b, then a ~ bin turn is replaced by a ~ 2b,
and so forth. Hence, if a is not a multiple of , then a is eventually replaced by
1 = a~ qb, where ris the remainder and q is the quotient wher a is divided by b.
‘Thus, all the successive subtractions can be replaced by the single invocation
amod b, where mod is the built-in function defined byCHAPTER 1: Introduction and Preliminaries ML 9
amod b = a~ [2]. cand bintegers, b#0,
where fora given positive real number x,|x] denotes the largest integer less than
or equal to x. For example, when calculating gcd(108,8), the 13 subtractions ex-
cecuted by the algorithm NaiveGCD (108 ~ 8, 100 ~ 8, 92 ~8,...,12 ~ 8) an be
replaced by the single calculation 108 mod 8 = 4.
‘The preceding discussion leads to an algorithm based on the following
formula:
gcd(a, b) = gcd(b, a mod b) (1.2)
Note that when a is a multiple of b, ged(a.b) = 6, and a mod b = 0. So the usual
convention ged(b,0) = b shows that Formula (1.1.2) remains valid when a is a
multiple of b
Euclid’s description of the ged algorithm based on (1.1.2) was complicated
by the fact that the algebraic concept of zero was not yet formalized. The follow-
ing is a modern version of Euclid’s algorithm:
function EucidGCO(a, 6)
Input: 0, b (two nonnegative integers)
Output: gcd(a, 5) (the greatest comman divisor of a and b)
while b ¥ 0 do
Remainder «a mod 6
aed
be Remainder
endvile
return(a)
end EucidGCD
‘The following illustrates EuclidGCD for input a = 10724, b = 864:
gcd(10724,864) = gcd(864,356) = ged(356,152) = ged( 152,52) = ged(52,48)
scd(48,4) = ged(4,0) = 4.
‘The problem of computing ged(a,b) has very important applications to mod-
em computing, particularly as it occurs in cryptography and commonly used
data security systems (see Chapter 18). It turns out that the while loop of
EuclidGCD never executes more than roughly log,(max(a,b}) times, so the algo-
rithm can be executed rapidly even when the integers a and b have hundreds of
digits each,0M PARTI: Introduction to Algorithms
FIGURE 1.2
Increasingly better
‘approximations
V5
of
1.1.3 Babylonian Square Roots
Another mathematical problem gained special significance in the sixth century
8.c. when the Pythagorean school of geometers made the startling discovery that,
the length of the hypotenuse of a right triangle with legs both equal to 1 cannot
be expressed as the ratio of two integers. This conclusion is equivalent to saying
that V2 is not a rational number and therefore its decimal expansion can never
be completely calculated. Long before the discovery of irrational numbers, peo-
ple were interested in calculating the square root of a given positive number a to
any desired degree of accuracy. A square root algorithm was already known to
the Babylonians by 1500 #.c. and is perhaps the first nontrivial mathematical
algorithm.
‘The Babylonian method for calculating Va is based on averaging two points
on either side of Va. The Babylonians may have discovered this algorithm by
considering the problem of laying outa square plot of a given area. For example,
for an area of 5, they may have considered, as a first approximation, a rectangu-
lar plot of dimensions 1 by 5. If they replaced one dimension by the average of
the previous two dimensions, they would obtain a “more square” plot of dimen-
sion 3 by 5/3. If they next replaced one of the new dimensions by the average of
3 and 5/3, the dimensions of the plot would be 7/3 by 5/(7/3) (roughly 2.33 by
2.14). More repetitions of this technique lead to plots having sides that are bet-
ter and better approximations to V5 (see Figure 1.2).
limiting square
105
— 1B a VG
ra
2r Me
We can use the Babylonian square root algorithm to calculate the square
root of any positive number a. Start with an initial guess x = x, for Va; any guess
will do, but a good initial guess leads to more rapid convergence. We calculate
successive approximations x,, X,.... «a using the formula
Hint t (a/x;
n= 7CHAPTER 1: Introduction and Preliminaries M11
We write the Babylonian square root algorithm as a function whose input para-
meters are the number a and a positive real number error measuring the desired
accuracy for computing Va, For simplicity, we use a as our initial approximation
of Va.
function BabylonianSQRT(a, error)
Input: 0 (@ positive number, error (@ positive real numbes)
Output: Vo accurate to within error
xeo
while ~ o/x| > enor do
xe + of/2
endwhile
Ereturn(s)
end BabylonionSQRT
Finding square roots is a special instance of the problem of determining the
approximate roots of a polynomial (note that Va is the positive root of the poly-
nomial ° ~ a). More generally, suppose fix) is a real-valued function, and we
wish to find a value of x (called a zero off) such that fix) = 0. If fis a continuous
function, then the intermediate value theorem of calculus guarantees that a
zero occurs in any closed interval (a, b] where fla) and fib) have opposite signs.
An algorithm (called the bisection method) for determining a zero of f proceeds by
bisecting such an interval (a, 6] in half, then narrowing the search for a zero 10
one of the two subintervals where a sign change of f occurs. By repeating this,
process m times, an approximation to a zero is computed that is no more than (b
= a)/2" from an actual zero. We leave the pseudocode for the bisection method
to the exercises.
For the case when fix) is a differentiable function, a more efficient method
for finding zeros of f was developed by Sir Isaac Newton in the 17th century.
Newton's method is based on constructing the tangent line to the graph of f
at an initial guess, say x,, of a zero of f. The point x, where this tangent line
crosses the x-axis is taken as the second approximation to a zero of f.This process
is then repeated at x,, yielding a third approximation, x, (see Figure 1.3)
Successive iterations yield points x,, x,, and so on, given by the formula (see
Exercise 1.16)
(1.13)2M PARTI: Introduction to Algorithms
FIGURE 1.3
Newton's method
for finding a zero of
a differentiable
function f
Curiously, when applied to the polynomial x? — a, Newton's method yields
exactly the Babylonian square root algorithm. In general, certain conditions
need to be imposed on the function fand starting point x, to guarantee that the
points x, given by Formula (1.1.3) actually converge to a zero.
1.1.4 Evaluating Polynomials
Abasic problem in mathematics is evaluating a polynomial p(x) = a,x" + a,_,x"-!
+ a,x + qyat a particular value v of x. The most straightforward solution to
the problem is to compute éach term a,x‘ independently, where i = 1, ....m, and
sum the individual terms. However, when computing each power v', where
is more efficient to obtain v! by multiplying the already calculated
v=" by v. This simple observation leads to the following algor
function Polyéval(a[0:n, v)
Input: o{0:n} (an array of real numbers), v (a real number)
Output: the value of the polynomial 0,1” + ,_.x0°1 +--+ a,x +a, ate =v
Sum + a0}
Product — 1
fori <1 ton do
Product — Product + v
Sum « Sum + ali) + Product
endfor
return(sum)
end Fovéval
PolyEval clearly does 2n multiplications and n additions, which might seem.
the best we can do, However, there is a simple algorithm for polynomial evalua-
tion that cuts the number of multiplications in half. Although the algorithm is
called Horner’s rule because W. G. Horner popularized it in 1819, the algorithm.
was actually devised by Sir Isaac Newton in 1699. Horner's rule for polynomial1.2
CHAPTER 1: Introduction and Preliminaries M13
evaluation is based on a clever parenthesizing of the polynomial. For example, a
fourth-degree polynomial ax* + aye? + a,x + a,x + dy is rewritten as
(lay #24 ay) ex +a) ext ay) ext ay
This rewriting can be done for a polynomial of any degree n, yielding the follow-
ing algorithm:
function HornerEval(a[0:nl, v)
Input: [0:7] (an array of real numbers), v (a real number)
Output: the value of the polynomial a,x° + 9,_ x0"! # =: #ayx + ag atx
‘Sum « afn]
for’ (De = Ela (2.33)
Formula (2.5.3) for A(n) is rarely used directly because it is simply too cum-
bersome to examine each term in .% directly. Also, the growth of the summa-
tion as a function of the input size 7 is usually hard to estimate, much less
calculate exactly. Thus, when analyzing the average complexity of a given al-
gorithm, we usually seek closed-form expressions or estimates for A(n), or for-
mulas that allow some gathering of terms in Formula (2.5.3). For example, let
p, denote the probability that the algorithm performs exactly iba:
operations;
that isp, = p(t = i). Then
‘9
An) = Ee) = Deine 254)
Formula (2.5.4) is often useful in practice and follows from Formula (2.5.3)
by simply gathering up, for each ibetween 1 and W(m), all the inputs such that
PD =i.
Frequently, there is a natural way to assign a probability distribution on «4,
For example, when analyzing comparison-based sorting algorithms, it is com-
mon to assume that each of the n! permutations (or orderings) of a list of size
is equally likely to be input to the algorithm. The average complexity of any
comparison-based sorting algorithm is then the sum of the number of compar-
isons generated by each of the n! permutations divided by n!. In practice, itis not
feasible to examine each permutation individually because n! simply grows too
fast. Fortunately, there are techniques that allow us to calculate this average
without resorting to permutation-by-permutation analysis.
Often it is possible to find a recurrence relation expressing (rt) in terms of
one or more of the values A(m) with m
2.6.1 Linear Search
We implement a linear search as a function that returns the first position in a list
(array of size n) L{0:n— 1] if the search element X occurs or returns -1 if X is not
in the list.
function LinearSearch(L(0:n ~ 1],X)
§ Input: {0:0 ~ 1] (list of size), X (a search itern)
2 Output: returns index of first occurrence of Xin the lst, or ~1 if is notin the list
fori Oton-1do
ifX=L{7] then
return()
endif
endfor
return(-1),
end LinearSearch
The basic operation of LinearSearch is the comparison of the search element to
alist element. Clearly, LinearSearch performs only one comparison when the input
Xis the first element in the list, so the best-case complexity is B(n) = 1. The most
‘comparisons are performed when X is not in the list or when X occurs in the last
position only. Thus, the worst-case complexity of LinearSearch is W(t) = 1.
‘To simplify the discussion of the average behavior of LinearSearch, we assume
that the search element X is in the list L{0:7 ~ 1] and is equally likely to be found
in any of the 1 positions. Note that i comparisons are performed when X is found
at position / in the list. Thus, the probability that LinearSearch performs i compar-
isons is given by p, = 1/n. Substituting these probabilities into (2.5.4) yields
wy (next) nt
A(n) = Divs = > ame aioe
A” Bia
(2.6.1)
Formula (2.6.1) is intuitively correct because under our assumptions, X is
equally likely to be found in either half of the list. To see that Formula (2.6.1)
truly reflects average behavior, suppose that we run LinearSearch m times (m
large) with a fixed list £{0:1 - 1] and with search element X being randomly cho-
sen as one of the list elements, Let m, denote the number of runs in which X was
found at position i. Then the total number of comparisons performed over the m
runs is given by lm, + 2m, + ... + nm,. Dividing this expression by m gives the
average number 4,,(1t) of comparisons over the entire m runs, as follows:
Aan) = 1(S) +2(%) + +0(%). ayCHAPTER 2: Design and Analysis Fundamentals ll 35
Note that for large values of m, the ratios m,/m occurring in Formula (2.6.2) ap-
proach the probability y, that X occurs in the ith position, Because we have as-
sumed that X is equally likely to be found in any of the 7 positions, each m, is
approximately equal to m/n, Hence, substituting m, = m/n into Formula (2.6.2)
yields
L+2ttn ntl
Aa(n) = AAS :
(n). (2.6.3)
‘To summarize, LinearSearch has best-case, worst-case, and average complex-
ities 1, n, and (m + 1)/2, respectively. The best-case complexity is a constant in-
dependent of the input size n, whereas the worst-case and average complexities
are both linear functions of n. For simplicity, we say that LinearSearch has constant
best-case complexity and linear worst-case and average complexities.
We calculated A(x) for LinearSearch under the assumption that the search el-
cement X is in the list. In Chapter 6, we will examine a more general situation
where we assume that X is in the list with probability p, 0 = p = 1
2.6.2 Binary Search
LinearSearch assumes nothing about the order of the elements in the list; in fact,
it is an optimal algorithm when no special order is assumed. However, Lin-
earSearch is not the algorithm to use when searching ordered lists, atleast when
direct access to each list element is possible (as with an array implementation of
the list). For example, if you are looking up the word riddle in a dictionary, and
you initially open the dictionary to the page containing the word middle, then
you know you only need to search for the word in the pages that follow. Simi-
larly, if you are looking up the word fiddle instead of riddle, then you need only
search for the word in the pages preceding the page containing middle, This sim-
ple observation is the basis of BinarySearch.
‘The objective of a binary search is to successively cut in half the range of in-
dices in the list where the search element X might be found. We assume that the
list is sorted in nondecreasing order. By comparing X with the element L[ymid] in
the middle of the list, we can determine whether X might be found in the first
half of the list or the second half. We have three possibilities:
X= L{mid), —_Xis found;
X L{mid}, search for Xin L[mid + 1:n— 1).
This process is repeated (if necessary) for the relevant “half list.” Thus, the num-
ber of elements in a sublist where X might be found is being cut roughly in half36M PARTI: Introduction to Algorithms
for each repetition. When we cut a sublist L{/ow:high] in half, if the size of the
sublist is even, then we take the midpoint index to be the smaller of the two
middle indices, so that mid = |(low + high)/2|, The following pseudocode im-
plements a binary search as a function with the same parameters and output as
LinearSearch, except that the list is assumed to be sorted in nondecreasing
order:
function BinarySearch(L{Oin ~ 1].%)
Input: L{0:n ~ 1] (an atay of list elements, sorted in nondecreasing order)
X (a search item)
Output: retuins the index of an occurrence of Xin the lst, or —1 ifXis not inthe list
Found « false.
low —0
high 9-1
while .not. found .and. low = high do
mid «|(low + high)/2|
it X= L(mil then
Found «true.
else
if X Maxvalue then
Marvalue < L{] //update MaxVolue
endif
endfor
return(MaxValue)
end Max
‘When analyzing the function Max, we choose a comparison between list ele-
‘ments (L{i] > MaxValue) as our basic operation. The only other operation per-
formed by Max is updating MaxValue. However, for any input list, the number of
comparisons between list elements clearly dominates the number of updates of
‘MaxValue, which justifies our choice of basic operation.
Note that Max performs n~ 1 comparisons for any input list of size n. Thus,
the best-case, worst-case, and average complexities of Max all equal n ~ 1. In
Chapter 3, we will use lower-bound theory to show that any comparison-based
algorithm for finding the maximum value of an element in a list of size must
perform at least 2 - 1 comparisons for any input, so that Max is an optimal algo-
rithm.
To find the minimum value in a list, we can use an analogous algorithm Min.
Sometimes itis useful to determine both the maximum and the minimum values
in a list L{0:n— 1}. An algorithm MaxMin! for solving this problem successively
invokes Max and Min. Clearly, MaxMin! has best-case, worst-case, and average
complexities all equal to 2n - 2. The following algorithm based on a single se-
quential scan through the list results in some improvement to the best-case and
average complexities:CHAPTER 2: Design and Analysis Fundamentals Ml 39
procedure MaxMin2(L(Oin ~ 1), MaxValue, MinValue)
Input: L[0:0 ~ 1} (alist of size n)
Output: MaxValue, MinValue (maximum and minimum values occuring in L[0:0 ~ 1})
Maxvalue — L[0]
MinVolue « L[0]
fori 1 ton-1 do
if L{] > Maxvélue then
Moxvoalue & Lf
else
if]. Maxvalue then MarVolue « b endit
endfor
else Hinis odd
MoxValue « L(0}; MinValue « L[0);
fori <1 ton —2 by 2do
MM(L{A, Li + 1], Bo)
if'a Maxvalue then MaxValue < b endif
endfor
endif
end MaxMin3
For n even, L{0] and L[1] are compared, and then the first for loop of
‘MaxMin3 performs 3(n - 2)/2 comparisons. For n odd, there is no initial compar-
ison, and the second for loop performs 3(7~ 1)/2 comparisons. Thus, the best-
case, worst-case, and average complexities of MaxMin3 for input size m are all
equal tof 3n/21—2.
We now design and analyze some basic comparison-based sorting algo-
rithms, We start with the simple sorting algorithm insertionsort. While inser-
tionsort is not efficient (in the worst case) for large lists, we shall see that it does
have its uses.
2.6.5 Insertionsort
Array Implementation of Insertionsort_Insertionsort sorts a given list L{0:n— 1] by
successively inserting the list element [i] into its proper place in the sorted list
L{0:i], i= 1, ..., m1. It works like a card player who inserts a newly dealt card
into a previously dealt and ordered hand that was already put in order. The card
player starts a scan at one end of the hand and stops at a place where the newFIGURE 2.2,
Action of
InsertionSort
(backward scan)
fora lst of size 6
‘CHAPTER 2: Design and Analysis Fundamentals MH 41
card can be inserted and still maintain an ordered hand. This scan can start at ei-
ther the low end of the hand (forward scan), or at the high end of the hand
(backward scan). The card player has no reason (other than a personality quirk)
to prefer one scan over the other. However, with insertionsort, there are several
reasons for preferring one scan over the other, depending on the situation.
Given the list {0:1 ~ 1], clearly the sublist consisting of only the element
L{0] is a sorted list. Suppose (possibly after reindexing) we have a list L where
the sublist L{0:/— 1] is already sorted. We can obtain a sorted sublist L{0:i] by in-
serting the element L{i] in its proper position. In a backward scan, we succes
sively compare L{i] with L{i - 1], L{i - 2], and so forth, until a list element
position] is found that is not larger than L[i]. We can then insert L{iJat L [posi
tion+ 1}. In a forward scan, we successively compare Li] with L[0], L[1]. and so
forth, until list element L [position] is found that is not smaller than L(#). L{i] can
then be inserted at L{position|
Figure 2.2 demonstrates the action of the backward scan version of inser
tionsort for a list of size 6,
35]40] 1 [23]61] 8
40[53[1 3[er] 8
a
Hta2
PARTI: Introduction to Algorithms
The following pseudocode for insertionsort uses a backward scan.
procedure inserionsor(.[07 ~ 1)
Input: (01 ~ 1] (@ list of size n)
Output: (On ~ 1] (Sorted in nondecreasing order)
fori 1 tom — 1 do //insertL{] in its proper poston in L[04 ~ 1]
Current « Uf
position i
while position = 0 and. Current < L{position} do
11Current must precede L{position}
Uposition+ 1] « L{position} //bump up L{position}
position « position — 1
endwhile
Ipositon + 1's now the proper position for Current = Lf]
Uposition +1] — Current
endfor
fend InsertionSort
When analyzing InsertionSort, we choose comparison between list elements
(Current < L{position}) as our basic operation. For any input list of size n, the
outer loop of Insertionsort is executed » ~ 1 times. If the input list is already
sorted in nondecreasing order, then the inner loop is iterated only once for each.
iteration of the outer loop. Hence, the best-case complexity of Insertionsort is
given by
Bin) = n=1 (2.6.5)
‘The worst-case complexity occurs when the inner loop performs the maximum
number of comparisons for each value of the outer loop variable i. For a given i,
this occurs when L{i] must be compared to each element L{i~ 1], Lli- 2]...
{0}, so that i 1 comparisons are performed in the inner loop. This, in turn, oc
curs when the list isin strictly decreasing order. Since i varies from 1 ton ~ 1, we
have
n(n = 1)
Won) = +2 tm # (n= 1) = (2.6.6)
Thus, W(n) for InsertionSort is quadratic in the input size m.
In Chapter 6, we will show that the average complexity (7) for InsertionSort
is about half of W(n) and therefore is also quadratic in n. We can see why this
is true, intuitively, when we recognize it is teasonable that when inserting the
(i + 1) element L{/] into its proper position in the list £{0], .... L{f- 1}, onCHAPTER 2: Design and Analysis Fundamentals ll 43
average il2 comparisons will be made. Hence, itis reasonable that A(n) should be
about 1 + (1/2)[2 +3 + #m=1] = 1+ (1/2)[ln—1)/2—1].
Because of its quadratic complexity, InsertionSort is impractical to use for
sorting general large lists. However, InsertionSort does have five advantages:
1. InsertionSort works quickly on small lists. Thus, InsertionSort is often used as a
threshold sorting algorithm used in conjunction with other sorting algo-
rithms like mergesort or quicksort.
2. InsertionSort works quickly on large lists that are close to being sorted in the
sense that no element has many larger elements occurring before it in the
list. This property of InsertionSort makes it useful in connection with other
sorts such as ShellSort (see Chapter 5).
3. InsertionSort is amenable to implementation as an on-line sorting algorithm.
In an on-line sorting algorithm, the entire list is not input to the algorithm in
advance; rather, elements are added to the list over time. On-line sorting al-
gorithms are required to maintain the dynamic list in sorted order.
4, InsertionSortis an in-place sorting algorithm. A sorting algorithm with input
parameter L{0:n~ 1] is called in-place if only a constant amount of memory is,
used (for temporary variables, loop control variables, sentinels, and so forth)
in addition to that needed for L.
5. InsertionSortis a stable sorting algorithm in the sense that it maintains the rel-
ative order of repeated elements. More precisely, an algorithm that sorts a
list £{0:n~ 1] into nondecreasing order is called stable if, given any two ele-
ments Li], Lj], with i
procedure MergeSort(L(0:n — 1], low, high) recursive
Input: L[0:n — 1] (an array of n list elements)
low, bigh (indices of 0:1 — 1})
Output: low high] (subaray sorted in nondecreasing order)
iflow < high then
‘mid & ((low + high)/2)
‘MergeSort(L(0:0 ~ 1), low, mic)
MergeSort(L[0:n ~ 1], mid+1, high)
Merge({(0:n = 1) low, mid, high)
endif
end MergeSort
In the following pseudocode for the procedure Merge called by MergeSort, an.
auxiliary array Temp is used to aid in the merging process. Merge utilizes two
pointers CurPos! and CurPos2, which refer to the current positions in the already
sorted sublists L{low:x] and L[x+I:high], respectively. Merge also uses a third
pointer Counter, which points to the next available position in Temp. CurPos!,
CurPos2, and Counter are initialized to low, x + 1, and low, respectively. During
each iteration of the while loop in Merge, the elements in the positions GurPos!
and CurPos2 are compared, and the smaller element is written to the position
Counter in Temp. Then Counter and either CurPos! or CurPos2, depending on which
‘one points to the element jyst written to Temp, are incremented by 1. When one
of the sublists has been completely written to Temp, the remaining elements in
the other sublist are written to Temp. Then Temp contains a sorting of L{/ow:high],
and the algorithm terminates after copying Temp{low:high] to L{low:high]. The
pseudocode for Merge follows:
procedure Merge({|0:n ~ 1, low, x, high)
Input: L[0:1 ~ 1] (an aray of list elements),
low, x, high (indices of array L[0:n ~ 1]; sublists Low, Let :high] are
assumed to be-sorted in nondecreasing order)
‘Output: Llowhigh] (sublst sorted in nondecreasing order)
Curost < low Hisiialize pointers
CurPos2 @x+1
Counter low
while CurPos? x then Iicopy remaining elements
‘| for k « CurPos2 to high do —_//in appropriate sublist //to Temp
Temp{Counter] « Lk]
Counter « Counter +1
endfor
else
for k CurPos! tox do
Temp{Counter} Lk]
Counter « Counter + 1
endfor
endif
for k « low to high do copy Templlow:high] to Llow:high]
Lk] Temp
endfor
end Merge
The tree of recursive calls to MergeSort is illustrated in Figure 2.4. A node in
the tree is labeled by the values fow, mid, and high involved in the call to Merge.
(In leaf nodes, mid is not computed, as indicated by the symbol *.) Initially, low =
O and high = 9. The path around the tree shown in Figure 2.4 indicates how the
recursion resolves. Following this path amounts to a postorder traversal of the
tree (see Chapter 4), where visiting a node corresponds to a call to Merge.
‘We now analyze the complexity of MergeSort, beginning with the worst-case
complexity. Note that each call to Merge for merging two sublists of sizes m, and
‘m, performs at most m, + m, - 1 comparisons. When we consider the tree of48 ME PARTI: Introduction to Algorithms
recursive calls to MergeSort for a list of size m (see Figure 2.4), we see that the leaf
nodes do not generate calls to Merge, whereas each internal node generates a sin-
gle call 1o Merge. At each level of the tree, the total number of comparisons made
by all the calls to Merge is at most 1. The depth of the tree of recursive calls is
Tlog,n' (see Exercise 2.38). It follows that MergeSort performs at most rf tog,y1
comparisons for any list of size n, so that W(n) = nflog,rr.
To find B(n), we note that each call to Merge for merging two sublists of sizes,
m, and m,, respectively, performs at least min(m,.m,) comparisons. We again
consider the tree of recursive calls to MergeSort for a list of size n. Except for the
last two levels, at each level of the tree of recursive calls, the total number of
comparisons made by all the calls to Merge is at least n/2. It follows that MergeSort
performs at least (7/2) ([log,n]~ 1) comparisons for any list of size n, so that Birt)
= (1/2) (log,r]= 1). Thus, we have
(12) (og, ]— 1) = Bin) = Aim) = W(n) < nf logyn.
Because B(n), A(n), and W(n) are all squeezed between functions that are
very close to (11/2)log,¥ and nlog,, for asymptotic analysis purposes (where we
ignore the effect of positive multiplicative constants), we simply say that these
quantities have mlogy complexity (see Chapter 3). Notice that we have omitted
the base in the log because change of base formulas show that logarithm func-
tions to two different bases differ by a constant. It turns out that this type of com-
plexity is optimal for A(t) and W(n) for any comparison-based sorting algorithm.
By examining Figure 2.4, we can easily come up with a bottom-up version
of MergeSort. At the bottom level of the tree, we are merging sublists of single ad-
jacent elements in the list. However, as the path around the tree indicates, for a
given input list L{0:— 1], MergeSort sorts the list L{0:mid] before going on to any
of the sublists of L[mid-+1, n ~ 1]. By contrast, a bottom-up version begins by di-
viding the list into pairs of adjacent elements, L{0]:L[1], L{21:L(3]. and so forth.
Next, these adjacent pairs are merged yielding the sorted lists L{0:1], L[2:3]. and
so forth. The process is repeated by merging the adjacent pairs of sorted two-
element sublists, L{0:1]:L{2:3], L[4:5]:L{6:7], and so forth. Continuing this
process, we arrive at the root having sorted the entire list L{0:m~ 1]
Figure 2.5 shows a tree representing this bottom-up merging of adjacent
sublists. Each node represents a call to Merge with the indicated values of low,
‘mid, and high. An asterisk (*) denotes the nodes in which a call to Merge is not
made. Note that the sublists and resulting tree are quite different from the sub-
lists generated by the tree of recursive calls of MergeSort given in Figure 2.4. All
the calls to Merge for a given level are completed before we go up to the next
level. The pseudocode for the nonrecursive version of MergeSort based on Figure
2.5 is left to the exercises.FIGURE 2.5,
Bottom-up
nonrecursive
(MergeSort for a lst
of size 10,
CHAPTER 2: Design and Analysis Fundamentals Ml 49)
0.7.9 @ noaall
to Merge.
0.01 223 44,5 66,7 389
2.6.7 Quicksort
We now discuss the comparison-based sorting algorithm quicksort, discovered
by C. A. R. Hoare in 1962. Quicksort is often the sorting algorithm of choice be-
cause of its good average behavior. Like mergesort, quicksort, is based on divid-
ing a list into two sublists (actually two sublists of a rearrangement of the original,
list), and then sorting the sublists recursively. The difference between the two
sorting strategies is that mergesort does most of the work in the combine
(merge) step, whereas quicksort does most of the work in the divide step. Quick-
sort is also an in-place sort, as opposed to mergesort, which used an auxiliary
array for merging (in-place versions of mergesort can be written, but they are
complicated).
In quicksort, the division into sublists is based on rearranging the list
L{low:high] with respect to a suitably chosen pivot element x. The list L{low:high] is
rearranged so that every list element in L{Jow:high] preceding x (having smaller
index than the index of x) is not larger, and every element following x in
L{low:high] is not smaller. For example, for the list (23,55,11,17,53.4) and pivot
element x = 23, the rearrangement might be (17,11,4,23,55,53). After the re-
arrangement, the pivot element x occupies a proper position in a sorting of the
list. Thus, if the sublists on either side of xare sorted recursively, then the entire
list will be sorted with no need to invoke an algorithm for combining the sorted
sublists.
Procedure QuickSort sorts L{low-high] into nondecreasing order by first call-
ing an algorithm Partition that rearranges the list with respect to pivot element
X= L{low], as previously described. Partition assumes that the element L{high*+ 1]
is defined and is at least as large as L{low]. The output parameter position of Par-
tition returns the index where xis placed. To sort the entire list L{0:m~ 1], Quick-
Sort would be called initially with iow = 0 and hight =n —1PART I: Introduction to Algorithms
=
procedure QuickSort(L(0:n ~ 1, ow, high) recursive
Input: L{0:n ~ 1] (en aay ofr list elements)
low, high (indices of L(O1n ~ 1})
/ffor convenience, Ln] is assumed to have the sentinel value +=)
‘Output: L[low:high] sorted in nondecreasing order
if high > low then
Porttion(L(0:n — 1], low, high, position)
QuickSort(L(0:n ~ 1], low, pasition — 1)
i QuickSort(L[0:n — 1], position+ 1, high)
P endif
end QuickSort
The algorithm Partition is based on the following clever interplay between
two moving variables moveright and moveleft, which contain the indices of ele-
ments in L and are initialized to low + 1 and high, respectively.
while moveright < moveleft do
‘moveright moves to the right (one index ata time) untl it assumes the index of a
list element not smaller than x then it stops.
moveleft moves to the left (one index ata time) untl it assumes the index ofa lis,
element not larger than x, then it stops.
if moveright < moveleft then
interchange L{moveright] and L|movelef]
endif
endwhile
To guarantee that moveright actually finds an element not smaller than x, we
assume that L{high+ 1] is defined and is not smaller than L{/ow]. As commented
in the pseudocode, this is arranged by introducing a sentinel value Lr] = + ».
We leave it as an exercise to check that the condition L[high+1] = L{Jow] is then
automatically guaranteed for all subsequent calls to Partition by QuickSort. Of
course, Partition could be written with explicit checking that moveright does not
run off the list. However, this checking requires additional comparisons, and we
prefer to implement the preconditioning. Figure 2.6 illustrates the movement of
‘moveright (mr) and moveleft (ml) for a sample list L{0:6]. The pseudocode for Par-
tition follows:
procedure Parttion(.[0:n — 1], low, high, position)
Input: [0:0 ~ 1] (an aay of list elements)
Jow, high (indices of L{07 ~ 1))
E ‘ulhigh +s assumed defined and 2 [low