Java Collections An Introduction to Abstract Data Types Data Structures and Algorithms 1st Edition David A. Watt all chapter instant download
Java Collections An Introduction to Abstract Data Types Data Structures and Algorithms 1st Edition David A. Watt all chapter instant download
https://ebookultra.com
https://ebookultra.com/download/java-collections-
an-introduction-to-abstract-data-types-data-
structures-and-algorithms-1st-edition-david-a-
watt/
https://ebookultra.com/download/data-structures-and-algorithms-in-
java-4th-edition-michael-t-goodrich/
ebookultra.com
https://ebookultra.com/download/data-structures-and-algorithms-in-
java-6th-edition-michael-t-goodrich/
ebookultra.com
https://ebookultra.com/download/growing-algorithms-and-data-
structures-4th-edition-david-scuse/
ebookultra.com
https://ebookultra.com/download/ai-algorithms-data-structures-and-
idioms-in-prolog-lisp-and-java-6th-edition-george-f-luger/
ebookultra.com
Learning F Functional Data Structures and Algorithms 1st
Edition Masood
https://ebookultra.com/download/learning-f-functional-data-structures-
and-algorithms-1st-edition-masood/
ebookultra.com
https://ebookultra.com/download/data-structures-algorithms-in-go-1st-
edition-hemant-jain/
ebookultra.com
https://ebookultra.com/download/learning-javascript-data-structures-
and-algorithms-2nd-edition-loiane-groner/
ebookultra.com
https://ebookultra.com/download/data-structures-and-algorithms-using-
python-1st-edition-rance-d-necaise/
ebookultra.com
https://ebookultra.com/download/concise-notes-on-data-structures-and-
algorithms-ruby-edition-christopher-fox/
ebookultra.com
Java Collections An Introduction to Abstract Data Types
Data Structures and Algorithms 1st Edition David A.
Watt Digital Instant Download
Author(s): David A. Watt, Deryck F. Brown
ISBN(s): 9780471899785, 047189978X
Edition: 1
File Details: PDF, 15.45 MB
Year: 2001
Language: english
Java Collections
This page intentionally left blank
Java Collections
An Introduction to Abstract
Data Types, Data Structures
and Algorithms
David A. Watt
University of Glasgow
Deryck F. Brown
The Robert Gordon University
All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or
transmitted in any form or by any means, electronic, mechanical, photocopying, recording,
scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or
under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court
Road, London WIT 4LP, UK, without the permission in writing of the Publisher, with the exception
of any material supplied specifically for the purpose of being entered and executed on a computer
system, for exclusive use by the purchaser of the publication.
Neither the authors) nor John Wiley & Sons, Ltd accept any responsibility or liability for loss or
damage occasioned to any person or property through using the material, instructions, methods or
ideas contained herein, or acting or refraining from acting as a result of such use. The authors)
and Publisher expressly disclaim all implied warranties, including merchantability of fitness for
any particular purpose.
Designations used by companies to distinguish their products are often claimed as trademarks. In all
instances where John Wiley & Sons Ltd is aware of a claim, the product names appear in initial capital
or capital letters. Readers, however, should contact the appropriate companies for more complete
information regarding trademarks and registration.
John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA
John Wiley & Sons Australia Ltd, 33 Park Road, Milton, Queensland 4064, Australia
John Wiley & Sons (Asia) Pte Ltd, 2 dementi Loop #02-01, Jin Xing Distripark, Singapore 129809
John Wiley & Sons (Canada) Ltd, 22 Worcester Road, Etobicoke, Ontario M9W 1L1
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print
may not be available in electronic books.
•
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Cover Image — Fondaco dei Turchi, Venice (w/c) by John Ruskin (1819–1900)
Ruskin Museum, Coniston, Cumbria, UK/Bridgeman Art Library
Printed and bound in Great Britain by Biddies Ltd, Guildford and King's Lynn
This book is printed on acid-free paper responsibly manufactured from sustainable forestry
in which at least two trees are planted for each one used for paper production.
Contents
Preface xi
1 Introduction i
I. I Historical background I
1.2 Algorithms and programs 6
1.3 Abstract data types and data structures 7
Summary 9
Exercises 9
2 Algorithms 11
2.1 Principles of algorithms 1 I
2.2 Efficiency of algorithms )4
2.3 Complexity of algorithms i7
2.4 Recursive algorithms 24
Summary 31
Exercises 3i
4.3 Deletion 81
4.4 Searching 86
4.5 Merging 88
4.6 Sorting 91
Summary 94
Exercises 95
Index 540
This page intentionally left blank
Preface
This book has four major themes: algorithms, data structures, abstract data types
(ADTs), and object-oriented methods. All of these themes are of central importance in
computer science and software engineering.
The book focuses on a number of important ADTs - stacks, queues, lists (sequences),
sets, maps (tables), priority queues, trees, and graphs - that turn up again and again in
software engineering. It uses these ADTs to motivate the data structures required to
implement them - arrays, linked lists, search trees, hash tables, and heaps - and algo-
rithms associated with these data structures - insertion, deletion, searching, merging,
sorting, and traversal. The book shows how to implement these data structures and
algorithms in the object-oriented programming language Java.
The book's typical approach is to introduce an ADT, illustrate its use by one or two
example applications, develop a 'contract' specifying the ADT's values and operations,
and finally consider which data structures are suitable for implementing the ADT. This
approach clearly distinguishes between applications and implementations of the ADT,
thus reinforcing a key software engineering principle: separation of concerns. The book
motivates the study of data structures and algorithms by introducing them on a need-to-
know basis. In other words, it adopts a practical (software engineering) approach to the
subject, rather than a theoretical one. It does not neglect the important topic of algorithm
analysis, but emphasizes its practical importance rather than the underlying mathematics.
Object-oriented methods have emerged as the dominant software technology during
the last decade, and Java has become the dominant programming language in computer
science education. These developments call for a fresh approach to the teaching of
programming and software engineering. This book takes such a fresh approach, using
object-oriented methods from the outset to design and implement ADTs. It avoids the
mistake of reworking an older book on algorithms and data structures that was written
originally for Pascal or C.
Several of the ADTs introduced in this book are directly supported by the Java 2
collection classes. Programmers will naturally prefer to use these collection classes rather
than design and implement their own. Nevertheless, choosing the right collection classes
for each application requires an understanding of the properties of different ADTs and
XI
xii Preface
their alternative implementations. This book aims to give readers the necessary under-
standing.
Readership
This book can be read by anyone who has taken a first programming course in Java.
The book is designed primarily as a text for a second-level course in algorithms and
data structures, in the context of a computer science program in which Java is the main
programming language. In terms of the 1991 and 2001 ACM Computing Curricula, the
book covers the knowledge units summarized in Table P. 1.
The book is also suitable for practicing software engineers who have just learned Java
and who now wish to gain (or refresh) the knowledge of algorithms and data structures
required to make effective use of the Java 2 collection classes.
The detailed prerequisites of this book are as follows:
• A knowledge of fundamental Java programming topics - expressions, statements,
objects, and classes - is essential. A knowledge of more advanced topics - inheritance,
exceptions, interfaces, inner classes - is desirable but not essential before reading this
book. Readers unfamiliar with the latter topics should refer to Appendix B whenever
necessary.
• Certain mathematical topics - powers, logarithms, series summations, and recurrences
- are needed to analyze the efficiency of algorithms. Most of these mathematical topics
are taught in secondary/high schools. Readers unfamiliar with any of these topics
should refer to Appendix A whenever necessary.
Outline
Chapter 1 introduces two of this book's major themes: algorithms and data structures.
Chapter 2 takes a closer look at algorithms, showing how we can analyze their efficiency
and introducing recursive algorithms.
Chapters 3 and 4 review two simple and ubiquitous data structures - arrays and linked
lists - together with their associated insertion, deletion, searching, merging, and sorting
algorithms. Later in the book, Chapters 10, 12, 13, and 16 introduce more sophisticated
data structures - binary search trees, hash tables, heaps, AVL-trees, and B-trees - again
together with their associated algorithms.
Chapter 5 takes a close look at the idea of an abstract data type (ADT). It introduces the
notion of a 'contract' that specifies the values and operations of an ADT without commit-
ting to a particular representation. Chapters 6 (stacks), 7 (queues), 8 (lists), 9 (sets), 11
(maps), 13 (priority queues), 14 (trees), and 15 (graphs) introduce a variety of collection
ADTs together with alternative implementations of each. The list, set, and map ADTs
covered in Chapters 8,9, and 11 are in fact simplified versions of the corresponding Java
2 collection classes.
Chapter 17 concludes the book. It discusses the role of ADTs in object-oriented design.
It also discusses the distinction between heterogeneous and homogeneous collections,
and an extension of Java that supports generic homogeneous collections.
Appendix A summarizes the mathematical topics needed to analyze the efficiency of
algorithms. Appendix B summarizes the features of Java that are most important in this
book, particularly the more advanced features that are likely to be omitted in a first
programming course. Appendix C summarizes the Java 2 collection classes.
Ideally, all the chapters of this book should be read in numerical order. Figure P.2
summarizes the dependencies between chapters. Clearly, Chapters 1-5 must be read first,
and Chapters 6-9 all rely on this introductory material. Chapter 10 depends on Chapters 6
and 9, and in turn Chapters 11,13 and 16 all depend on Chapter 10. Readers may vary the
order of some of the later chapters, as indicated by Figure P.2. Readers may also omit any
of Chapters 13–17 if time is short.
1 Introduction
2 Algorithms
10 Binary Trees
Scope
This book is an introductory text. As such, it covers several topics, such as algorithm
complexity, hash tables, and graphs, in rather less depth than would be possible
in a more advanced text. Moreover, it omits altogether a variety of interesting algo-
rithms, such as compression, encryption, geometric, linear algebra, and scheduling
algorithms.
Under Further Reading there is a list of books that do cover these more advanced
topics, together with seminal papers on topics covered by this book. There are also
bibliographic notes that suggest further reading on the topics covered by each chapter
and by the book as a whole.
This Web site contains the book's contents and preface, answers to selected exercises,
and Java code for all ADTs and case studies. New features will be added from time to
time.
Case studies
Case studies are an important feature of this book. They can be found in Chapters 6–9,11,
13-15, and 17.
Each case study leads the reader through the object-oriented design and implementa-
tion of a fair-sized application program. The presentation focuses on the selection of an
ADT appropriate for the application, and later on the choice of the most suitable imple-
mentation of that ADT (from among those presented earlier in the chapter). The object-
oriented design of the application program itself, and the development of algorithms to be
employed by the application program, are also discussed. User interface issues are
generally omitted here, however.
Complete Java source code for all case studies may be found at the companion Web
site. For some case studies, there is an application program that can be downloaded and
run. For others, there is an applet that can be run immediately using a Java-enabled Web
browser. Where appropriate, each case study comes in two versions, one using the classes
developed in this book and the other using the Java collection classes.
Exercises
Each chapter is followed by a set of exercises. Some of these exercises are drill, or simple
modifications to designs, algorithms, or implementations presented in the chapter. The
more challenging exercises are clearly marked as * (hard) or ** (harder).
Sample answers to selected exercises can be found at the companion Web site.
Acknowledgements
Most of the algorithms and data structures described in this textbook have long since
passed into computer science folklore, and are almost impossible to attribute to indivi-
duals. Instead, we acknowledge those colleagues who have particularly influenced us
through discussions on the topics of this book: David Davidson, Rob Irving, Alistair
Kilgour, John McCall, and Joe Morris. We wish to thank the Wiley reviewers for reading
and providing valuable comments on an earlier draft of this book. In addition, we would
like to thank William Teahan for his extensive and detailed comments. Finally, we are
delighted to acknowledge the assistance of the staff at John Wiley & Sons, particularly
our editors Gaynor Redvers-Mutton and Gemma Quilter.
David A. Watt
Glasgow
Deryck F. Brown
Aberdeen
August 2000
1
Introduction
B
Initially:
After step 2:
After step 4:
The greatest common divisor (GCD) of two positive integers is the largest positive
integer that exactly divides both of them. Euclid gave his name to an elegant and efficient
algorithm to solve this problem.
Euclid's algorithm is paraphrased as Algorithm 1.3. Note that (p modulo q) is the
remainder when we divide p by q.
Step 2 controls a loop. The subsidiary step 2.1 is performed repeatedly until the stated
condition arises, i.e., until q exactly divides p.
Our story now moves to Europe. The Scots scholar John Napier (1550–1617), obser-
ving that computations involving multiplication, division, powers, and roots were slow
and error-prone, invented logarithms. The logarithm of a number y is the number of times
that 10 must be multiplied by itself to give y; thus log1010 = 1, log10100 = 2 (since
102 = 100), log101000 = 3 (since 103 = 1000), and so on. Intermediate numbers have
fractional logarithms; thus log!052 = 1.716, accurate to 3 decimal places.
Logarithms (and their mechanical equivalents, slide rules) quickly became essential
tools for scientists, engineers, astronomers, and everyone else who had to do complex
calculations. They continued to be used routinely until they were superseded by electro-
nic calculators and computers in the 1960s.
The English mathematician and scientist, Isaac Newton (1643–1727), invented the
differential calculus. This gave us algorithms for differentiation (determining the slope of
a given curve) and integration (determining the area under a given curve). Differential
calculus became a keystone of mathematics and physics, and has found a vast array of
applications in engineering, weather forecasting, economics, and so on.
every day. So the team proceeded to build and use a machine, Colossus, that could break
the codes hundreds of times faster than mere humans.
The first general-purpose digital computers were built in the USA and Great Britain.
You are probably familiar with the story of computers since then. Computers are now
ubiquitous, whether as general-purpose computers or as components embedded in
devices as diverse as digital watches, kitchen appliances, and aircraft. And every one
of these computers is a machine designed to perform algorithms.
We are also surrounded by algorithms intended for performance by humans. Whether
or not a device has a computer inside it, it probably comes with a user's manual. Such
manuals contain algorithms necessary for us to use the devices effectively: to operate a
washing machine, to set a digital watch to give an alarm signal or to record lap times, to
change a car wheel, to change a light bulb, to assemble a piece of furniture from a flat-
pack, and so on.
return q;
Figure 1.9 Representation of the character string "Java" using: (a) an array, (b) a linked list.
Figure 1.10 Representation of the word set {bat, cat, mat. rat. sat) using: (a) an array. (b) a singly
linked list, (c) a binary search tree.
Of course, when you wish to write a Java program that manipulates character strings,
you neither know nor care how strings are represented. You simply declare variables of
type String, and manipulate them by calling methods provided by the Java. lang.
String class. String is a simple example of what we call an abstract data type. You
are told the name of the abstract data type, and the general properties of objects of that
type, and the names of methods for manipulating these objects. But you are not told how
these objects are represented.
Once an abstract data type has been designed, the programmer responsible for imple-
menting that type is concerned only with choosing a suitable data structure and coding up
the methods (without worrying much about the type's future applications). On the other
hand, application programmers are concerned only with using that type and calling its
methods (without worrying much about how the type is implemented).
This separation of concerns is a fundamental principle of modular software design. An
object-oriented programming language typically provides a rich library of predefined
abstract data types (classes), but the idea of an abstract data type goes well beyond that.
When an application program is being designed, a major part of the designer's job is to
identify abstract data types specialized to the requirements of the particular application.
A large application program might require tens or even hundreds of abstract data types.
Their identification by the designer helps to decompose the program into manageably-
sized modules that can be specified, coded, tested, and debugged separately. Such decom-
position is essential for successful software development.
Summary
in this chapter:
• We have introduced the idea of an algorithm, illustrating the idea with a variety of
algorithms of historical interest, and distinguishing between algorithms and programs.
• We have introduced the idea of a data structure, distinguishing between static and
dynamic data structures.
• We have introduced the idea of an abstract data type, distinguishing between the
separate concerns of the programmer who implements an abstract data type and the
programmers who use it.
In Chapters 2 and 5 we shall study algorithm and abstract data type concepts in greater
detail. In other chapters, we shall study a variety of data structures in conjunction with
the algorithms that operate on them, and we shall study a variety of abstract data types
showing which data structures can be used to implement them.
Exercises
1.1 Use Euclid's algorithm, given in Algorithm 1.3, to find the GCD of the following pairs of
numbers: 6 and 9, 12 and 18, 15 and 21, and 11 and 15.
1.2 Consider Newton's algorithm, given in Algorithm 1.6, to calculate the square root of a
number.
(a) Use this algorithm to calculate the square roots of the following numbers: 4, 6, 8 and 9. In
each case calculate your answer to an accuracy of 0.01, i.e., the absolute difference
between a and r2 is less than 0.01.
(b) Write a Java program to implement the algorithm and use it to check your answers to part
(a) above.
(c) What would happen if step 2 of the algorithm were as follows?
2. Until r2=a, repeat:
1.3 Give some examples of algorithms used in everyday life, not requiring a calculator or
computer.
1.4 Write an algorithm to perform each of the following tasks:
(a) Use an automated bank teller to withdraw cash from your account.
(b) Set the current time on your watch.
(c) Cook a frozen meal in a microwave oven.
(d) Match the pairs of socks in a bundle of freshly laundered socks.
1.5 Try to find further examples of early algorithms like the ones given in this chapter.
1.6 Devise an algorithm, similar to Algorithm 1.4, to find the roots of the general quadratic
equation ax2 + bx + c = 0. The roots are the two values of the formula (–b ± /(b2–4ac)V2a.
1.7 Consider the algorithms you wrote in Exercise 1.4. How easy would it be for each of these
algorithms to be performed by a human? by a suitable machine?
10 Java Collections
1.8 Consult the documentation for your Java development environment for the list of predefined
classes (or consult the on-line documentation at Java, sun.com/products/jdk/1. 3/
docs/api/index.html). Consider some of the classes provided from the point of view
of abstract data types, and determine the list of operations provided for each class.
2
Algorithms
Problems
Concerning problems, we can state the following principles:
• An algorithm must be designed to solve a stated problem, which is a well-defined task
that has to be performed.
• The problem must be solvable by an algorithm.
We have already (in Section 1.1) encountered a number of problems that can be solved
by algorithms. We can also pose some problems that are not solvable by algorithms. To
say that a problem is unsolvable does not just mean that an algorithm has not yet been
found to solve it. It means that such an algorithm can never be found. A human might
eventually solve the problem, but only by applying insight and creativity, not by follow-
ing a step-by-step procedure; moreover, there can be no guarantee that a solution will be
found. Here is an example of a problem that is unsolvable by an algorithm.
II
12
The problem is to predict whether a given computer program, with given input data, will
eventually halt.
This is a very practical problem for us programmers: we all occasionally write a
program that gets into a never-ending loop. One of the most famous results in computer
science is that this problem cannot be solved by any algorithm. It turns out that any
'algorithm' that purports to solve this problem will itself get into a never-ending loop. for
at least some programs that might be given to it. As we shall see later in this section. we
insist that every algorithm must eventually terminate.
If we can never find an algorithm to predict whether a given program halts with given
input data, we clearly can never find an algorithm to prove whether a given program
behaves correctly for all possible input data.
It may still be possible for a human to prove that a particular program is correct.
Indeed, this has been done for some important small programs and subprograms. But we
can never automate such proofs of correctness.
In fact, many problems in mathematics and computer science are unsolvable by algo-
rithms. In a way, this is rather reassuring: we can be sure that mathematicians and
computer scientists will never be made redundant by machines!
From now on, we shall consider only problems that are solvable by algorithms.
Algorithms
Concerning algorithms themselves, we can state the following principles:
• The algorithm will be performed by some processor, which may be a machine or a
human.
• The algorithm must be expressed in steps that the processor is capable of performing.
• The algorithm must eventually terminate, producing the required answer.
Some algorithms, as we have already seen, are intended to be performed by humans
rather than machines. But no algorithm is allowed to rely on qualities, such as insight and
creativity, that distinguish humans from machines. This suggests a definition:
The principle that the algorithm must be expressed in steps that can be performed by
the processor should now be clear. If the processor has to work out for itself what steps to
follow, then what we have is not an algorithm.
The principle that every algorithm must eventually terminate should also be clear. If it
never terminates, it never produces an answer, therefore it is not an algorithm! So an
algorithm must avoid getting into a never-ending loop.
Algorithms f3
Notation
Concerning notation, we have one fundamental principle:
• The algorithm must be expressed in a language or notation that the processor 'under
stands'.
This principle should be self-evident. We cannot expect a weaving machine, or even a
computer, to perform an algorithm expressed in natural language. A machine must be
programmed in its own language.
On the other hand, an algorithm intended for humans need not necessarily be expressed
in natural language. Special-purpose notations are commonly used for certain classes of
algorithm. A musical score is an algorithm to be performed by a group of musicians, and
is expressed in the standard musical notation. A knitting pattern is an algorithm for either
a human or a knitting machine, and is generally expressed in a concise notation invented
for the purpose.
Here we restrict our attention to computational algorithms. Even so, we have a variety
of possible notations including natural language, programming language, mathematical
notation, and combinations of these. In this book we shall express all algorithms in
English, occasionally (and where appropriate) augmented by mathematical notation.
The choice of a natural language gives us the greatest possible freedom of expression;
both programming languages and mathematical notation are sometimes restrictive or
inconvenient.
We should remember, however, that expressing an algorithm in a natural language
always carries a risk of vagueness or even ambiguity. We must take great care to express
the individual steps of the algorithm, and the order in which these steps are to be
performed, as precisely as possible.
We have already seen several examples of algorithms, in Section 1.1, which you
should now re-examine. Note the use of layout and numbering to show the structure of
an algorithm. We number the steps consecutively, and arrange one below another in the
intended order, e.g.:
1. Do this.
2. Do that.
3. Do the other.
We use indentation and the numbering system to show when one or more steps are to
be performed only if some condition is satisfied:
Likewise, we use indentation and the numbering system to show when one or more
steps are to be performed repeatedly while (or until) some condition is satisfied:
14 Java Collections
or when one or more steps are to be performed repeatedly as a variable v steps through a
sequence of values:
but this tells us nothing about the quality of the algorithm itself. And where the processor
is a modern computer, the difficulty is compounded by the presence of software and
hardware refinements - such as multiprogramming, pipelines, and caches - that increase
the average speed of processing, but make it harder to predict the time taken by an
individual algorithm.
We prefer to measure an algorithm's time efficiency in terms of the algorithm itself.
One idea is simply to count the number of steps taken by the algorithm until it terminates.
The trouble with this idea is that it depends on the granularity of the algorithm steps.
Algorithms 2.2(a) and (b) solve the same problem in 3 and 7 steps, respectively. But they
are just different versions of the same algorithm, one having course-grained (big) steps,
while the other has fine-grained (small) steps.
(a) (b)
To find the area of a triangle with sides a, b, c: To find the area of a triangle with sides a, b, c:
1. Let s = (a + b + c)/2. 1. Let s = (a + b + c)/2.
2. Let A = V(s (s – a ) ( s – b) (s – c)). 2. Let p = s.
3. Terminate with answer A. 3. Multiply p by (s – a).
4. Multiply p by (s – b).
5. Multiply p by (s – c).
6. Let A be the square root of p.
1. Terminate with answer A.
Given a nonnegative integer n, the nth power of a number b. written bn. is defined by:
b" = b X - - - X b (2.1)
(where n copies of b are multiplied together). For example:
b23 = b x b x b
b 1== b X b
b b
Algorithm 2.3 (the 'simple' power algorithm) is based directly on definition (2.1). The
variable p successively takes the values 1, b, b2, b3, and so on - in other words, the
successive powers of b. Program 2.4 is a Java implementation of Algorithm 2.3.
Let us now analyze Algorithm 2.3. The characteristic operations are obviously multi-
plications. The algorithm performs one multiplication for each iteration of the loop, and
there will be n iterations, therefore:
No. of multiplications = n (2.2)
2 3
Algorithm 2.3 is fine if we want to compute small powers like b and b . but it is very
time-consuming if we want to compute larger powers like b20 and b100.
Fortunately, there is a better algorithm. It is easy to see that b20 = b10 x b10. So once
we know b10, we can compute b20 with only one more multiplication, rather than ten
more multiplications. This shortcut is even more effective for still larger powers: once we
know b50, we can compute b100 with only one more multiplication, rather than fifty.
Likewise, it is easy to see that b21 = b10 X b10 X b. So once we know b10. we can
compute b21 with only two more multiplications, rather than eleven.
Algorithm 2.5 (the 'smart' power algorithm) takes advantage of these observations.
The variable q successively takes the values b, b2, b4, b8, and so on. At the same time, the
variable m successively takes the values n, n/2, n/4, and so on (neglecting any remain-
ders) down to 1. Whenever m has an odd value, p is multiplied by the current value of q.
Program 2.6 is a Java implementation of Algorithm 2.5.
This algorithm is not easy to understand, but that is not the issue here. Instead, let us
focus on analyzing its efficiency.
First of all, note that steps 2.1 and 2.2 each performs a multiplication, but the multi-
plication in step 2.1 is conditional. Between them, these steps perform at most two
multiplications.
17
Next, note that these steps are contained within a loop, which is iterated as often as we
can halve the value of n (neglecting any remainder) until we reach zero. It can be shown
(see Appendix A.2) that the number of iterations is floor(log2n) + 1, where floor(r) is the
function that converts a real number r to an integer by discarding its fractional part,
Putting these points together:
Maximum no. of multiplications = 2 (floor(log2n) + 1)
= 2 floor(log 2 n) + 2 (2.3)
The exact number of multiplications depends on the value of n in a rather complicated
way. For n = 15 the actual number of multiplications corresponds to (2.3), since halving
15 repeatedly gives a series of odd numbers; while for n = 16 the actual number of
multiplications is smaller, since halving 16 repeatedly gives a series of even numbers.
Equation (2.3) gives us the maximum number of multiplications for any given n, which is
a pessimistic estimate.
Figure 2.7 plots (2.2) and (2.3) for comparison. The message should be clear. For
small values of n, there is little to choose between the two algorithms. For larger values of
/?., the smart power algorithm is clearly better; indeed, its advantage grows as n grows.
multipli-
cations
50–
simple powery
algorithm
40
30
20–
10-
0
10 20 30 40 50 n
Figure 2.7 Time efficiency of the simple and smart power algorithms.
might take much more time than others. All else being equal, we prefer the algorithm
whose time requirement grows most slowly with n.
If we have a formula for an algorithm's time requirement, we can focus on its growth
rate as follows:
• Take the fastest-growing term in the formula, and discard all slower-growing terms.
• Discard any constant factor in the fastest-growing term.
The resulting formula is called the algorithm's time complexity. We define space
complexity similarly.
20 lava Collections
40–
30-
20–
10-
10 20 30 40 n
Figure 2.8 shows that an O(log n) algorithm is inherently better than an O(n) algorithm.
Regardless of constant factors and slower-growing terms, the O(log n) algorithm will
eventually overtake the O(n) algorithm as n grows.
We have now introduced the O-notation for algorithm complexity. The notation O(X)
stands for 'of order X', and means that the algorithm's time (or space) requirement grows
proportionately to X. X characterizes the algorithm's growth rate, neglecting slower-
growing terms and constant factors. In general, X depends on the algorithm's input data.
Table 2.9 summarizes the most common time complexities. These are common
enough to have acquired verbal descriptions: for example, we say that the simple
power algorithm is a linear-time algorithm, while the smart power algorithm is a log-
time algorithm. The complexities are arranged in order of growth rate: 0(1) is the slow-
est-growing, and 0(2") is the fastest-growing.
It is important to develop our intuitions about what these complexities tell us, in
practical terms, about the time efficiency of algorithms. Table 2.10 shows some numer-
ical information, and Figure 2.11 shows the same information graphically. As we have
already noted, log n grows more gradually than n, so an 0(log n) algorithm is better than
an O(n) algorithm that solves the same problem. Of course, the constant 1 does not grow
at all, so an 0(1) or constant-time algorithm is best of all - if we can find one!
Now study the growth rate of n log n. This grows more steeply than n, so an O(n)
algorithm is better than an O(n log n) algorithm that solves the same problem.
Now study the growth rates of n2 and n3 . These grow ever more steeply, so an
O(n log n) algorithm is better than an O(n2) algorithm, which in turn is better than
an 0(«3) algorithm, that solves the same problem. One way to look at these algorithms
is this: every time n is doubled, O(n 2 ) is multiplied by four and O(n3) is multiplied by
eight. And every time n is multiplied by 10, O ( n 2 ) is multiplied by 100 and O(n3) is
multiplied by 1000! These numbers are discouraging. If the best algorithm we can find
is O(n2) or O(n 3 ), we have to accept that the algorithm will rapidly slow down as n
increases. Such an algorithm is often too slow to be of practical use.
While n2 and n3 grow steeply, 2" grows at a stupendous rate. Every time n is incre-
mented by 10, 0(2") is multiplied by over 1000! As n is increased from 10 to 20 to 30 to
40, 2" grows from a thousand to a million to a billion to a trillion. If the algorithm
performs 2" operations at the rate of a million per second, its time requirement will
grow from a millisecond to a second to over 16 minutes to over 11 days! Such an
120–
n log n
100-
80–
60-
40–
20–
10 20 30 40 n
algorithm is nearly always far too slow to be used in practice, and even a much faster
processor makes hardly any difference (see Example 2.4).
We say that an algorithm is feasible if it is fast enough to be used in practice. Likewise.
we say that a problem is feasible if it can be solved by a feasible algorithm.
Algorithms of complexity up to O(n log n) are feasible. Algorithms of complexity
O(n2) or O(n3) might be feasible, but only for small values of n. Algorithms of complex-
ity 0(2n) are infeasible, except possibly for very small values of n.
In one second, all three algorithms can process the same amount of data (by coincidence).
But there the similarity ends. In one minute, Algorithm A can process by far the most
data, and Algorithm C the least. If an hour is allowed, Algorithm A is out of sight!
How much difference does it make if we use a processor that is ten times faster? This
reduces each algorithm's running time by a factor of ten. The effects are as follows:
Ironically, Algorithm A (already the fastest) benefits the most, and Algorithm C (already
the slowest) benefits the least, from using the faster processor! Algorithm A can now
process ten times as much data in any given time, Algorithm B about three times as much
data, and Algorithm C only three extra data items.
Even if we handicap Algorithm A by leaving it to run on the slower processor, in as
little as a minute it beats Algorithm B and outclasses Algorithm C.
The moral of Example 2.4 is this. Constant factors are important only when comparing
algorithms of the same time complexity, such as two O(n) algorithms, or two O(n2)
algorithms. But an O(n) algorithm will beat an 0(n2) algorithm, sooner or later, regard-
24
less of constant factors. More generally, O(1) beats O(log n) beats O(n) beats O(n log n)
beats O(n2) beats O(n 3 ) beats O(2 n ).
Example 2.5 suggests guidelines for ensuring that a recursive algorithm terminates:
• The problem must have one or more 'easy' cases and one or more 'hard' cases.
• In an 'easy' case, the algorithm must give a direct answer without calling itself.
• In a 'hard' case, the algorithm may call itself, but only to deal with an 'easier" case of
the same problem.
In Example 2.5, the problem had one easy case, n = 0, and one hard case, n > 0. In the
hard case, the algorithm called itself to deal with an easier case, n–1, and used that to
compute its answer.
Many algorithms can be expressed using either iteration or recursion. (Compare Algo-
rithms 2.3 and 2.11, and compare Algorithms 2.5 and 2.14.) Typically the recursive
algorithm is more elegant and easier to understand, but less efficient, than the correspond-
ing iterative algorithm.
We do not always have a straight choice between iteration and recursion. For some
problems an iterative solution would be extremely awkward, and a recursive solution is
much more elegant.
Rendering means converting data into a form suitable for printing or display on a screen.
Most often, data are rendered as character strings (although some data are suitable for
rendering graphically).
The problem is to render a given integer i to a given base (or radix) r between 2 and 10.
The rendered integer is to be signed only if negative. For example:
Rendering
+ 29 2 "11101"
+ 29 8 "35"
–29 8 "–35"
+ 29 10 "29"
Three vertical poles are mounted on a platform. A number of disks are provided, all of
different sizes, and each with a central hole allowing it to be threaded on to any of the
poles. Initially, all of the disks are on pole 1, forming a tower with the largest disk at the
bottom and the smallest disk at the top. Only a single disk may be moved at a time, from
the top of any tower to the top of another tower, but no larger disk may be moved on top
of a smaller disk. The problem is to move the tower of disks from pole 1 to pole 2.
According to legend, this task was originally set for the monks at the monastery of
Hanoi. Sixty-four disks were provided. Once the monks completed their task, the
universe would come to an end. How long should this take?
Rather than the particular problem of moving the tower of 64 disks from pole 1 to pole
2, it will prove helpful to address the more general problem of moving a tower of n disks
from pole source to pole dest.