CHAPTER 1

BASIC CONCEPTS

1.1 OVERVIEW: SYSTEM LIFE CYCLE

We assume that our readers have a strong background in structured programming, typi­
cally attained through the completion of an elementary programming course. Such an
initial course usually emphasizes mastering a programming language’s syntax (its gram­
mar rules) and applying this language to the solution of several relatively small prob­
lems. These problems are frequently chosen so that they use a particular language con­
struct. For example, the programming problem might require the use of arrays or while
loops.
In this text we want to move you beyond these rudiments by providing you with
the tools and techniques necessary to design and implement large-scale computer sys­
tems. We believe that a solid foundation in data abstraction, algorithm specification, and
performance analysis and measurement provides the necessary methodology. In this
chapter, we will discuss each of these areas in detail. We also will briefly discuss recur­
sive programming because many of you probably have only a fleeting acquaintance with
this important technique. However, before we begin we want to place these tools in a
context that views programming as more than writing code. Good programmers regard
large-scale computer programs as systems that contain many complex interacting parts.
As systems, these programs undergo a development process called the system life cycle.
We consider this cycle as consisting of requirements, analysis, design, coding, and
verification phases. Although we will consider them separately, these phases are highly

1
2 Basic Concepts

interrelated and follow only a very crude sequential time frame. The Selected Readings
and References section lists several sources on the system life cycle and its various
phases that will provide you with additional information.

(1) Requirements. All large programming projects begin with a set of specifications
that define the purpose of the project. These requirements describe the information that
we, the programmers, are given (input) and the results that we must produce (output).
Frequently the initial specifications are defined vaguely, and we must develop rigorous
input and output descriptions that include all cases.

(2) Analysis. After we have delineated carefully the system’s requirements, the analysis
phase begins in earnest. In this phase, we begin to break the problem down into manage­
able pieces. There are two approaches to analysis; bottom-up and top-down. The
bottom-up approach is an older, unstructured strategy that places an early emphasis on
the coding fine points. Since the programmer does not have a master plan for the project,
the resulting program frequently has many loosely connected, error-ridden segments.
Bottom-up analysis is akin to constructing a building from a generic blueprint. That is,
we view all buildings identically; they must have walls, a roof, plumbing, and heating.
The specific purpose to which the building will be put is irrelevant from this perspective.
Although few of us would want to live in a home constructed using this technique, many
programmers, particularly beginning ones, believe that they can create good, error-free
programs without prior planning.
In contrast, the top-down approach begins with the purpose that the program will
serve and uses this end product to divide the program into manageable segments. This
technique generates diagrams that are used to design the system. Frequently, several
alternate solutions to the programming problem are developed and compared during this
phase.

(3) Design. This phase continues the work done in the analysis phase. The designer
approaches the system from the perspectives of both the data objects that the program
needs and the operations performed on them. The first perspective leads to the creation
of abstract data types, while the second requires the specification of algorithms and a
consideration of algorithm design strategies. For example, suppose that we are design­
ing a scheduling system for a university. Typical data objects might include students,
courses, and professors. Typical operations might include inserting, removing, and
searching within each object or between them. That is, we might want to add a course to
the list of university courses, or search for the courses taught by some professor.
Since the abstract data types and the algorithm specifications are language­
independent, we postpone implementation decisions. Although we must specify the
information required for each data object, we ignore coding details. For example, we
might decide that the student data object should include name, social security number,
major, and phone number. However, we would not yet pick a specific implementation
for the list of students. As we will see in later chapters, there are several possibilities
including arrays, linked lists, or trees. By deferring implementation issues as long as
 Overview 3

possible, we not only create a system that could be written in several programming
languages, but we also have time to pick the most efficient implementations within our
chosen language.

(4) Refinement and coding. In this phase, we choose representations for our data
objects and write algorithms for each operation on them. The order in which we do this
is crucial because a data object’s representation can determine the efficiency of the algo­
rithms related to it. Typically this means that we should write those algorithms that are
independent of the data objects first.
Frequently at this point we realize that we could have created a much better sys­
tem. Perhaps we have spoken with a friend who has worked on a similar project, or we
realize that one of our alternate designs is superior. If our original design is good, it can
absorb changes easily. In fact, this is a reason for avoiding an early commitment to cod­
ing details. If we must scrap our work entirely, we can take comfort in the fact that we
will be able to write the new system more quickly and with fewer errors. A delightful
book that discusses this "second system" phenomenon is Frederick Brooks’s, The Mythi­
cal Man-Month cited in the Selected Readings and References section.

(5) Verification. This phase consists of developing correctness proofs for the program,
testing the program with a variety of input data, and removing errors. Each of these
areas has been researched extensively, and a complete discussion is beyond the scope
of this text. However, we want to summarize briefly the important aspects of each area.

Correctness proofs: Programs can be proven correct using the same techniques that
abound in mathematics. Unfortunately, these proofs are very time-consuming, and
difflcult to develop for large projects. Frequently scheduling constraints prevent the
development of a complete set of proofs for a large system. However, selecting algo­
rithms that have been proven correct can reduce the number of errors. In this text, we
will provide you with an arsenal of algorithms, some of which have been proven correct
using formal techniques, that you may apply to many programming problems.

Testing: We can construct our correctness proofs before and during the coding phase
since our algorithms need not be written in a specific programming language. Testing,
however, requires the working code and sets of test data. This data should be developed
carefully so that it includes all possible scenarios. Frequently beginning programmers
assume that if their program ran without producing a syntax error, it must be correct.
Little thought is given to the input data, and usually only one set of data is used. Good
test data should verify that every piece of code runs correctly. For example, if our pro­
gram contains a switch statement, our test data should be chosen so that we can check
each case within the switch statement.
Initial system tests focus on verifying that a program runs correctly. While this is
a crucial concern, a program’s running time is also important. An error-free program that
runs slowly is of little value. Theoretical estimates of running time exist for many algo­
rithms and we will derive these estimates as we introduce new algorithms. In addition.
4 Basic Concepts

we may want to gather performance estimates for portions of our code. Constructing
these timing tests is also a topic that we pursue later in this chapter.

Error removal. If done properly, the correctness proofs and system tests will indicate
erroneous code. The ease with which we can remove these errors depends on the design
and coding decisions made earlier. A large undocumented program written in
"spaghetti" code is a programmer’s nightmare. When debugging such programs, each
corrected error possibly generates several new errors. On the other hand, debugging a
well-documented program that is divided into autonomous units that interact through
parameters is far easier. This is especially true if each unit is tested separately and then
integrated into the system.

1.2 ALGORITHM SPECIFICATION

1.2.1 Introduction

The concept of an algorithm is fundamental to computer science. Algorithms exist for

many common problems, and designing efficient algorithms plays a crucial role in
developing large-scale computer systems. Therefore, before we proceed further we need
to discuss this concept more fully. We begin with a definition.

Definition: An algorithm is a finite set of instructions that, if followed, accomplishes a

particular task. In addition, all algorithms must satisfy the following criteria:

(1) Input. There are zero or more quantities that are externally supplied.
(2) Output. At least one quantity is produced.
(3) Definiteness. Each instruction is clear and unambiguous.
(4) Finiteness. If we trace out the instructions of an algorithm, then for all cases, the
algorithm terminates after a finite number of steps.
(5) Effectiveness. Every instruction must be basic enough to be carried out, in princi­
ple, by a person using only pencil and paper. It is not enough that each operation
be definite as in (3); it also must be feasible. □

In computational theory, one distinguishes between an algorithm and a program, the

latter of which does not have to satisfy the fourth condition. For example, we can think
of an operating system that continues in a wait loop until more jobs are entered. Such a
program does not terminate unless the system crashes. Since our programs will always
terminate, we will use algorithm and program interchangeably in this text.
 Algorithm Specification 5

We can describe an algorithm in many ways. We can use a natural language like
English, although, if we select this option, we must make sure that the resulting instruc­
tions are definite. Graphic representations called flowcharts are another possibility, but
they work well only if the algorithm is small and simple. In this text we will present
most of our algorithms in C, occasionally resorting to a combination of English and C for
our specifications. Two examples should help to illustrate the process of translating a
problem into an algorithm.

Example 1.1 [Selection sort]'. Suppose we must devise a program that sorts a set of
n > 1 integers. A simple solution is given by the following:

From those integers that are currently unsorted, find the smallest and place it next
in the sorted list.

Although this statement adequately describes the sorting problem, it is not an algo­
rithm since it leaves several unanswered questions. For example, it does not tell us
where and how the integers are initially stored, or where we should place the result. We
assume that the integers are stored in an array, list, such that the /th integer is stored in
the /th position, list [Z], 0 < i < n. Program 1.1 is our first attempt at deriving a solution.
Notice that it is written partially in C and partially in English.

for (i = 0; i < n; i++) {

Examine list[i] to list[n-l] and suppose that the
smallest integer is at list[min] ;

Interchange list[i] and list[min];

}

Program 1.1: Selection sort algorithm

To turn Program 1.1 into a real C program, two clearly defined subtasks remain: finding
the smallest integer and interchanging it with list[i ]. We can solve the latter problem
using either a function (Program 1.2) or a macro. The function's code is easier to read
than that of the macro but the macro works with any data type. Using the function, sup­
pose a and b are declared as ints. To swap their values one would say:

swap (&a, &;b) ;

passing to swap the addresses of a and h. The macro version of swap is:

ttdefine SWAP{x,y,t) {(t) M, (X) (Y)z {y} (t) }

6 Basic Concepts

void swap(int *x
*,X, int *y)
*
/ both parameters are pointers to ints */

int temp = *x; /

* declares temp as an int and assigns
to it the contents of what x points to */
★
*x *y; / stores what y points to into the location
where x points
*/
temp; /
places
* the contents of temp in location
pointed to by y */
}

Program 1.2: Swap function

We can solve the first subtask by assuming that the minimum is list[i], checking
list[i] with ZZ5r[Z+l], ZZ5t[Z+2], • • • , list[n-l]. Whenever we find a smaller number
we make it the new minimum. When we reach list[n-].] we are finished. Putting all
these observations together gives us sort (Program 1.3). Program 1.3 contains a com­
plete program which you may run on your computer. The program uses the rand func­
tion defined in math.h to randomly generate a list of numbers which are then passed into
sort. This program has been successfully compiled and run on several systems including
Turbo C and Turbo C++ under DOS 5.0. All programs in this book follow the rules of
ANSI C, which are slightly different from those of Kemighan & Ritchie C (K&R C).
Appendix A shows you the changes required to transform our ANSI C programs into
K&R C. At this point, we should ask if this function works correctly.

Theorem 1.1: Function sort(list,n) correctly sorts a set of n > 1 integers. The result
remains in list [0], • • • , list [n -1] such that list [0] < ZZ5? [1] < ■ • • < list [n-1].

Proof: When the outer for loop completes its iteration for i = q, we have list [q ] <
list [r <r <n. Further, on subsequent iterations, i > q and list [0] through ] are
unchanged. Hence following the last iteration of the outer for loop (i.e., i = n - 2), we
have list [0] < ZZ^t [1] < • • • < list [n-1]. □

Example 1.2 {Binary search}". Assume that we have n > 1 distinct integers that are
already sorted and stored in the array list. That is, ZZ^zfO] < ZZ^rfl] < • • • < list[n-\].
We must figure out if an integer searchnum is in this list. If it is we should return an
index, Z, such that list[i] = searchnum. If searchnum is not present, we should return -1.
Since the list is sorted we may use the following method to search for the value.
Let left and right, respectively, denote the left and right ends of the list to be
searched. Initially, left = 0 and right = n~l. Let middle = (left+right)/2 be the middle
position in the list. If we compare list [middle ] with searchnum, we obtain one of three
results:
 Algorithm Specification 7

#include stdio.h>
#include <math.h>
#define MAX-SIZE 101
#define SWAP(x,y,t) ((t) (X), {x)= (y), (y) (t) )
void sort(int [],int); *
/
selection sort
void main(void)
{
int i,n;
int list[MAX-SIZE] ;
printf("Enter the number of numbers to generate: ")
scanf("%d", &n) ;
if ( n Ilin MAX-SIZE) {
fprintf(stderr, "Improper value of n\n");
exit(1);

for (i 0; i < n; i++) randomly

*
{/ generate numbers
/
*
list[i] - randO % 1000;
printf ('’%d ", 1 i s t [ i ] ) ;
}
sort(list,n);
printf("\n Sorted array:\n ");
for (i =0; i < n; i++) / ★ print out sorted numbers
/
printf("%d ", 1 i s t [ i ] ) ;
printf("in") ;
}
void sort(int list[],int n)
{
int i, j, min, temp;
for (i = 0; i < n-1; i + +) {
min = iI
for (j i + 1; : : n; j++)
if (list[j] list[min] )
min = j;
SWAP(list[i] ,list[min] ,temp);
}
}

Program 1.3: Selection sort

8 Basic Concepts

(1) searchnum < list[middle]. In this case, if searchnum is present, it must be in the
positions between 0 and middle - 1. Therefore, we set right to middle - 1.
(2) searchnum = list[middle]. In this case, we return middle.
(3) searchnum > list[middle]. In this case, if searchnum is present, it must be in the
positions between middle + 1 and n - 1. So, we set left to middle + 1.

If searchnum has not been found and there are still integers to check, we recalculate
middle and continue the search. Program 1.4 implements this searching strategy. The
algorithm contains two subtasks: (1) determining if there are any integers left to check,
and (2) comparing searchnum to list[middle].

while (there are more integers to check ) {

middle (left + right) / 2;
if (searchnum list[middle])
right = middle - 1;
else if (searchnum == list[middle])
return middle;
else left = middle + 1;
}

Program 1.4: Searching a sorted list

We can handle the comparisons through either a function or a macro. In either

case, we must specify values to signify less than, equal, or greater than. We will use the
strategy followed in C’s library functions:
• We return a negative number (-1) if the first number is less than the second.
• We return a 0 if the two numbers are equal.
• We return a positive number (1) if the first number is greater than the second.
Although we present both a function (Program 1.5) and a macro, we will use the macro
throughout the text since it works with any data type. The macro version is:

ttdefine COMPARE(x,y) {((x) < (y)) 7 -1: ((x) (y))? 0: 1)

We are now ready to tackle the first subtask: determining if there are any elements
left to check. You will recall that our initial algorithm indicated that a comparison could
cause us to move either our left or right index. Assuming we keep moving these indices,
we will eventually find the element, or the indices will cross, that is, the left index will
have a higher value than the right index. Since these indices delineate the search boun­
daries, once they cross, we have nothing left to check. Putting all this information
together gives us binsearch (Program 1.6).
 Algorithm Specification 9

int compare(int x, int y)

{
/
* compare x and y, return -1 for less than, 0 for equal,
1 for greater */
if (X y) return -1;
else if (x == y) return 0;
else return 1;
)

Program 1.5: Comparison of two integers

int binsearch{int list[], int searchnum, int left,

int right)
{
/★ search list[0]
list[0] <- list[l] <= - - - <- list[n-l] for
searchnum. Return its position if found. Otherwise
return -1 */
int middle;
while (left <= right) {
middle = (left + right)/2;
switch (COMPARE(list[middle], searchnum)) {
case -1; left = middle + 1;
break;
case 0 : return middle;
case 1 : right = middle - 1;
}
}
return -1;
}

Program 1.6: Searching an ordered list

The search strategy just outlined is called binary search. □

The previous examples have shown that algorithms are implemented as functions
in C. Indeed functions are the primary vehicle used to divide a large program into
manageable pieces. They make the program easier to read, and, because the functions
can be tested separately, increase the probability that it will run correctly. Often we will
declare a function first and provide its definition later. In this way the compiler is made
aware that a name refers to a legal function that will be defined later. In C, groups of
functions can be compiled separately, thereby establishing libraries containing groups of
10 Basic Concepts

logically related algorithms.

1.2.2 Recursive Algorithms

Typically, beginning programmers view a function as something that is invoked (called)

by another function. It executes its code and then returns control to the calling function.
This perspective ignores the fact that functions can call themselves {direct recursion) or
they may call other functions that invoke the calling function again {indirect recursion).
These recursive mechanisms are not only extremely powerful, but they also frequently
allow us to express an otherwise complex process in very clear terms. It is for these rea­
sons that we introduce recursion here.
Frequently computer science students regard recursion as a mystical technique that
is useful for only a few special problems such as computing factorials or Ackermann's
function. This is unfortunate because any function that we can write using assignment,
if-else, and while statements can be written recursively. Often this recursive function is
easier to understand than its iterative counterpart.
How do we determine when we should express an algorithm recursively? One
instance is when the problem itself is defined recursively. Factorials and Fibonacci
numbers fit into this category as do binomial coefficients where:
n n!
m m \{n — mf.
can be recursively computed by the formula:
n n-1 n-1
+
m m m-1
We would like to use two examples to show you how to develop a recursive algo­
rithm. In the first example, we take the binary search function that we created in Exam­
ple 1.2 and transform it into a recursive function. In the second example, we recursively
generate all possible permutations of a list of characters.

Example 1.3 [Binary search]: Program 1.6 gave the iterative version of a binary search.
To transform this function into a recursive one, we must (1) establish boundary condi­
tions that terminate the recursive calls, and (2) implement the recursive calls so that each
call brings us one step closer to a solution. If we examine Program 1.6 carefully we can
see that there are two ways to terminate the search: one signaling a success (list[fniddle]
= searchnum), the other signaling a failure (the left and right indices cross). We do not
need to change the code when the function terminates successfully. However, the while
statement that is used to trigger the unsuccessful search needs to be replaced with an
equivalent if statement whose then clause invokes the function recursively.
Creating recursive calls that move us closer to a solution is also simple since it
requires only passing the new left or right index as a parameter in the next recursive call.
Program 1.7 implements the recursive binary search. Notice that although the code has
 Algorithm Specification 11

changed, the recursive function call is identical to that of the iterative function. □

int binsearch(int list[], int searchnum, int left,

int right)
{
/'^ search list[0]
list [0] <- list[l] <- • • • = list[n-l] for
searchnum. Return its position if found, Otherwise
return -1
int middle;
if (left = right) {
middle - (left + right)/2;
switch (COMPARE(list[middle], searchnum)) {
case -1: return
binsearch(list, searchnum, middle + 1, right);
case 0 : return middle;
case 1 : return
binsearch(list, searchnum. left, middle - 1);
}
}
return -1;
}

Program 1.7: Recursive implementation of binary search

Example 1.4 [Permutations}

*, Given a set of n > 1 elements, print out all possible permu­
tations of this set. For example, if the set is [a, b. c), then the set of permutations is {(^z,
b, c), (a, c, b}, (b, a, c), {b, c, d}, {c, a, b\ (c, b, «)). It is easy to see that, given n ele­
ments, there are n! permutations. We can obtain a simple algorithm for generating the
permutations if we look at the set [a, b, c, J). We can construct the set of permutations
by printing:
(1) a followed by all permutations of (b, c, d)
(2) b followed by all permutations of (a, c, d}
(3) c followed by all permutations of {a, b, d}
(4) d followed by all permutations of (6Z, b, c}

The clue to the recursive solution is the phrase "followed by all permutations." It implies
that we can solve the problem for a set with n elements if we have an algorithm that
works on n - 1 elements. These considerations lead to the development of Program 1.8.
We assume that list is a character array. Notice that it recursively generates permuta­
tions until i = n. The initial function call is perm(list. 0, n-J);
12 Basic Concepts

void perm{char int i, int n)

*
/ generate all the permutations of listti] to list[n3
{
int j, temp;
if (i == n) {
for (j 0; j n; zi++)
printf (’’%c", list [ j] ) ;
printf(" ) ;
}
else {
/* list[i] to list[n] has more than one permutation,
generate these recursively */
for (j i; j <= n; j++) {
SWAP{list[i]zlist[j],temp);
perm(list,i+l,n) ;
SWAP(list[i],list[j],temp);
}
}
)

Program 1.8: Recursive permutation generator

Try to simulate Program 1.8 on the three-element set (a, b, c}. Each recursive call
of perm produces new local copies of the parameters list, i, and n. The value of i will
differ from invocation to invocation, but n will not. The parameter list is an array pointer
and its value also will not vary from call to call. □
We will encounter recursion several more times since many of the algorithms that
appear in subsequent chapters are recursively defined. This is particularly true of algo­
rithms that operate on lists (Chapter 4) and binary trees (Chapter 5).

EXERCISES
In the last several examples, we showed you how to translate a problem into a program.
We have avoided the issues of data abstraction and algorithm design strategies, choosing
to focus on developing a function from an English description, or transforming an itera­
tive algorithm into a recursive one. In the exercises that follow, we want you to use the
same approach. For each programming problem, try to develop an algorithm, translate it
into a function, and show that it works correctly. Your correctness "proof" can employ
an analysis of the algorithm or a suitable set of test runs.
 Algorithm Specification 13

1. Consider the two statements:

(a) Is n = 2 the largest value of n for which there exist positive integers x, y, and
2 such that x" + y” = z” has a solution?
(b) Store 5 divided by zero into x and go to statement 10.

Both fail to satisfy one of the five criteria of an algorithm. Which criterion do they
violate?
2. Homer’s rule is a strategy for evaluating a polynomial A (x) =

+ a^-xx n-\ + ' • ' + a [ + gq

at point xq using a minimum number of multiplications. This rule is:

(^o) - ( ■ ■ ■ ^n-l) -^0+ ■■■ + ^l)-’^o + ^o)

Write a C program to evaluate a polynomial using Homer’s rule.

3. Given n Boolean variables X]1,, • • • ,Xn,v^c wish to print all possible combinations
of truth values they can assume. For instance, if n = 2, there are four possibilities:
<true, true>. <false, true>, <true, false>, and <false, false>. Write a C program to
do this.
4. Write a C program that prints out the integer values of x, y, z in ascending order.
5. The pigeon hole principle stales that if a function f has n distinct inputs but less
than n distinct outputs then there are two inputs a and b such that a-^b and/(a) =
f {b}. Write a C program to find the values a and b for which the range values are
equal.
6. Given n, a positive integer, determine if n is the sum its divisors, that is, if n is the
sum of all t such that 1 < r < n and t divides zt.
7. The factorial function n ! has value 1 when n < 1 and value n
*
(n-l) ’ when n > 1.
Write both a recursive and an iterative C function to compute n !.
8. The Fibonacci numbers are defined as: /q = 0, f\ - 1, and = fj_x ■\-fi-2 for z > 1.
Write both a recursive and an iterative C function to compute /.
9. Write an iterative function to compute a binomial coefficient, then transform it into
an equivalent recursive function.
10. Ackerman’s function A {m, n} is defined as:
n + 1 , if m = 0
z4 (m, n) = (m — 1, 1) , if A? = 0
A (w - 1, A (/M, « - 1)) , otherwise
This function is studied because it grows very quickly for small values of m and n.
Write recursive and iterative versions of this function.
14 Basic Concepts

11. [Towers of Hanoi} There are three towers and 64 disks of different diameters
placed on the first tower. The disks are in order of decreasing diameter as one
scans up the tower. Monks were reputedly supposed to move the disk from tower
1 to tower 3 obeying the rules:
(a) Only one disk can be moved at any time.
(b) No disk can be placed on top of a disk with a smaller diameter.
Write a recursive function that prints out the sequence of moves needed to accom­
plish this task.
12. If S is a set of n elements the powerset of S is the set of all possible subsets of S.
For example, if S= [a, b, c), then powerset (S) = { {), {cf), {Z?), {c}, [a, b], [a, cj,
{b, c}, [a, b, c)}. Write a recursive function to compute powerset(S).

1.3 DATA ABSTRACTION

The reader is no doubt familiar with the basic data types of C. These include char, int,
float, and double. Some of these data types may be modified by the keywords short,
long, and unsigned. Ultimately, the real world abstractions we wish to deal with must
be represented in terms of these data types. In addition to these basic types, C helps us
by providing two mechanisms for grouping data together. These are the array and the
structure. Arrays are collections of elements of the same basic data type. They are
declared implicitly, for example, int list[5] defines a five-element array of integers whose
legitimate subscripts are in the range 0 • • • 4. Structs are collections of elements whose
data types need not be the same. They are explicitly defined. For example,

struct student {
char last—name;
int student—id;
char grade;
}

defines a structure with three fields, two of type character and one of type integer. The
structure name is student. Details of C structures are provided in Chapter 2.
C also provides the pointer data type. For every basic data type there is a
corresponding pointer data type, such as pointer-to-an-int, pointer-to-a-real, pointer-to-
a-char, and pointer-to-a-float. A pointer is denoted by placing an asterisk, *, before a
variable’s name. So,

int i, *pi;

declares i as an integer and pi as a pointer to an integer.

All programming languages provide at least a minimal set of predefined data
types, plus the ability to construct new, or user-defined types. It is appropriate to ask the
 Data Abstraction 15

question, "What is a data type?"

Definition: A data type is a collection of objects and a set of operations that act on those
objects. □

Whether your program is dealing with predefined data types or user-defined data types,
these two aspects must be considered: objects and operations. For example, the data
type int consists of the objects {0, +1,-1, +2, -2, ■ • • , IN I'- MAX. INT-MIN), where
INT-MAX and INT-MIN are the largest and smallest integers that can be represented
on your machine. (They are defined in limits.h.) The operations on integers are many,
and would certainly include the arithmetic operators +, * ,!, and %. There is also test­
ing for equality/inequality and the operation that assigns an integer to a variable. In all
of these cases, there is the name of the operation, which may be a prefix operator, such as
atoi, or an infix operator, such as +. Whether an operation is defined in the language or
in a library, its name, possible arguments and results must be specified.
In addition to knowing all of the facts about the operations on a data type, we
might also want to know about how the objects of the data type are represented. For
example on most computers a char is represented as a bit string occupying 1 byte of
memory, whereas an int might occupy 2 or possibly 4 bytes of memory. If 2 eight-bit
bytes are used, then INT-MAXis 2*^ - 1 = 32,767.
Knowing the representation of the objects of a data type can be useful and
dangerous. By knowing the representation we can often write algorithms that make use
of it. However, if we ever want to change the representation of these objects, we also
must change the routines that make use of it. It has been observed by many software
designers that hiding the representation of objects of a data type from its users is a good
design strategy. In that case, the user is constrained to manipulate the objects solely
through the functions that are provided. The designer may still alter the representation
as long as the new implementations of the operations do not change the user interface.
This means that users will not have to recode their algorithms.

Definition: An abstract data type (ADT) is a data type that is organized in such a way
that the specification of the objects and the specification of the operations on the objects
is separated from the representation of the objects and the implementation of the opera­
tions. □

Some programming languages provide explicit mechanisms to support the distinction

between specification and implementation. For example, Ada has a concept called a
package, and C++ has a concept called a class. Both of these assist the programmer in
implementing abstract data types. Although C does not have an explicit mechanism for
implementing ADTs, it is still possible and desirable to design your data types using the
same notion.
How does the specification of the operations of an ADT differ from the implemen­
tation of the operations? The specification consists of the names of every function, the
type of its arguments, and the type of its result. There should also be a description of
16 Basic Concepts

what the function does, but without appealing to internal representation or implementa­
tion details. This requirement is quite important, and it implies that an abstract data type
is implementation-independent. Furthermore, it is possible to classify the functions of a
data type into several categories:
(1) Creator/constructor: These functions create a new instance of the designated
type.
(2) Transformers: These functions also create an instance of the designated type,
generally by using one or more other instances. The difference between construc­
tors and transformers will become more clear with some examples.
(3) Observers/reporters: These functions provide information about an instance of
the type, but they do not change the instance.
Typically, an ADT definition will include at least one function from each of these three
categories.
Throughout this text, we will emphasize the distinction between specification and
implementation. In order to help us do this, we will typically begin with an ADT
definition of the object that we intend to study. This will permit the reader to grasp the
essential elements of the object, without having the discussion complicated by the
representation of the objects or by the actual implementation of the operations. Once the
ADT definition is fully explained we will move on to discussions of representation and
implementation. These are quite important in the study of data structures. In order to
help us accomplish this goal, we introduce a notation for expressing an ADT.

Example 1.5 [Abstract data type Natural-Number}

.
* As this is the first example of an
ADT, we will spend some time explaining the notation. Structure 1.1 contains the ADT
definition of Natural-Number. The structure definition begins with the name of the
structure and its abbreviation. There are two main sections in the definition: the objects
and the functions. The objects are defined in terms of the integers, but we make no
explicit reference to their representation. The function definitions are a bit more compli­
cated. First, the definitions use the symbols x and y to denote two elements of the set of
Natural-Numbers, while TRUE and FALSE are elements of the set of Boolean values. In
addition, the definition makes use of functions that are defined on the set of integers,
namely, plus, minus, equals, and less than. This is an indication that in order to define
one data type, we may need to use operations from another data type. For each function,
we place the result type to the left of the function name and a definition of the function
to the right. The symbols rt " should be read as "is defined as."
The first function. Zero, has no arguments and returns the natural number zero.
This is a constructor function. The function Successor(x) returns the next natural
number in sequence. This is an example of a transformer function. Notice that if there is
no next number in sequence, that is, if the value of x is already INT-MAX, then we
define the action of Successor to return INT-MAX. Some programmers might prefer that
in such a case Successor return an error flag. This is also perfectly permissible. Other
transformer functions are Add and Subtract. They might also return an error condition.
 Data Abstraction 17

structure Natural-Number is
objects: an ordered subrange of the integers starting at zero and ending at the
maximum integer {INT-MAX} on the computer
functions:
for all X, y g Nat-Number\ TRUE, FALSE e Boolean
and where +, <, and == are the usual integer operations
Nat-No Zero() 0
Boolean Is-Zero(x) if (x) return FALSE
else return TRUE
Nat-No AddU, y) if ({x + y) <= INT-MAX) return x + y
else return INT-MAX
Boolean Equal(x, y) if (x == y) return TRUE
else return FALSE
Nat-No Successor(x) if (x == INT-MAX) return x
else return x + 1
Nat-No Subtract(.T, y) if (x < y) return 0
else return x - y
end Natural-Number

Structure 1.1: Abstract data type Natural-Number

although here we decided to return an element of the set Natural-Number. □

Structure 1.1 shows you the general form that all ADT definitions will follow.
However, we will not often be able to provide a definition of the functions that is so
close to C functions. In fact, the nature of an ADT argues that we avoid implementation
details. Therefore, we will usually use a form of structured English to explain the mean­
ing of the functions.

EXERCISES
For each of these exercises, provide a definition of the abstract data type using the form
illustrated in Structure 1.1.
1. Add the following operations to the Natural-Number ADT: Predecessor,
Is-Greater, Multiply, Divide.
2. Create an ADT, Set. Use the standard mathematics definition and include the fol­
lowing operations: Create, Insert, Remove, Is-ln, Union, Intersection, Difference.
3. Create an ADT, Bag. In mathematics a bag is similar to a set except that a bag may
contain duplicate elements. The minimal operations should include: Create,
Insert, Remove, and Is-In.
1$ Basic Concepts

4. Create an ADT, Boolean. The minimal operations are And, Or, Not, Xor (Exclusive
or), Equivalent, and Implies.

1.4 PERFORMANCE ANALYSIS

One of the goals of this book is to develop your skills for making evaluative judgments
about programs. There are many criteria upon which we can judge a program, including:

(1) Does the program meet the original specifications of the task?
(2) Does it work correctly?
(3) Does the program contain documentation that shows how to use it and how it
works?
(4) Does the program effectively use functions to create logical units?
(5) Is the program’s code readable?

Although the above criteria are vitally important, particularly in the development of
large systems, it is difficult to explain how to achieve them. The criteria are associated
with the development of a good programming style and this takes experience and prac­
tice. We hope that the examples used throughout this text will help you improve your
programming style. However, we also can judge a program on more concrete criteria,
and so we add two more criteria to our list.

(6) Does the program efficiently use primary and secondary storage?
(7) Is the program’s running time acceptable for the task?

These criteria focus on performance evaluation, which we can loosely divide into
two distinct fields. The first field focuses on obtaining estimates of time and space that
are machine independent. We call this field performance analysis, but its subject matter
is the heart of an important branch of computer science known as complexity theory.
The second field, which we call performance measurement, obtains machine-dependent
running times. These times are used to identify inefficient code segments. In this section
we discuss performance analysis, and in the next we discuss performance measurement.
We begin our discussion with definitions of the space and time complexity of a program.

Definition: The space complexity of a program is the amount of memory that it needs to
run to completion. The time complexity of a program is the amount of computer time
that it needs to run to completion. □
 Performance Analysis 19

1.4.1 Space Complexity

The space needed by a program is the sum of the following components:

(1) Fixed space requirements: This component refers to space requirements that do not
depend on the number and size of the program’s inputs and outputs. The fixed require­
ments include the instruction space (space needed to store the code), space for simple
variables, fixed-size structured variables (such as structs), and constants.

(2) Variable space requirements: This component consists of the space needed by
structured variables whose size depends on the particular instance, /, of the problem
being solved. It also includes the additional space required when a function uses recur­
sion. The variable space requirement of a program P working on an instance / is denoted
Sp(I). Sp(l) is usually given as a function of some characteristics of the instance 1.
Commonly used characteristics include the number, size, and values of the inputs and
outputs associated with I. For example, if our input is an array containing n numbers
then n is an instance characteristic. If n is the only instance charcteristic we wish to use
when computing Sp{l), we will use Sp(n) to represent Sp(I).
We can express the total space requirement S (F) of any program as:

S(P) = c + Sp(I)

where c is a constant representing the fixed space requirements. When analyzing the
space complexity of a program we are usually concerned with only the variable space
requirements. This is particularly true when we want to compare the space complexity
of several programs. Let us look at a few examples.

Example 1.6 : We have a function, abc (Program 1.9), which accepts three simple vari­
ables as input and returns a simple value as output. According to the classification
given, this function has only fixed space requirements. Therefore, SabcU) = 0. □

float abc(float a, float b, float c)

{
return a+b+b
c
* +(a+b-c)/(a+b)+4.00;
}

Program 1.9: Simple arithmetic function

Example 1.7 : We want to add a list of numbers (Program 1.10). Although the output is a
simple value, the input includes an array. Therefore, the variable space requirement
depends on how the array is passed into the function. Programming languages like
20 Basic Concepts

Pascal may pass arrays by value. This means that the entire array is copied into tem­
porary storage before the function is executed. In these languages the variable space
requirement for this program is = n, where n is the size of the array. C
passes all parameters by value. When an array is passed as an argument to a function, C
interprets it as passing the address of the first element of the array. C does not copy the
array. Therefore, = 0. □

float sum(float list[], int n)

{
float tempsum = 0;
int i;
for 0; i < n; i++)
(i =0;
tempsum += list[i];
return tempsum;
}

Program LIO: Iterative function for summing a list of numbers

Example 1.8 : Program 1.11 also adds a list of numbers, but this time the summation is
handled recursively. This means that the compiler must save the parameters, the local
variables, and the return address for each recursive call.

float rsum(float list[], int n)

{
if (n) return rsum(list,n-1) + list[n-l];
return 0;
}

Program 1.11: Recursive function for summing a list of numbers

In this example, the space needed for one recursive call is the number of bytes
required for the two parameters and the return address. We can use the sizeof function to
find the number of bytes required by each type. On an 80386 computer, integers and
pointers require 2 bytes of storage and floats need 4 bytes. Figure 1.1 shows the number
of bytes required for one recursive call.
If the array has n = MAX-SIZE numbers, the total variable space needed for the
recursive version is S,,^^{MAX-SIZE) = e^MAX-SIZE. If MAX-SIZE = 1000, the vari­
able space needed by the recursive version is 6 1000 = 6,000 bytes. The iterative ver­
*
sion has no variable space requirement. As you can see, the recursive version has a far
greater overhead than its iterative counterpart. □
 Performance Analysis 21

Type_______________________ Name Number of bytes

parameter: float /w/[] 2
parameter: integer n 2
return address: (used internally) 2 (unless a far address)
TOTAL per recursive call 6

Figure 1.1: Space needed for one recursive call of Program 1.11

EXERCISES

1. Determine the space complexity of the iterative and recursive factorial functions
created in Exercise 7, Section 1.2.
2. Determine the space complexity of the iterative and recursive Fibonacci number
functions created in Exercise 8, Section 1.2.
3. Determine the space complexity of the iterative and recursive binomial coefficient
functions created in Exercise 9, Section 1.2.
4. Determine the space complexity of the function created in Exercise 5, Section 1.2
(pigeon hole principle).
5. Determine the space complexity of the function created in Exercise 12, Section 1.2
(powerset problem).

1.4.2 Time Complexity

The time, T{P\ taken by a program, P, is the sum of its compile time and its run (or exe­
cution) time. The compile time is similar to the fixed space component since it does not
depend on the instance characteristics. In addition, once we have verified that the pro­
gram runs correctly, we may run it many times without recompilation. Consequently, we
are really concerned only with the program’s execution time, Tp.
Determining Tp is not an easy task because it requires a detailed knowledge of the
compiler’s attributes. That is, we must know how the compiler translates our source pro­
gram into object code. For example, suppose we have a simple program that adds and
subtracts numbers. Letting n denote the instance characteristic, we might express Tp{n)
as:
Tp{n) = CaADD{n) + c,SUB(n) + CiLDA(n) + c,,STA{n)
where c^,a ■’ Q, q, c^-i are constants that refer to the time needed to perform each operation,
and ADD., SUB, LDA, STA are the number of additions, subtractions, loads, and stores
that are performed when the program is run with instance characteristic n.
22 Basic Concepts

Obtaining such a detailed estimate of running time is rarely worth the effort. If we
must know the running time, the best approach is to use the system clock to time the pro­
gram. We will do this later in the chapter. Alternately, we could count the number of
operations the program performs. This gives us a machine-independent estimate, but we
must know how to divide the program into distinct steps.

Definition: A program step is a syntactically or semantically meaningful program seg­

ment whose execution time is independent of the instance characteristics. □

Note that the amount of computing represented by one program step may be
different from that represented by another step. So, for example, we may count a simple
assignment statement of the form a = 2 as one step and also count a more complex state­
ment such as a = 2b+3=^c/d~e+f/g/a/b/c as one step. The only requirement is that the
*
time required to execute each statement that is counted as one step be independent of the
instance characteristics.
We can determine the number of steps that a program or a function needs to solve
a particular problem instance by creating a global variable, count, which has an initial
value of 0 and then inserting statements that increment count by the number of program
steps required by each executable statement.

Example 1.9 [Iterative summing of a list of numbers]'. We want to obtain the step count
for the sum function discussed earlier (Program 1.10). Program 1.12 shows where to
place the count statements. Notice that we only need to worry about the executable
statements, which automatically eliminates the function header, and the second variable
declaration from consideration.
Since our chief concern is determining the final count, we can eliminate most of
the program statements from Program 1.12 to obtain a simpler program Program 1.13
that computes the same value for count. This simplification makes it easier to express
the count arithmetically. Examining Program 1.13, we can see that if counfs initial
value is 0, its final value will be 2n + 3. Thus, each invocation of sum executes a total of
2/14-3 steps. □

Example 1.10 [Recursive summing of a list of numbers]'. We want to obtain the step
count for the recursive version of the summing function. Program 1.14 contains the ori­
ginal function (Program 1.11) with the step counts added.
To determine the step count for this function, we first need to figure out the step
count for the boundary condition of n = 0. Looking at Program 1.14, we can see that
when n = 0 only the if conditional and the second return statement are executed. So, the
total step count for n = 0 is 2. For n > 0, the if conditional and the first return statement
are executed. So each recursive call with n > 0 adds two to the step count. Since there
are n such function calls and these are followed by one with n = 0, the step count for the
function is 2n 4- 2.
 Performance Analysis 23

float sum(float list[], int n)

{
float tempsum = 0; ★
count++; / for assignment */
int i ;
for (i 0; i n; i + +) {
count++; /
* for the for loop */
tempsum += list[i]; count++; /
* for assignment */
}
*
count++; / last execution of for */
count++; / for return */
*
/ return tempsum;
}

Program 1.12: Program 1.10 with count statements

float sum (float list[], int n)

{
float tempsum = 0;
int i;
for {i 0; i n; i + +)
count += 2;
count +=3;
return 0;
}

Program 1.13: Simplified version of Program 1.12

Surprisingly, the recursive function actually has a lower step count than its itera­
tive counterpart. However, we must remember that the step count only tells us how
many steps are executed, it does not tel! us how much time each step takes. Thus,
although the recursive function has fewer steps, it typically runs more slowly than the
iterative version as its steps, on average, take more time than those of the iterative ver­
sion. □

Example 1.11 [Matrix addition]: We want to determine the step count for a function
that adds two-dimensional arrays (Program 1.15). The arrays a and h are added and the
result is returned in array c. All of the arrays are of size rows x cols. Program 1.16
shows the add function with the step counts introduced. As in the previous examples, we
want to express the total count in terms of the size of the inputs, in this case rows and
cols. To make the count easier to decipher, we can combine counts that appear within a
single loop. This operation gives us Program 1.17.
24 Basic Concepts

float rsum(float list[], int n)

{
count++; /
* for if conditional */
if (n) {
count++; /*
for return and rsum invocation */
/
return rsum(list,n-1) + list[n-l];
}
count++;
return list[0] ;
}

Program 1.14: Program 1.11 with count statements added

For Program 1.17, we can see that if count is initially 0, it will be 2rows • cols +
2rows -I- 1 on termination. This analysis suggests that we should interchange the
matrices if the number of rows is significantly larger than the number of columns. □

By physically placing count statements within our functions we can run the func­
tions and obtain precise counts for various instance characteristics. Another way to
obtain step counts is to use a tabular method. To construct a step count table we first
determine the step count for each statement. We call this the steps/execution, or s/e for
short. Next we figure out the number of times that each statement is executed. We call
this the frequency. The frequency of a nonexecutable statement is zero. Multiplying s/e
by the frequency, gives us the total steps for each statement. Summing these totals,
gives us the step count for the entire function. Although this seems like a very compli­
cated process, in fact, it is quite easy. Let us redo our three previous examples using the
tabular approach.

Example 1.12 [Iterative function to sum a list of numbers]: Figure 1.2 contains the
step count table for Program 1.10. To construct the table, we first entered the
steps/execution for each statement. Next, we figured out the frequency column. The for
loop at line 5 complicated matters slightly. However, since the loop starts at 0 and ter­
minates when i is equal to n, its frequency is n + 1. The body of the loop (line 6) only
executes n times since it is not executed when i = n. We then obtained the total steps for
each statement and the final step count. □

Example 1.13 [Recursive function to sum a list of numbers]: Figure 1.3 shows the step
count table for Program 1.12. □

Example 1.14 [Matrix addition]: Figure 1.4 contains the step count table for the matrix
addition function. □
 Performance Analysis 25

void add{int a[][MAX-SIZE], int b[][MAX-SIZE],

int c[] [MAX—SIZE] , int rows, int cols)
{
int i, 3 ;
for (i = 0; 1 < rows;
i rows; i++)
for (j = 0; j< cols; j++)
c[i] [j] = a[i] [j] + b[i] [j] ;
}

Program 1.15: Matrix addition

void add(int a[][MAX-SIZE], int b[][MAX-SIZE],

int c[] [MAX—SIZE], int rows, int cols)
{
int i, i ;
for (i = 0; i rows; i++) {
count++; for i for loop ■^ /
for (j = 0; j ; cols; j++) {
★
count++; ! for j for loop */
c[i][j] =a[i][j] +b[i][j];
count+ +; /'^ for assignment statement ■^/
}
*
count++; / last time of j for loop
)
★
count++; / last time of i for loop */
}

Program 1.16: Matrix addition with count statements

EXERCISES

1. Redo Exercise 2, Section 1.2 (Homer’s rule for evaluating polynomials), so that
step counts are introduced into the function. Express the total count as an equation.
2. Redo Exercise 3, Section 1.2 (truth tables), so that steps counts are introduced into
the function. Express the total count as an equation.
3. Redo Exercise 4, Section 1.2 so that step counts are introduced into the function.
Express the total count as an equation.
26 Basic Concepts

void add(int a [ HMAX-SIZE] , int b [ HMAX-SIZE] ,

int c[] [MAX—SIZE] , int rows, int cols)
{
int i, j ;
for (i - 0; i < rows ; i++) {
for (j = 0; j < cols; 3++)
count += 2;
count += 2;
}
count++;
}

Program 1.17: Simplification of Program 1.16

Statement s/e Frequency Total steps

float sum(float list[], int n) 0 0 0
{ 0 0 0
float tempsum = 0; 1 1 1
int i; 0 0 0
for (i = 0; i < n; i++) 1 n+1 n+1
tempsum += list[il; 1 n n
return tempsum; 1 1 1
2 0 0 0
Total 2rt+3

Figure 1.2: Step count table for Program 1.10

4. (a) Rewrite Program 1.18 so that step counts are introduced into the function.
(b) Simplify the resulting function by eliminating statements.
(c) Determine the value of count when the function ends.
(d) Write the step count table for the function.
5. Repeat Exercise 5 with Program 1.19.
6. Repeat Exercise 5 with Program 1.20
 Performance Analysis 27

Statement s/e Frequency Total steps

float rsumffloat list[], int n) 0 0 0
{ 0 0 0
if (n) 1 n -1-1 n -1-1
return rsum(list,n-l) + listfn—1 ]; 1 n n
return listfO]; 1 1 1
} 0 0 0
Total 2n +2

Figure 1.3 : Step count table for recursive summing function

Statement s/e Frequency Total Steps

void add(int a[][MAX-SIZE] ••• ) 0 0 0
0 0 0
int i,j; 0 0 0
for (i=0; i<rows; i++) 1 rows+l r<7W5-l-l
for (j = 0; j < cols; ]++) 1 rows • (cols+\) rows • cols + rows
c(i]U] =a[i]0] + b[i]Ul; 1 rows • cols rows • cols
j 0 0 0
Total 2rows • cols + 2rows+i

Figure 1.4 : Step count table for matrix addition

7. Repeat Exercise 5 with Program 1.21

Summary

The time complexity of a program is given by the number of steps taken by the program
to compute the function it was written for. The number of steps is itself a function of the
instance characteristics. While any specific instance may have several characteristics
(e.g., the number of inputs, the number of outputs, the magnitudes of the inputs and out­
puts, etc.), the number of steps is computed as a function of some subset of these. Usu­
ally, we choose those characteristics that are of importance to us. For example, we might
wish to know how the computing (or run) time (i.e., time complexity) increases as the
28 Basic Concepts

void print—matrix{int matrix[][MAX-SIZE], int rows,

int cols)
{
int i, j ;
for = 0; i < rows
(1=0; rows; i++) {
for (j =0;
0; j < cols; j-i-+)
printf("%d",matrix[i][j]);
printf("\n");
}
}

Program 1.18: Printing out a matrix

void mult(int a[][MAX-SIZE], int b[][MAX-SIZE],

int c[][MAX-SIZE])
{
int i, j, k;
for {i = 0; i < MAX-SIZE; i + +)
for (j - I0; j - MAX-SIZE; j-n-) {
c[i] [ j ]1=0;
for (k - 0; k MAX-SIZE; k++)
c[i] [j] a[i] [k] * b[k] [j] ;
}
}

Program 1.19: Matrix multiplication function

number of inputs increase. In this case the number of steps will be computed as a func­
tion of the number of inputs alone. For a different program, we might be interested in
determining how the computing time increases as the magnitude of one of the inputs
increases. In this case the number of steps will be computed as a function of the magni­
tude of this input alone. Thus, before the step count of a program can be determined, we
need to know exactly which characteristics of the problem instance are to be used.
These define the variables in the expression for the step count. In the case of sum,
chose to measure the time complexity as a function of the number, n, of elements being
added. For function add the choice of characteristics was the number of rows and the
number of columns in the matrices being added.
Once the relevant characteristics (n, m, p, q, f\ . ..) have been selected, we can
define what a step is. A step is any computation unit that is independent of the charac­
teristics (n, m, p. q, r,. . .). Thus, 10 additions can be one step; 100 multiplications can
 Performance Analysis 29

void prod(int a[] [MAX—SIZE] , int b[] [MAX-SIZE] ,

int c[][MAX—SIZE], int rowsa, int colsb, int colsa)
{
int i, 3, k;
for (i = 0; i < rowsa; i + +}
for (j- I0; 3 ' colsb; 3++) {
c[i] [ j ]1=0;
for (k - 0; k colsa; k++}
c[i] [j] += a[i] [k] •k b[k] [j] ;
}
}

Program 1.20: Matrix product function

void transpose(int a[j [MAX-SIZE]}

{
int i, j, temp;
for (i = 0; i < MAX-SIZE-1; i++)
for (j i + 1;
- i+1; j < MAX-SIZE; j++}
SWAP(a[i][j], a[j][i], temp);
}

Program 1.21: Matrix transposition function

also be one step; but n additions cannot. Nor can m/2 additions, p +q subtractions, etc.,
be counted as one step.
The examples we have looked at so far were sufficiently simple that the time com­
plexities were nice functions of fairly simple characteristics like the number of elements,
and the number of rows and columns. For many programs, the time complexity is not
dependent solely on the number of inputs or outputs or some other easily specified
characteristic. Consider the function binsearch. (Program 1.6). This function searches an
ordered list. A natural parameter with respect to which you might wish to determine the
step count is the number, n, of elements in the list. That is, we would like to know how
the computing time changes as we change the number of elements n. The parameter n is
inadequate. For the same n, the step count varies with the position of the element
searchnum that is being searched for. We can extricate ourselves from the difficulties
resulting from situations when the chosen parameters are not adequate to determine the
step count uniquely by defining three kinds of steps counts: best case, worst case and
average.
30 Basic Concepts

The best case step count is the minimum number of steps that can be executed for
the given paramenters. The worst case step count is the maximum number of steps that
can be executed for the given paramenters. The average step count is the average
number of steps executed on instances with the given parameters.

1.4.3 Asymptotic Notation (O, Q, 0)

Our motivation to determine step counts is to be able to compare the time complexities
of two programs that compute the same function and also to predict the growth in run
time as the instance characteristics change.
Determining the exact step count (either worst case or average) of a program can
prove to be an exceedingly difficult task. Expending immense effort to determine the
step count exactly isn’t a very worthwhile endeavor as the notion of a step is itself inex­
act. (Both the instructions x = y and x = y -I- z + {x/y) -I- (jc *y *z -x/z) count as one step.)
Because of the inexactness of what a step stands for, the exact step count isn’t very use­
ful for comparative purposes. An exception to this is when the difference in the step
counts of two programs is very large as in 3n+3 versus 100a7+10. We might feel quite
safe in predicting that the program with step count l>n +3 will run in less time than the
one with step count lOOn +10. But even in this case, it isn’t necessary to know that the
exact step count is lOOn +10. Something like, “it’s about 80n, or 85rt, or 75n,’’ is ade­
quate to arrive at the same conclusion.
For most situations, it is adequate to be able to make a statement like c^n^ < Tp{n}
< C'yn^ or rg(Az,m) = C\n + c^.^ where Cj and c^ are nonnegative constants. This is so
because if we have two programs with a complexity of c^n^ + C2.n and c^n, respectively,
then we know that the one with complexity c^n will be faster than the one with complex­
ity C\n^ + c^n for sufficiently large values of n. For small values of n, either program
could be faster (depending on Cj, c^, and C3). If Ci = 1, C2 = 2, and c^ = 100 then C]n^
+ C2n < c^n for n < 98 and + C2n > c^n for n > 98. If cj = 1, C2 = 2, and C3 = 1000,
then + C2n < c^n for n < 998.
No matter what the values of Ci, C2, and C3, there will be an n beyond which the
program with complexity C2,n will be faster than the one with complexity c^n^ + C2n.
This value of n will be called the break even point. If the break even point is 0 then the
program with complexity €3^ is always faster (or at least as fast). The exact break even
point cannot be determined analytically. The programs have to be run on a computer in
order to determine the break even point. To know that there is a break even point it is
adequate to know that one program has complexity Cin + C2n and the other c^^n for
some constants cj, C2, and C3. There is little advantage in determining the exact values
of C], C2, and C3.
With the previous discussion as motivation, we introduce some terminology that
will enable us to make meaningful (but inexact) statements about the time and space
complexities of a program. In the remainder of this chapter, the functions f and g are
nonnegative functions.
 Performance Analysis 31

Definition; [Big “oh”]/(«) = O(g(A2)) (read as “/of n is big oh of g of n”) iff (if and
only if) there exist positive constants c and Hq such that/(n) < eg («) for all n, n > Hq. □

Example 1.15: 3m + 2 = O(m) as 3m + 2 < 4m for all m > 2. 3m + 3 = O(m) as 3m + 3 < 4m

for all M > 3. 100m + 6 = as 100m + 6 < 101m for n > 10. IOm^ + 4m + 2 = O(n^) as
IOm^ + 4m + 2 < 1 Im^ for m > 5. IOOOm^ + 100m - 6 = O(m2) as IOOOm^ + 100m - 6 <
IOOIm^ for M > 100. 6 2" + M^ = 0(2”) as 6*2'^ + M^ < 7
* 2' ’for M > 4. 3M + 3 = O(M2)as
*
3m + 3 < 3m2 for m > 2. IOm^ + 4m + 2 = ©(m*^) as IOm^ + 4m + 2 < IOm"^ for m > 2. 3n + 2
7^: 0(1) as 3m + 2 is not less than or equal to c for any constant c and all m, m > mq. IOm^^-
4« + 2 5feO(n). □

We write 0(1) to mean a computing time which is a constant. O(m) is called

linear, O(m2) is called quadratic, O(m^) is called cubic, and 0(2") is called exponential.
If an algorithm takes time O(Iog m) it is faster, for sufficiently large m, than if it had taken
O(m). Similarly, O(m logM) is better than O(m2) but not as good as O(m). These seven
computing times, 0(1), O(logM), O(m), O(m log m), O(m2), O(m^), and 0(2") are the
ones we will see most often in this book.
As illustrated by the previous example, the statement/(m) = O(g(M)) only states
that g (m) is an upper bound on the value of/(M) for all m, m > mq. It doesn’t say anything
about how good this bound is. Notice that n = n= n = O(m^), m = 0(2"),
etc. In order for the statement /(m) = 0(g (m)) to be informative, g (n) should be as small
a function of m as one can come up with for which/(m) = O(g(M)). So, while we shall
often say 3m + 3 = 0(m), we shall almost never say 3m + 3 = O(m2) even though this
latter statement is correct.
From the definition of O, it should be clear that/(M) = O(g(M)) is not the same as
O(g (m)) = /(m). In fact, it is meaningless to say that O(g (m)) =f{n}. The use of the sym­
bol is unfortunate as this symbol commonly denotes the “equals” relation. Some
of the confusion that results from the use of this symbol (which is standard terminology)
can be avoided by reading the symbol “=” as “is” and not as “equals.”
Theorem 1.2 obtains a very useful result concerning the order of/(M) (i.e., the
g {n} in/(m) = O(g (m))) when/(M) is a polynomial in m.

Theorem 1.2: If/(M)= m^m"' + . . . + a^n + Aq, then/(n) = O(zi"').

m
Proof:/(m)< 2 I My |m'
z=0

m
I-tn
1^/1^
0

m
< I «,■ I , for M > 1
0
32 Basic Concepts

So,/(n) = O(rt'"). □

Definition: [Omega] /(«) = O(g (n)) (read as “/ of n is omega of g of n”) iff there exist
positive constants c and riQ such that/(rt) >cg(n) for all h, n > mq- □

Example 1.16: 3n + 2 = £l(n) as 3n + 2> 3n forn> 1 (actually the inequality holds for
n > 0 but the definition of Q. requires an no>O). 3« + 3 = as 3n + 3 > 3n for n > 1.
lOOn + 6 = O(n) as lOOn + 6 > lOOn for n > 1. lOn^ + 4n + 2 = as 10«^ + 4/1 + 2 >
for /I > 1. 6*
2" + = 0(2") as 6*2" + >2^ for n > 1. Observe also that 3/1 + 3 =
0(1); 10/1^ + 4/1 + 2 = O(/i); lO/i^ + 4^ + 2 = 0(1); 6 2" +
* ^);
= O(/i
** 6^2'^ + =
6^2^ +n^ = 6^2^ + = Q.(n); and 6*
2" + /i^ = 0(1). □

As in the case of the “big oh’’ notation, there are several functions g (n) for which
/(/i) = O(g (/i)). g (n) is only a tower bound on/(/i). For the statement/(/i) = O(g (/i)) to
be informative, g(/2) should be as large a function of n as possible for which the state­
ment /(/i) = O(g(/i)) is true. So, while we shall say that 3n + 3 - and that
2" + /i^ = 0(2"), we shall almost never say that 3n + 3 = 0(1) or that 6*
6
* 2" + /i^ =
0(1) even though both these statements are correct.
Theorem 1.3 is the analogue of Theorem 1.2 for the omega notation.

Theorem 1.3: If/(/i) = + . .. + a J n + aQ and > 0, then/(n) = Q.{n^').

Proof: Left as an exercise. □

Definition: [Theta]/(n) = Q(g(n)) (read as “/of n is theta of g of n"') iff there exist
positive constants c j, c » and n o such that c j (n) <f(n) < c 2g («) for all n, n > n o. □

Example 1.17: 3n + 2 = 0(n) as 3n + 2 > 3n for all n > 2 and 3n + 2 < 4n for all n > 2, so
Cl 3, C2 = 4, and «o = 2. 3n + 3 = 0(n); lOn^ + 4n + 2 = ©(n^); 6
2" +
* = 0(2"); and
log
10
* n + 4 = 0(log n). 3n + 2 0(1); 3n + 3 + 4n + 2 0(n); lOn^ +
4n + 2 0(1); 6 2" +
* *
2"
6 '“); and 6*
2" + 0(1). □

The theta notation is more precise than both the “big oh” and omega notations.
/(«) = 0(g (n)) iff g (n) is both an upper and lower bound on/(/i).
Notice that the coefficients in all of the g(/i)’s used in the preceding three exam­
ples has been 1. This is in accordance with practice. We shall almost never find our­
selves saying that 3n + 3 = O(3n), or that 10 = 0(100), or that lO/i^ + 4^2 + 2 = 0(4/12),
or that 62"
* + /i^ = 0(6
*
2"), or that 6*
2" + = 0(4
2"),
* even though each of these
statements is true.

Theorem 1.4: If/(n) = 6Z^« m + . . . + + tio and > 0, then/(rt) = 0(n"').

Proof: Left as an exercise. □

 Performance Analysis 33

Let us reexamine the time complexity analyses of the previous section. For func­
tion sum (Program 1.11) we had determined that T^umM = 2n+3. So, ^^^^(h) = 0(z7).
T rsum (z?) = 2n + 2 = ©(«) and Taddk^ows, cols') = Irows.cols + Irows + 1 = ^{rows.cols}.
While we might all see that the O, Q, and 0 notations have been used correctly in
the preceding paragraphs, we are still left with the question: “Of what use are these
notations if one has to first determine the step count exactly?” The answer to this ques­
tion is that the asymptotic complexity (i.e., the complexity in terms of O, O, and 0) can
be determined quite easily without determining the exact step count. This is usually
done by first determining the asymptotic complexity of each statement (or group of state­
ments) in the program and then adding up these complexities.

Example 1.18 [Complexity of matrix addition}'. Using a tabular approach, we construct

the table of Figure 1.5. This is quite similar to Figure 1.4. However, instead of putting
in exact step counts, we put in asymptotic ones. For nonexecutable statements, we enter
a step count of 0. Constructing a table such as the one in Figure 1.5 is actually easier
than constructing the one is Figure 1.4. For example, it is harder to obtain the exact step
count of row5.(coZ5 + l) for line 5 than it is to see that line 5 has an asymptotic complex­
ity that is Q{rows.cols}. To obtain the asymptotic complexity of the function, we can add
the asymptotic complexities of the individual program lines. Alternately, since the
number of lines is a constant (i.e., is independent of the instance characteristics), we may
simply take the maximum of the line complexities. Using either approach, we obtain
Q{rows.cols} as the asymptotic complexity. □

Statement Asymptotic complexity

void add(int a[][MAX_SIZE] • • • ) 0
0
int i,j; 0
for (i=0; i<rows; i++) Q(rows)
for (j = 0; j < cols; j++) Q(rows.cols)
c[ilU] = a[i]UJ +b[i]|j]; Q^rows.cols)
} 0
Total Q(rows.cols)

Figure 1.5: Time complexity of matrix addition

Example 1.19 [Binary search}'. Let us obtain the time complexity of the binary search
function hinseareh (Program 1.6). The instance characteristic we shall use is the number
n of elements in the list. Each iteration of the while loop takes ©(1) time. We can show
that the while loop is iterated at most [ log2(n + l) ] times (see the book by S. Sahni
cited in the references). Since an asymptotic analysis is being performed, we don't need
34 Basic Concepts

such an accurate count of the worst case number of iterations. Each iteration except for
the last results in a decrease in the size of the segment of list that has to be searched by a
factor of about 2. That is, the value of right - left + 1 reduces by a factor of about 2 on
each iteration. So, this loop is iterated 0(log n) times in the worst case. As each itera­
tion takes 0(1) time, the overall worst case complexity of binsearch is 0(log n). Notice
that the best case complexity is 0(1) as in the best case searchnum is found in the first
iteration of the while loop. □

Example 1.20 [Permutations]'. Consider function perm (Program 1.8). When i = n, the
time taken is 0(n). When i < n, the else clause is entered. The for loop of this clause is
entered n - i + 1 times. Each iteration of this loop takes 0(az + Tp^nn (/ -I-1, n)) time. So,
T'pertn^h uf =■ 0((zi — Z + l)(n + Tpgf.ff^ (i -I-1, n))) when i < n. Since, Tp^rm (/ -I- 1, n), is at
least n when i + 1 < n, we get Tp^rm (z, n) = 0((z2 -i -I- i)Tp^rm(i + Solv-
ing this recurrence, we obtain Tp^rm (1,«) = 0(zi (rt !)), > 1. n

Example 1.21 [Magic square]'. As our last example of complexity analysis, we use a
problem from recreational mathematics, the creation of a magic square.
A magic square is an n x n matrix of the integers from 1 to n^ such that the sum of
each row and column and the two major diagonals is the same. Figure 1.6 shows a magic
square for the case n = 5. In this example, the common sum is 65.

15 8 1 24 17
16 14 7 5 23
22 20 13 6 4
3 21 19 12 10
9 2 25 18 11

Figure 1.6: Magic square for n = 5

Coxeter has given the following simple rule for generating a magic square when n
is odd:

Put a one in the middle box of the top row. Go up and left assigning numbers in
increasing order to empty boxes. If your move causes you to jump off the square (that is,
you go beyond the square's boundaries), figure out where you would be if you landed on
a box on the opposite side of the square. Continue with this box. If a box is occupied, go
down instead of up and continue.
 Performance Analysis 35

We created Figure 1.6 using Coxeter’s rule. Program 1.22 contains the coded
algorithm. Let n denote the size of the magic square (i.e., the value of the variable size
in Program 1.22. The if statements that check for errors in the value of n take 0(1) time.
The two nested for loops have a complexity ©(n^). Each iteration of the next for loop
takes 0(1) time. This loop is iterated ©(n^) time. So, its complexity is ©(n^). The
nested for loops that output the magic square also take ©(n^) time. So, the asymptotic
complexity of Program 1.22 is ©(n^). □

#include stdio.h>
#define MAX-SIZE 15 /
* maximum size of square */
void main(void)
/
* construct a magic square, iteratively *
/
{
Static int square[MAX-SIZE][MAX-SIZE];
int i, jj,Z row, column; /■^
/
* indices
int count ; /
*
/ counter ■^ /
int size; *
/ Square size */

printf("Enter the size of the square:

scanf("%d", Scsize) ;
check for input errors
if (size 1 I I size MAX-SIZE +1) {
fprintf(stderr, "Error 1 Size is out of rangeXn");
exit(1);
}
Q,
if (!(size2) ) {•Q

fprintf(stderr, Error! Size is evenin'');

exit(1);
}
for (i = 0; i s 1 z e ; i++)
for (j 0; j < size; j++)
square[i][j] 0;
square[0][(size-1) 1; / ★ middle of
/ 2] first row
*
/ i and j are current position */
i 0;
j = (size-1) / 2;
* size; count++) {
for (count = 2; count <= size
row = (i-1 < 0) ? ((size - 1) : (i - 1); up
/
*
column = (j-i
(j-1 < 0) ? (size - 1) : (j - 1) ; left
/
*
if (square[row][column]) down
I* /
i = (++i) % size;
else { *
/ square is unoccupied
i = row;
36 Basic Concepts

j (j-1 0) ? (size - 1) —j ;
}
square[i][j] count;
}
/
* output the magic square */
printf(" Magic Square of size %d : \n\n", size);
for Q; 1i < size; i + +) <
(i = 0;
for (j = 0; j < size; j++)
printf ( "%5d'’, square[i] [j] ) ;
printf("\n");
}
printf("\n\n");
}

Program 1.22: Magic square program

When we analyze programs in the following chapters, we will normally confine

ourselves to providing an upper bound on the complexity of the program. That is, we will
normally use only the big oh notation. We do this because this is the current trend in
practice. In many of our analyses the theta notation could have been used in place of the
big oh notation as the complexity bound obtained is both an upper and a lower bound for
the program.

EXERCISES

1. Show that the following statements are correct:

(a) 5n^ -6n =
(b)
(c) 2^2 + n log n =
n
(d)
i=Q
n
(e)
i=Q

(f) n 2" + 6-2" = 0(^2")

(g) n-"3 + lO^n^ =
(h) ! (log n + 1) = O(n^)
(i) n +A7 log/i = 0(n^-^
* )
 Performance Analysis 37

(j) +n+ n = 0(«^logz2) for all > 1.

(k) lOn^ + 15n4 H-lOOn^?” = O(n^2^)
2. Show that the following statements are incorrect:
(a) lOn^ + 9 = O(n)
(b) M^log n =
(c) /io^n- Q(n^)
(d) n^2^ + 6n^3^ = O(n^2'^}
(e) 3" =0(2'^)
3. Prove Theorem 1.3.
4. Prove Theorem 1.4.
5. Determine the worst case complexity of Program 1.18.
6. Determine the worst case complexity of Program 1.21.
7. Compare the two functions and 20z2 +4 for various values of n. Determine
when the second function becomes smaller than the first.
8. Write an equivalent recursive version of the magic square program (Program
1,22).

1.4.4 Practical Complexities

Wq have seen that the time complexity of a program is generally some function of the
instance characteristics. This function is very useful in determining how the time
requirements vary as the instance characteristics change. The complexity function may
also be used to compare two programs P and Q that perform the same task. Assume that
program P has complexity 0(n) and program Q is of complexity We can assert
that program P is faster than program Q for “sufficiently large” n. To see the validity of
this assertion, observe that the actual computing time of P is bounded from above by cn
for some constant c and for all n, n > n i, while that of Q is bounded from below by dn^
for some constant d and all n, « > n2. Since cn < dn^ for n > c/d, program P is faster
than program Q whenever n > max{n ।, n2, c/d}.
You should always be cautiously aware of the presence of the phrase “sufficiently
large” in the assertion of the preceding discussion. When deciding which of the two
programs to use, we must know whether the n we are dealing with is, in fact,
“sufficiently large.” If program P actually runs in milliseconds while program Q
runs in n^ milliseconds and if we always have n < 10^, then, other factors being equal,
program Q is the one to use, other factors being equal.
To get a feel for how the various functions grow with n, you are advised to study
Figures 1.7 and 1.8 very closely. As you can see, the function 2" grows very rapidly
with n. In fact, if a program needs 2" steps for execution, then when n = 40, the number
of steps needed is approximately 1.1* 10^^. On a computer performing 1 billion steps per
38 Basic Concepts

second, this would require about 18.3 minutes. If n = 50, the same program would run
for about 13 days on this computer. When n = 60, about 310.56 years will be required to
execute the program and when n = 100, about 4*10^^ years will be needed. So, we may
conclude that the utility of programs with exponential complexity is limited to small n
(typically n < 40).

Instance characteristic n

Time Name 1 2 4 8 16 32

1 Constant 1 1 1 1 1 1
log n Logarithmic 0 1 2 3 4 5
n Linear I 2 4 8 16 32
n log n Log linear 0 2 8 24 64 160
Quadratic 1 4 16 64 256 1024
Cubic 1 8 64 512 4096 32768

2” Exponential 2 4 16 256 65536 4294967296

n! Factorial 1 2 24 40326 20922789888000 26313 X 10^^

Figure 1.7 Function values

Programs that have a complexity that is a polynomial of high degree are also of
limited utility. For example, if a program needs steps, then using our 1 billion steps
per second computer we will need 10 seconds when n = 10; 3,171 years when n = 100;
and 3.17
*10^^ years when n = 1000. If the program’s complexity had been steps
instead, then we would need 1 second when n = 1000; 110.67 minutes when n = 10,000;
and 11.57 days when n = 100,000.
Figure 1.9 gives the time needed by a 1 billion instructions per second computer to
execute a program of complexity f(n) instructions. You should note that currently only
the fastest computers can execute about 1 billion instructions per second. From a practi­
cal standpoint, it is evident that for reasonably large n (say n > 100), only programs of
small complexity (such as n, nlogn, n^, are feasible. Further, this is the case even if
one could build a computer capable of executing 10^^ instructions per second. In this
case, the computing times of Figure 1.9 would decrease by a factor of 1000. Now, when
n = 100 it would take 3.17 years to execute 10^^
instructions, and 4
* 0 years to execute
2" instructions.

1.5 PERFORMANCE MEASUREMENT

Although performance analysis gives us a powerful tool for assessing an algorithm’s

space and time complexity, at some point we also must consider how the algorithm exe­
cutes on our machine. This consideration moves us from the realm of analysis to that of
 Performance Measurement 39

2”

60

50

40

wlogn
to
30
f

20

n
10

logn
o ■©- -©■ -o
0 J 1 ±
1 J
2 3 4 5 6 7 8 9 10
n

Figure 1.8 Plot of function values

measurement. We will concentrate our discussion on measuring time.

The functions we need to time events are part of C’s standard library, and are
accessed through the statement: ^include <time.h>. There are actually two different
methods for timing events in C. Figure 1.10 shows the major differences between these
two methods.
Method 1 uses clock to time events. This function gives the amount of processor
time that has elapsed since the program began running. To time an event we use clock
twice, once at the start of the event and once at the end. The time is returned as a built-in
type, clock-t. The total time required by an event is its start time subtracted from its
stop time. Since this result could be any legitimate numeric type, we type cast it to
40 Basic Concepts

CZ5 « u .£
c 5E T3
04 A £ &0
»-
*
L —
co co 22
-

II 00 "d 00
se o
*
*
04
CO

u
Um Um
o z £ 13 13
0'^CO'0’”* '^f
2 cn m f<i
Si CO Lrdood
II C<)
04 so ro
(U 04 * * * *
O' O' O' O'
3
O, f<i m c<)
£
o
u
Q 72727252525-'?- t-J-
5^ ±-:S.±.£ £ £ £"O
"S' Si O O O so IT) Q O~' 'O'
II so T— >02 04 o
CZ5
•
C s: 04 so fo * V2
O' (ZJ

co
C -2
§ g

cc u 6
C 1
o .C
(Z1

72 Si
72 72
±.
72
±.
72
±.
72
±.
52
£
U
« o I
c II CZ3 £ "O >> O
.2 <X>
■■ ■
O' 3 lO
Si 04 04 ''O 'T) O' fl
u d — If
s ■£
—’ co
o S
72 y O
C y y 72 72
u .£ 72 i> *0 y
72 72 72 72 72 725222 P 52 C 3 S 52 fa
Si Si
11
±.±.±.±.±.±.£ E •u -
72
O O'
e
_ y
e"“
S
’£ <u
<u
oP . t£
e O5
c fa JS
<3 <L>5ft
—• C 72
72 C JS T3
Si 04
o d II II IIII II IIII IIII II
'<1 2 o c 2=
fa
04
£ =i- £ g >>
H Si
72 72 72 72 72 72 72 72 5252
00 ±.±.i:i.±.±.±.=i.£ £
O COOSlO^OOKOOCO^Ol
'll O0'-
04
* 04'OO^C’''00^
d d d
Si co —

72 72 72 72 72 72 72 72 72 52
Si
II S.a=LAa.=L=LaL=L£
<O) O
-^04c0-3'iO)OQOQO
O
OOOOO O ’-'OOOO o
s; - d 8 "■
................................

o o o o o
04 co Tj- IT) §§§§§
s: —■ O■§§

Figure 1.9 Times on a 1 billion instruction per second computer

 Performance Measurement 41

Method 1 Method 2
Start timing start = clockO; start = time(NULL);
Stop timing stop = clockO; stop = time(NULL);
Type returned clock-t time-t
Result in seconds duration = duration =
((double) (stop-start)) / (double) difftime(stop,start);
CLK-TCK;

Figure 1.10: Event timing in C

double. In addition, since this result is measured as internal processor time, we must
divide it by the number of clock ticks per second to obtain the result in seconds. On our
compiler the ticks per second is held in the built-in constant, CLK-TCK. We found that
this method was far more accurate on our machine. However, the second method does
not require a knowledge of the ticks per second, which is why we also present it here.
Method 2 uses time. This function returns the time, measured in seconds, as the
built-in type time-t. Unlike clock, time has one parameter, which specifies a location to
hold the time. Since we do not want to keep the time, we pass in a NULL value for this
parameter. As was true of Method 1, we use time at the start and the end of the event we
want to time. We then pass these two times into diffiime, which returns the difference
between two times measured in seconds. Since the type of this result is time-t, we type
cast it to double before printing it out.
The exact syntax of the timing functions varies from computer to computer and
also depends on the operating system and compiler in use. For example, the constant
CLK-TCK does not exist on a SUN Sparcstation running SUNOS 4.1 Instead, the
clock() function returns the time in microseconds. Similarly, the function difliime() is
not available and one must use (stop-start) to calculate the total time taken.
We now want to look at two examples of event timing. In each case, we analyze
the worst case performance.

Example 1.22 case performance of the selection function}” . The worst case for
selection sort occurs when the elements are in reverse order. That is, we want to sort
into ascending order an array that is currently in descending order. To conduct our tim­
ing tests, we varied the size of the array from 0, 10, 20 , • • • ,90, 100, 200 , ■ • ■ , 1600.
Program 1.23 contains the code we used to conduct the timing tests. (We have not
included the sort function code again since it is found in Program 1.3).
To conduct the timing tests, we used a for loop to control the size of the array. At
each iteration, a new reverse ordered array of sizelist [/1 was created. We called clock
immediately before we invoked sort and immediately after it returned. The results of the
tests are displayed in Figures 1.11 and 1.12. The tests were conducted on an IBM
42 Basic Concepts

#include <stdio.h>
#include <time.h>
#define MAX_SIZE 1601
#define ITERATIONS 26
#define SWAP(x, y, t) ((t) (x) , (X) (y), (y) (t))
void main(void)
{
int i,j,position;
int list[MAX_SIZE];
int sizelist[] {0, 10, 20, 30, 40, 50, 60, 70, 80, 90,
100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100,
1200, 1300, 1400, 1500, 1600};
clock_t start, stop;
double duration;
printf {’’ n time\n");
for (i = 0; i < ITERATIONS; i++) {
for {j = 0; j < sizelist[i]; j++)
list[j] = sizelist[i] - j;
start clock(};
sort(list,sizelist[i]);
stop = clock ();
/
* CLK_TCK = number of clock ticks per second *
/
duration = ((double) (stop-start)) / CLK_TCK;
printf ('’%6d %f\n",sizelist[i], duration);
}
}

Program 1.23: Timing program for the selection sort function

compatible PC with an 80386 cpu, an 80387 numeric coprocessor, and a turbo accelera­
tor. We used Borland’s Turbo C compiler.
What confidence can we have in the results of this experiment? The measured
time for n < 100 is zero. This cannot be accurate. The sort time for all n must be larger
than zero as some nonzero amount of work is being done. Furthermore, since the value
of CLK -TCK is 18 on our computer, the number of clock ticks measured for n < 500 is
less than 10. Since there is a measurement error of ±1 tick, the measured ticks for n <
500 are less than 10% accurate. In the next example, we see how to obtain more accu­
rate times.
The accuracy of our experiment is within 10% for n 500. We may regard this as
acceptable. The curve of Figure 1.12 resembles the curve displayed in Figure 1.8.
This agrees with our analysis of selection sort. The times obtained suggest that while
selection sort is good for small arrays, it is a poor sorting algorithm for large arrays. □
 Performance Measurement 43

n Time n Time
30 ■■■ 100 .00 1.86
~ 200 .11 1000 2.31
300 .22 1100 2.80
400 ■38 1200 3.35
500 .60 1300 3.90
600 .82 1400 434
700 1.15 iW 5.22
1.48 1600 5.93

Figure 1.11 : Worst case performance of selection sort (in seconds)

tine

7
6 --

5 --

4 -

3 --

2-
1 --

-d— -- 1— -I---
0 500 1000 1500 2000

n
>

Figure 1.12 : Graph of worst case performance for selection sort

Example 1.23 [WorsZ case performance of sequential search]'. As pointed out in the
last example, the straightforward timing scheme used to time selection sort isn't ade­
quate when we wish to time a function that takes little time to complete. In this exam­
ple, we consider a more elaborate timing mechanism. To illustrate this, we consider
obtaining the worst case time of a sequential search function seqsearch (Program 1.24).
This begins at the start of the array and compares the number it seeks, searchnum-, with
the numbers in the array until it either finds this number or it reaches the end of the array.
The worst case for this search occurs if the number we seek is not in the array. In this
case, all the numbers in the array are examined, and the loop is iterated n •+• I times.
44 Basic Concepts

int seqsearch(int list[], int searchnum, int n)

{
/^search an array, list, that has n numbers. Return i,
if list[i] searchnum. Return —1, if searchnum is not in
the list ■*/
int i;
list[n] searchnum;
I =
for (i = 0; list[i] searchnum; i++)

return ((i < n ) ? i : -1);

}

Program 1.24: Sequential search function

Since searching takes less time than sorting, the timing strategy of Program 1.23 is
inadequate even for small arrays. So, we needed another method to obtain accurate
times. In this case, the obvious choice was to call the search function many times for
each array size. Since the function runs more quickly with the smaller array sizes, we
repeated the search for the smaller sizes more times than for the larger sizes. Program
1.25 shows how we constructed the timing tests. It also gives the array sizes used and
the number of repetitions used for each array size. Note the need to reset element
/ist [sizeiist [Z ]] after each invocation of seqsearch.
The number of repetitions is controlled by numtimes. We started with 30,000
repetitions for the case n = 0 and reduced the number of repetitions to 200 for the largest
arrays. Picking an appropriate number of repetitions involved a trial and error process.
The repetition factor must be large enough so that the number of elapsed ticks is at least
10 (if we want an accuracy of at least 10%). However, if we repeat too many times the
total computer time required becomes excessive. Figure 1.13 shows the results of our
timing tests. These were conducted on an IBM PS/2 Model 50 using Turbo C. The
linear dependence of the times on the array size becomes more apparent for larger values
of n. This is because the effects of the constant additive factor is more dominant for
small n. □

Generating Test Data

Generating a data set that results in the worst case performance of a program isn’t always
easy. In some cases, it is necessary to use a computer program to generate the worst case
data. In other cases, even this is very difficult. In these cases, another approach to
estimating worst case performance is taken. For each set of values of the instance
characteristics of interest, we generate a suitably large number of random test data. The
run times for each of these test data are obtained. The maximum of these times is used
 Performance Measurement 45

#include stdio.h
#include time.h>
#define MAX-SIZE 1001
#define ITERATIONS 16
int seqsearch(int [], int, int);
void main(void)
{
int i, j, position;
int list[MAX-SIZE];
int sizelist[] {0, 10, 20, 30, 40, 50, 60, 70, 80, 90,
100, 200, 400, 600, 800, 1000};
int numtimes[] {30000, 12000, 6000, 5000, 4000, 4000,
4000, 3000, 3000, 2000, 2000,
1000, 500, 500, 500, 200};
clock—t start, stop;
double duration,total;
for (i = 0; 1 i < MAX-SIZE; i++)
list [ i]I = i;
for (i = 0; i ITERATIONS; i++) {
start = clock();
for (j = 0; j < numtimes [i];
numtimes[i]; j++)
position - seqsearch(list, -1, sizelist[i]};
stop = clock();
total ((double)(stop-start))/CLK—TCK;
duration = total/numtimes [i];
printf("%5d %d %d %f %f\n''. sizelist[i], numtimes[i],
(int)(stop-start), total,
duration);
list [sizelist[i]] = sizelist[i]; *
/ reset value */
}

Program 1.25: Timing program for sequential search

as an estimate of the worst case time for this set of values of the instance characteristics.
To measure average case times, it is usually not possible to average over all possi­
ble instances of a given characteristic. While it is possible to do this for sequential and
binary search, it is not possible for a sort program. If we assume that all keys are dis­
tinct, then for any given n ! different permutations need to be used to obtain the aver­
age time.
46 Basic Concepts

n Iterations Ticks Total time (sec) Duration

0 30000 16 0.879121 0.000029
10 12000 16 0.879121 0.000073
20 6000 14 0.769231 0.000128
30 5000 16 0.879121 0.000176
40 4000 16 0.879121 0.000220
50 4000 20 1.098901 0.000275
60 4000 23 1.263736 0.000316
70 3000 20 1.098901 0.000366
80 3000 23 1.263736 0.000421
90 2000 17 0.934066 0.000467
100 2000 18 0.989011 0.000495
200 1000 18 0.989011 0.000989
400 500 18 0.989011 0.001978
600 500 27 1.483516 0.002967
800 500 35 1.923077 0.003846
1000 300 27 1.483516 0.004945

Figure 1.13 : Worst case performance of sequential search

Obtaining average case data is usually much harder than obtaining worst case
data. So, we often adopt the strategy outlined above and simply obtain an estimate of the
average time.
Whether we are estimating worst case or average time using random data, the
number of instances that we can try is generally much smaller than the total number of
such instances. Hence, it is desirable to analyze the algorithm being tested to determine
classes of data that should be generated for the experiment. This is a very algorithm
specific task and we shall not go into it here.

EXERCISES
For each of the exercises that follow we want to determine the worst case performance.
Create timing programs that do this. For each program, pick arrays of appropriate sizes
and repetition factors, if necessary. Present you results in table and graph form, and
summarize your findings.
 Performance Measurement 47

1. Repeat the experiment of Example 1.23. This time make sure that all measured
times have an accuracy of at least 10%. Times are to be obtained for the same
values of n as in the example. Plot the measured times as a function of n.
2. Plot the run times of Figure 1.14 as a function of n.
3. Compare the worst case performance of the iterative (Program 1.10) and recursive
(Program 1.11) list summing functions.
4. Compare the worst case performance of the iterative (Program 1.6) and recursive
(Program 1.7) binary search functions.
5. (a) Translate the iterative version of sequential search (Program 1.24) into an
equivalent recursive function.
(b) Analyze the worst case complexity of your function.
(c) Measure the worst case performance of the recursive sequential search func­
tion and compare with the results we provided for the iterative version.
6. Measure the worst case performance of the matrix addition function (Program
1.15).
7. Measure the worst case performance of the matrix multiplication function (Pro­
gram 1.19).

1.6 SELECTED READINGS AND REFERENCES

For a discussion of programming techniques and how to develop programs, see D. Gries,
The Science of Programming, Springer Verlag, NY, 1981; E. Dijkstra, A Discipline of
Programming, Prentice-Hall, Englewood Cliffs, NJ 1976; and B. W. Kemighan and P. J.
Plauger, The Elements of Programming Style, Second Edition, McGraw Hill, NY 1978.
A good discussion of tools and procedures for developing very large software sys­
tems appears in the texts: E. Horowitz, Practical Strategies for Developing Very Large
Software Systems, Addison-Wesley, Reading, Mass., 1975; I. Sommerville, Software
Engineering, Third Edition, Addison-Wesley, Workingham, England, 1989; and F.
Brooks, The Mythical Man-Month, Addison-Wesley, Reading, Mass., 1979.
For a more detailed discussion of performance analysis and measurement, see S.
Sahni, Software Development in Pascal, Second Edition, Camelot Publishing, 1989.
For a further discussion of abstract data types see B. Liskov and J. Guttag,
Abstraction and Specification in Program Development, MIT Press, Cambridge, Mass.,
1988; J. Kingston, Algorithms and Data Structures, Addison-Wesley, Reading, Mass.,
1990, and D. Stubbs and N. Webre, Data Structures with Abstract Data Types and Pas­
cal, Brooks/Cole Publishing Co., Monterey, CA, 1985.
Writing a correct version of binary search is discussed in the papers J. Bentley,
'Programming pearls: Writing correct programs, CACM, vol. 26, 1983, pp. 1040-1045,
and R. Levisse, "Some lessons drawn from the history of the binary search algorithm".
The Computer Journal, vol. 26, 1983, pp, 154-163,
48 Basic Concepts

For a general discussion of permutation generation, see the paper R. Sedgewick,

"Permutation generation methods, Computer Surveys, vol. 9, 1977, pp. 137-164.