ANALYSIS OF ALGORITHM

CSC3422

What is an algorithm
Although there is no universally agreed-on wording to describe this notion, there
is general agreement about what the concept means:
An algorithm is a sequence of unambiguous instructions for solving a
problem, i.e., for obtaining a required output for any legitimate input in
a finite amount of time.

W hy do you need to study algorithms? If you are going to be a computer

professional, there are both practical and theoretical reasons to study algorithms.
From a practical standpoint, you have to know a standard set of important
algorithms from different areas of computing; in addition, you should be able to
design new algorithms and analyze their efficiency. From the theoretical standpoint,
the study of algorithms, sometimes called algorithmics, has come to be
recognized as the cornerstone of computer science.

Several important points to note when designing algorithm:

 The nonambiguity requirement for each step of an algorithm cannot be compromised.
 The range of inputs for which an algorithm works has to be specified carefully.
 The same algorithm can be represented in several different ways.
 There may exist several algorithms for solving the same problem.

 Algorithms for the same problem can be based on very different ideas and
can solve the problem with dramatically different speeds.

Recall that the greatest common divisor of two nonnegative, not-both-zero

integers m and n, denoted gcd(m, n), is defined as the largest integer that divides
both m and n evenly, i.e., with a remainder of zero. Euclid of Alexandria (third
century b.c.) outlined an algorithm for solving this problem in one of the volumes
of his Elements most famous for its systematic exposition of geometry. In modern
terms, Euclid’s algorithm is based on applying repeatedly the equality
gcd(m, n) = gcd(n, m mod n),

ATM online Page 1

where m mod n is the remainder of the division of m by n, until m mod n is equal
to 0. Since gcd(m, 0) = m (why?), the last value of m is also the greatest common
divisor of the initial m and n.
For example, gcd(60, 24) can be computed as follows:
gcd(60, 24) = gcd(24, 12) = gcd(12, 0) = 12.
(If you are not impressed by this algorithm, try finding the greatestcommondivisor
of larger numbers, such as those in Problem 6 in this section’s exercises.)

Here is a more structured description of this algorithm:

Euclid’s algorithm for computing gcd(m, n)
Step 1 If n = 0, return the value of m as the answer and stop; otherwise,
proceed to Step 2.
Step 2 Divide m by n and assign the value of the remainder to r.
Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.

Alternatively, we can express the same algorithm in pseudocode:

ALGORITHM Euclid(m, n)
//Computes gcd(m, n) by Euclid’s algorithm
//Input: Two nonnegative, not-both-zero integers m and n
//Output: Greatest common divisor of m and n
while n _= 0 do
r ←m mod n
m←n
n←r
return m

Fundamentals of algorithm problem solving

1. Understanding the Problem
2. Ascertaining the Capabilities of the Computational Device
3. Choosing between Exact and Approximate Problem Solving
4. Algorithm Design Techniques
5. Designing an Algorithm and Data Structures
6. Methods of Specifying an Algorithm
7. Proving an Algorithm’s Correctness
8. Analyzing an Algorithm
9. Coding an Algorithm

ATM online Page 2

Understanding the Problem
From a practical perspective, the first thing you need to do before designing an
algorithm is to understand completely the problem given. Read the problem’s
description carefully and ask questions if you have any doubts about the problem,
do a few small examples by hand, think about special cases, and ask questions
again if needed.

Ascertaining the Capabilities of the Computational Device

Once you completely understand a problem, you need to ascertain the capabilities
of the computational device the algorithm is intended for.

Choosing between Exact and Approximate Problem Solving

The next principal decision is to choose between solving the problem exactly or
solving it approximately. In the former case, an algorithm is called an exact algorithm;
in the latter case, an algorithm is called an approximation algorithm. Why
would one opt for an approximation algorithm? First, there are important problems
that simply cannot be solved exactly for most of their instances; examples
include extracting square roots, solving nonlinear equations, and evaluating definite
integrals. Second, available algorithms for solving a problem exactly can be
unacceptably slow because of the problem’s intrinsic complexity.

Algorithm Design Techniques

An algorithm design technique (or “strategy” or “paradigm”) is a general

approach to solving problems algorithmically that is applicable to a variety
of problems from different areas of computing.

Designing an Algorithm and Data Structures

While the algorithm design techniques do provide a powerful set of general approaches
to algorithmic problem solving, designing an algorithm for a particular
problem may still be a challenging task. Some design techniques can be simply
inapplicable to the problem in question. Sometimes, several techniques need to
be combined, and there are algorithms that are hard to pinpoint as applications
of the known design techniques. Even when a particular design technique is applicable,
getting an algorithm often requires a nontrivial ingenuity on the part of
the algorithm designer. With practice, both tasks—choosing among the general
techniques and applying them—get easier, but they are rarely easy.

ATM online Page 3

Methods of Specifying an Algorithm
Once you have designed an algorithm, you need to specify it in some fashion. In
Section 1.1, to give you an example, Euclid’s algorithm is described in words (in a
free and also a step-by-step form) and in pseudocode. These are the two options
that are most widely used nowadays for specifying algorithms.

Proving an Algorithm’s Correctness

Once an algorithm has been specified, you have to prove its correctness. That is,
you have to prove that the algorithm yields a required result for every legitimate
input in a finite amount of time.

Analyzing an Algorithm
We usually want our algorithms to possess several qualities. After correctness,
by far the most important is efficiency. In fact, there are two kinds of algorithm
efficiency: time efficiency, indicating how fast the algorithm runs, and space efficiency,
indicating how much extra memory it uses.

Coding an Algorithm
Most algorithms are destined to be ultimately implemented as computer programs.
Programming an algorithm presents both a peril and an opportunity. The
peril lies in the possibility of making the transition from an algorithm to a program
either incorrectly or very inefficiently.

Important problem types

In the limitless sea of problems one encounters in computing, there are a few
areas that have attracted particular attention from researchers. By and large,
their interest has been driven either by the problem’s practical importance or by
some specific characteristics making the problem an interesting research subject;
fortunately, these two motivating forces reinforce each other in most cases.
In this section, we are going to introduce the most important problem types:
 Sorting
 Searching
 String processing
 Graph problems

ATM online Page 4

  Combinatorial problems
 Geometric problems
 Numerical problems

Sorting
The sorting problem is to rearrange the items of a given list in nondecreasing
order. Of course, for this problem to be meaningful, the nature of the list items
must allow such an ordering.

Searching
The searching problem deals with finding a given value, called a search key, in a
given set (or a multiset, which permits several elements to have the same value).

String Processing
In recent decades, the rapid proliferation of applications dealing with nonnumerical
data has intensified the interest of researchers and computing practitioners in
string-handling algorithms. A string is a sequence of characters from an alphabet.

Graph Problems
One of the oldest and most interesting areas in algorithmics is graph algorithms.
Informally, a graph can be thought of as a collection of points called vertices, some
of which are connected by line segments called edges.

Combinatorial Problems
From a more abstract perspective, the traveling salesman problem and the graphcoloring
problem are examples of combinatorial problems. These are problems
that ask, explicitly or implicitly, to find a combinatorial object—such as a permutation,
a combination, or a subset—that satisfies certain constraints.

Geometric Problems
Geometric algorithms deal with geometric objects such as points, lines, and polygons. Of
course, today people are interested in geometric algorithms
with quite different applications in mind, such as computer graphics, robotics, and
tomography.

Numerical Problems
Numerical problems, another large special area of applications, are problems
that involve mathematical objects of continuous nature: solving equations and
systems of equations, computing definite integrals, evaluating functions, and so on.

ATM online Page 5

Fundamentals data structure
Linear Data Structures
The two most important elementary data structures are the array and the linked
list.A(one-dimensional) array is a sequence of n items of the same data type that are stored
contiguously in computer memory and made accessible by specifying a
value of the array’s index.

 Arrays are used for implementing a variety of other data structures. Prominent
among them is the string, a sequence of characters from an alphabet terminated
by a special character indicating the string’s end. Strings composed of zeros
and ones are called binary strings or bit strings.

………..
Item [0] Item [1] Item [n –1]

Array of n elements.

 A linked list is a sequence of zero or more elements called nodes, each

containing two kinds of information: some data and one or more links called
pointers to other nodes of the linked list.

Graphs
As we mentioned in the previous section, a graph is informally thought of as
a collection of points in the plane called “vertices” or “nodes,” some of them
connected by line segments called “edges” or “arcs.”

Trees
A tree (more accurately, a free tree) is a connected acyclic graph.
A graph that has no cycles but is not necessarily connected is called a forest:

Sets and Dictionaries

The notion of a set plays a central role in mathematics. A set can be described as
an unordered collection (possibly empty) of distinct items called elements of the set.

ATM online Page 6

Worst-Case, Best-Case, and Average-Case Efficiencies

In the beginning of this section, we established that it is reasonable to measure

an algorithm’s efficiency as a function of a parameter indicating the size of the
algorithm’s input. But there are many algorithms for which running time depends
not only on an input size but also on the specifics of a particular input. Consider,
as an example, sequential search. This is a straightforward algorithm that searches
for a given item (some search key K) in a list of n elements by checking successive
elements of the list until either a match with the search key is found or the list
is exhausted. Here is the algorithm’s pseudocode, in which, for simplicity, a list is
implemented as an array. It also assumes that the second condition A[i] _= K will
not be checked if the first one, which checks that the array’s index does not exceed
its upper bound, fails.

ALGORITHM SequentialSearch(A[0..n − 1], K)

//Searches for a given value in a given array by sequential search
//Input: An array A[0..n − 1] and a search key K
//Output: The index of the first element in A that matches K
// or −1 if there are no matching elements
i ←0
while i < n and A[i] _= K do
i ←i + 1
if i < n return i
else return −1

The worst-case efficiency of an algorithm is its efficiency for the worst-case

input of size n, which is an input (or inputs) of size n for which the algorithm
runs the longest among all possible inputs of that size. The way to determine
the worst-case efficiency of an algorithm is, in principle, quite straightforward:
analyze the algorithm to see what kind of inputs yield the largest value of the basic
operation’s count C(n) among all possible inputs of size n and then compute this
worst-case value Cworst(n).

The best-case efficiency of an algorithm is its efficiency for the best-case input
of size n, which is an input (or inputs) of size n for which the algorithm runs the
fastest among all possible inputs of that size. Accordingly, we can analyze the bestcase
efficiency as follows. First, we determine the kind of inputs for which the count
C(n) will be the smallest among all possible inputs of size n. (Note that the best

ATM online Page 7

case does not mean the smallest input; it means the input of size n for which the
algorithm runs the fastest.) Then we ascertain the value of C(n) on these most
convenient inputs. For example, the best-case inputs for sequential search are lists
of size n with their first element equal to a search key; accordingly, Cbest(n) = 1
for this algorithm.

To analyze the algorithm’s averagecase

efficiency, we must make some assumptions about possible inputs of size n.
Let’s consider again sequential search. The standard assumptions are that
(a) the probability of a successful search is equal to p (0 ≤ p ≤ 1) and (b) the
probability of the first match occurring in the ith position of the list is the same
for every i. Under these assumptions—the validity of which is usually difficult to
verify, their reasonableness notwithstanding—we can find the average number
of key comparisons Cavg(n) as follows. In the case of a successful search, the
probability of the first match occurring in the ith position of the list is p/n for
every i, and the number of comparisons made by the algorithm in such a situation
is obviously i. In the case of an unsuccessful search, the number of comparisons
will be n with the probability of such a search being (1− p). Therefore,

Cavg(n) = p(n + 1)/2+ n(1− p).

ATM online Page 8