Design and Analysis of Algorithms Module 1

DESIGN AND ANALYSIS OF ALGORITHMS-15CS43

Course objectives: This course will enable students to
 Explain various computational problem solving techniques.
 Apply appropriate method to solve a given problem.
 Describe various methods of algorithm analysis.

Course Outcomes: After studying this course, students will be able to

 Develop algorithm for solving problems using design techniques.
 Analyze the correctness and the running time of the basic algorithms for classic
problems in computer science domain
 Demonstrate greedy technique for graph related problems.
 Apply Dynamic Programming for classical problems in engineering.
 Discuss on limitations of algorithm power.

Text Books:
T1. Introduction to the Design and Analysis of Algorithms, Anany Levitin:, 2rd Edition, 2009.
Pearson.
T2. Computer Algorithms/C++, Ellis Horowitz, Sartaj Sahni and Rajasekaran, 2nd Edition, 2014,
Universities Press

Module 1

Introduction: What is an Algorithm? (T2:1.1), Algorithm Specification (T2:1.2), Analysis

Framework (T1:2.1), Performance Analysis: Space complexity, Time complexity (T2:1.3).
Asymptotic Notations: Big-Oh notation (O), Omega notation (Ω), Theta notation (Q), and
Little-oh notation (o), Mathematical analysis of Non-Recursive and recursive Algorithms with
Examples (T1:2.2, 2.3, 2.4). Important Problem Types: Sorting, Searching, String processing,
Graph Problems, Combinatorial Problems. Fundamental Data Structures: Stacks, Queues,
Graphs, Trees, Sets and Dictionaries. (T1:1.3, 1.4)

Module 1: INTRODUCTION

1. What is an Algorithm? (T2:1.1)

Algorithms play the central role both in the science and practice of computing. From the theoretical
standpoint, the study of algorithms, sometimes called algorithmics.

Computer programs would not exist without algorithms. And with computer applications becoming
indispensable in almost all aspects of our professional and personal lives, studying algorithms becomes a
necessity for more and more people. Studying algorithms is useful in developing analytical skills.

Notion of Algorithm:

 An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for

obtaining a required output for any legitimate input in a finite amount of time.

This definition can be illustrated by a simple diagram (Figure 1.1).

Dept. of CSE, SJEC Page 1
Design and Analysis of Algorithms Module 1

FIGURE 1.1 the notion of the algorithm.

Algorithm can also be defined as follows:

An algorithm is a finite set of instructions followed to accomplish a given task.
 An algorithm should satisfy the following criteria:
 Input: Each algorithm should have zero or more inputs and it should work for a range of inputs
 Output: the algorithm should produce correct result. It should produce at least one output
 Definiteness: Each instruction should be clear and unambiguous
 Effectiveness: The instructions should be simple and should transform the given input to the
desired o/p.
 Finiteness: The algorithm should terminate after a finite sequence of instructions.

 The properties of an algorithm:

 The nonambiguity requirement for each step of an algorithm cannot be compromised.
 The range of inputs for which an algorithm works has to be specified carefully.
 The same algorithm can be represented in several different ways.
 There may exist several algorithms for solving the same problem.
 Algorithms for the same problem can be based on very different ideas and can solve the problem
with dramatically different speeds.

Fundamentals of Algorithmic Problem Solving: [not there in the syllabus]

We can consider algorithms to be procedural solutions to problems. These solutions are not answers but
specific instructions for getting answers. The sequence of steps followed in designing and analyzing an
algorithm (Figure 1.2).

1. Understanding the Problem:

 From a practical perspective, the first thing we need to do before designing an algorithm is to
understand completely the problem given. Read the problem‘s description carefully.
 An input to an algorithm specifies an instance of the problem the algorithm solves. It is very
important to specify exactly the set of instances the algorithm needs to handle.
 A correct algorithm is the one that works correctly for all legitimate inputs.

2. Decide:
a) Ascertaining the Capabilities of the Computational Device:
 Once understanding the problem completely, we need to ascertain the capabilities of the
computational device the algorithm is intended for.

Dept. of CSE, SJEC Page 2

Design and Analysis of Algorithms Module 1

 Some of the algorithms follow RAM model; its central assumption is that instructions are
executed one after another, one operation at a time. Accordingly, algorithms designed to be
executed on such machines are called sequential algorithms.
 Some computers can execute operations concurrently, i.e., in parallel. Algorithms that take
advantage of this capability are called parallel algorithms.
 There are important problems, however, that are very complex by their nature, or have to process
huge volumes of data, or deal with applications where the time is critical. In such situations, it is
imperative to be aware of the speed and memory available on a particular computer system.

b) Choosing between Exact and Approximate Problem Solving:

 The next principal decision is to choose between solving the problem exactly or solving it
approximately. In the former case, an algorithm is called an exact algorithm; in the latter case, an
algorithm is called an approximation algorithm.
 Why approximation algorithm?
 First, there are important problems that simply cannot be solved exactly for most of their
instances; examples include extracting square roots, solving nonlinear equations etc.
 Second, available algorithms for solving a problem exactly can be unacceptably slow
because of the problem‘s intrinsic complexity. This happens, in particular, for many
problems involving a very large number of choices
 Third, an approximation algorithm can be a part of a more sophisticated algorithm that
solves a problem exactly.
 Deciding on Appropriate Data Structures: Some of the algorithmic design techniques depend
intimately on structuring or restructuring data specifying a problem‘s instance. Hence data
structures remain important for both design and analysis of algorithms.

Dept. of CSE, SJEC Page 3

Design and Analysis of Algorithms Module 1

c) Algorithm Design Techniques:

 An algorithm design technique (or “strategy” or “paradigm”) is a general approach to solving
problems algorithmically that is applicable to a variety of problems from different areas of
computing.
 Learning these techniques is of utmost importance for the following reasons.
 First, they provide guidance for designing algorithms for new problems
 Second, algorithms are the cornerstone of computer science, they can serve as a natural
way to both categorize and study algorithms.

3. Designing an algorithm
 Now design the algorithm and specify it using any of the following notations.
 Methods of Specifying an Algorithm
Once designing an algorithm, it needs to be specified in some fashion. There exist different
notations, used for specifying algorithms.
 Natural language
However, clear description of algorithms using natural language is difficult.
 Pseudocode is a mixture of a natural language and programming language like
constructs. Pseudocode is usually more precise than natural language, and its usage often
yields more succinct algorithm descriptions.
 Flowchart: a method of expressing an algorithm by a collection of connected geometric
shapes containing descriptions of the algorithm‘s steps. This representation technique has
proved to be inconvenient for all but very simple algorithms

4. Proving an Algorithm’s Correctness:

 Once an algorithm has been specified, we have to prove its correctness. That is, we have to prove
that the algorithm yields a required result for every legitimate input in a finite amount of time. For
some algorithms, a proof of correctness is quite easy; for others, it can be quite complex.
 A common technique for proving correctness is to use mathematical induction because an
algorithm‘s iterations provide a natural sequence of steps needed for such proofs. In order to
show that an algorithm is incorrect, we need just one instance of its input for which the algorithm
fails.

5. Analyzing an Algorithm:
 Analyzing an algorithm deals with the efficiency measurement.
 There are two kinds of algorithm efficiency:
 Time efficiency, indicates how fast the algorithm runs
 Space efficiency, indicates how much extra memory it uses.
 Another desirable characteristic of an algorithm is simplicity. Simplicity is about designing
algorithms that are simpler to understand and code.
 Yet another desirable characteristic of an algorithm is generality. Generality is about designing
most general algorithms.
6. Coding an Algorithm:
 The final step is to implement algorithms as computer programs. As a practical matter, the
validity of programs is established by testing.

Dept. of CSE, SJEC Page 4

Design and Analysis of Algorithms Module 1

2. Algorithm Specification: (T2:1.2)

An algorithm can be described in many ways.

1. Using Natural Language: An algorithm can be written in natural language if the resulting
instructions are definite.
2. Using Graphical representations: Graphical representation like flowcharts can be used
only when the algorithm is simple and small.
3. A combination of a programming language and English Language can be used.

Suppose we must device a program that sorts a collection of n>=1 element of type Type. A
simple solution could be as follows
From those elements that are currently unsorted, find the smallest and place it next in the
sorted list.
Although the above statements describe the sorting problem, it is not an algorithm. The above
statements does not tell us where and how the elements are initially stored or where we should
place the result. We assume that the elements are stored in an array, a, such that the i th position,
a[i], 1<=i<=n.

for(i=1;i<=n;i++){
examine a[i] to a[n] and suppose the smallest element is at a[j];
interchange a[i] and a[j];
}

Selection sort Algorithm

void selectionsort(type a[],int n)

//sort the array a[1:n]into nondecreasing order
{
for(int i=1;i<=n;i++){
int j=i;
for(int k=i+1;k<=n;k++)
if(a[k]<a[j]) j=k;
type t=a[j]; a[i]=a[j]; a[j]=t;
}
}
Code for selection sort

3. Analysis Framework (T1:2.1)

General framework for analyzing the efficiency of algorithms:

1. Measuring an Input’s Size

The obvious observation is that, almost all algorithms run longer on larger inputs. For
example, it takes longer to sort larger arrays, multiply larger matrices, and so on. Therefore,
it is logical to investigate an algorithm’s efficiency as a function of some parameter n
indicating the algorithm’s input size.

Dept. of CSE, SJEC Page 5

Design and Analysis of Algorithms Module 1

Eg: for sorting, searching, and finding list‘s smallest/largest element and for problems
dealing with lists the input size would be size of the list. For evaluating polynomial of degree
n, the input size could be the degree or number of its coefficients.

Some problems require more than one parameter to indicate the size of their inputs. For some
problems it depends on the operations performed. For example a spell checking algorithm, if
it examines individual characters of its input, then the input size is measured by the number
of characters, if it works by processing words, then the input size is number of words.
For algorithms involving properties of numbers the size can be measured by the number b of
bits in the n‘s binary representation as b=log2n +1

2. Units for Measuring Running Time

We can simply use some standard unit of time measurement such as second, millisecond etc.
to measure the running time of a program implementing the algorithm. However, there are
obvious drawbacks to such an approach. Because, this unit is
 dependent on the speed of a particular computer
 dependent on the quality of a program implementing the algorithm
 dependent on compiler used in generating the machine code
 And difficulty of clocking the actual running time of the program.
We have to choose a metric that doesn‘t depend on these extraneous factors.
The following are the standard methods of computing the time efficiency of algorithms:
 Operation Counts
 Asymptotic notations(Mathematical analysis)

Operation Counts:
 This approach deals with counting the number of times each of the algorithm‘s operations
is executed.
 To do this,
1. Identify the most important operation of the algorithm, the basic operation, the
operation contributing the most to the total running time. It is usually the most time-
consuming operation in the algorithm‘s innermost loop.
Eg: (1) For sorting algorithms, basic operation is key comparison.
(2) Algorithms for mathematical problems: division is the most time consuming
operation, followed by multiplication and then addition and subtraction.
2. Count the number of times the algorithm‘s basic operation is executed on inputs of
size n.

 Let cop be the execution time of an algorithm‘s basic operation on a particular computer
Let C(n) be the number of times this operation needs to be executed for this algorithm.
Then, the running time T(n) of a program implementing this algorithm on that computer
is
T (n) ≈ cop * C(n).
The above formula helps us to answer questions such as how much faster would an algorithm run
on a machine that is ten times faster than the one we have.
As an example consider C(n)=1/2*n*(n-1), how much longer will the algorithm run if we double
its input size? It runs four times longer, for all but very small values of n,

Dept. of CSE, SJEC Page 6

Design and Analysis of Algorithms Module 1

 The efficiency analysis framework ignores multiplicative constants and concentrates on

the count‘s order of growth to within a constant multiple for large-size inputs.

3. Orders of Growth:
 The behavior of some algorithms change with increase in the large value of n.
 The change in behavior of an algorithm as the value of n increases is called Order of
Growth.
 This change in the behavior of the algorithm and the efficiency of the algorithm can be
analyzed by considering the highest order of n.
 Basic Efficiency Classes
The time efficiencies of a large number of algorithms fall into only a few classes. These
classes are listed below in increasing order of their orders of growth, along with their
names

Class Name
1 constant
log n logarithmic
n linear
n log n n-log-n
n2 quadratic
n3 cubic
2n exponential
n! factorial

 The following Table shows values of above mentioned functions

 The function growing the slowest among these is the logarithmic function.
 On the other end, exponential function 2n and the factorial function n! grow so fast
that their values become astronomically large even for rather small values of n. Both
are often referred to as ―exponential-growth functions‖ (or simply ―exponential‖)

Dept. of CSE, SJEC Page 7

Design and Analysis of Algorithms Module 1

Also, algorithms that require an exponential number of operations are practical for
solving only problems of very small sizes.
 Another way to appreciate the qualitative difference among the orders of growth of
the functions is to consider how they react to, say, a twofold increase in the value of
their argument n.
 The function log2n increases in value by just 1 (because, log22n = log22 + log2n =
1+ log2n)
 The linear function n increases twofold
 The nlog2n increases slightly more than twofold
 The quadratic function n2 increase fourfold (because (2n)2 = 4n2) )
 The cubic function n3 increase eightfold
 The function 2n gets squared (because 22n = (2n)2 )
 n! increases much more than that

4. Worst-Case, Best-Case, and Average-Case Efficiencies:

There are many algorithms for which running time depends not only on an input size but also on
the specifics of a particular input.
 Consider Sequential search algorithm:
ALGORITHM SequentialSearch(A[0..n − 1], K)
//Searches for a given value in a given array by sequential search
//Input: An array A[0..n − 1] and a search key K
//Output: The index of the first element in A that matches K
// or −1 if there are no matching elements
i ←0
while i < n and A[i] != K do
i ←i + 1
if i < n return i
else return −1

The running time of this algorithm can be different for the same list size n. In the worst case,
when there are no matching elements or the first matching element happens to be the last one on
the list, the algorithm makes the largest number of key comparisons among all possible inputs of
size n.

 Worst-case-efficiency:
 The worst-case efficiency of an algorithm is its efficiency for the worst-case input of size
n, which is an input (or inputs) of size n for which the algorithm runs the longest among
all possible inputs of that size.
 The way to determine the worst-case efficiency of an algorithm:
Analyze the algorithm to see what kind of inputs yield the largest value of the basic
operation‘s count C(n) among all possible inputs of size n and then compute this worst-
case value Cworst(n).
Eg: For sequential search, the worst case is when there are no matching elements or the
first matching element happens to be the last one on the list. Here, the algorithm makes
the largest number of key comparisons among all possible inputs of size n: Therefore,
Cworst(n)= n.
 The worst-case analysis provides very important information about an algorithm‘s
efficiency by bounding its running time from above. In other words, it guarantees that for

Dept. of CSE, SJEC Page 8

Design and Analysis of Algorithms Module 1

any instance of size n, the running time will not exceed Cworst(n), its running time on the
worst-case inputs.
 Best-case-efficiency:
 The best-case efficiency of an algorithm is its efficiency for the best-case input of size n,
which is an input (or inputs) of size n for which the algorithm runs the fastest among all
possible inputs of that size.
 We can analyze the best-case-efficiency as follows:
o Determine the kind of inputs for which the count C(n) will be the smallest among
all possible inputs of size n.
o Then ascertain the value of C(n) on these most convenient inputs.
For example, the best-case inputs for sequential search are lists of size n with their first
element equal to a search key; accordingly, Cbest(n) = 1 for this algorithm.
 The analysis of the best-case efficiency is not nearly as important as that of the worst-
case efficiency. But it is not completely useless, either.
 If the best-case-efficiency of an algorithm is unsatisfactory, we can immediately discard
it without further analysis.

Average-case-efficiency:
 The Average-case efficiency of an algorithm is its efficiency for the ―random‖ input.
 To analyze the algorithm‘s average-case-efficiency, we must make some assumptions
about possible inputs of size n.
Ex: consider again sequential search. The standard assumptions are that
a) the probability of a successful search is equal to p (0 ≤ p ≤ 1)
b) the probability of the first match occurring in the ith position of the list is the same for
every i.
Under these assumptions we can find the average number of key comparisons Cavg(n)
as follows.
i. In the case of a successful search, the probability of the first match occurring in the ith
position of the list is p/n for every i, and the number of comparisons made by the
algorithm in such a situation is i.
ii. In the case of an unsuccessful search, the number of comparisons will be n with the
probability of such a search being (1− p). Therefore,

if p = 1(the search must be successful), the average number of key comparisons made by
sequential search is (n + 1)/2; that is, the algorithm will inspect, on average, about half of
the list‘s elements.
If p = 0 (the search must be unsuccessful), the average number of key comparisons will
be n because the algorithm will inspect all n elements on all such inputs.
 Investigation of the average-case efficiency is considerably more difficult than
investigation of the worst-case and best-case efficiencies.
 Average-case efficiency cannot be obtained by taking the average of the worst-case and
the best-case efficiencies.

Dept. of CSE, SJEC Page 9

Design and Analysis of Algorithms Module 1

Amortized efficiency: It‘s another type of efficiency. It applies not to a single run of an
algorithm but rather to a sequence of operations performed on the same data structure. In some
situations a single operation can be expensive, but the total time for an entire sequence of n such
operations is always significantly better than the worst-case efficiency of that single operation
multiplied by n.

Points to remember:
 Both time and space efficiencies are measured as functions of the algorithm‘s input size.
 Time efficiency is measured by counting the number of times the algorithm‘s basic
operation is executed.
 Space efficiency is measured by counting the number of extra memory units consumed
by the algorithm.
 The efficiencies of some algorithms may differ significantly for inputs of the same size.
For such algorithms, we need to distinguish between the worst-case, average-case, and
best-case efficiencies.
 The framework‘s primary interest lies in the order of growth of the algorithm‘s running
time as its input size goes to infinity.

4. Performance Analysis: Space complexity, Time complexity (T2:1.3)

There are two kinds of efficiency:
1. Space efficiency, also called space complexity, refers to the amount of memory units
required by the algorithm in addition to the space needed for its input and output.
2. Time efficiency, also called time complexity, indicates how fast an algorithm in question
runs.
Space complexity:
 The space complexity of an algorithm depends on the following components:
 Instruction/program space: space required to store machine code
 Data space: space needed for constants, variables etc.
 Stack space: to store return address, return values and etc.

 Let CP= space required for the static part of the program
SP= space required for the dynamic part of the program
Then, the total space required for a program P, is S(P)=CP+SP
Examples:
Iterative type algorithms:
1) To add three integers :
Add(a,b,c)
return a+b+c

Space required for a,b,c,= 3 words (or 6 bytes)

Therefore, the total space= 3+0= 3 words

2) To add integers using non-recursive function

Add(x,n) // x is an array of integer elements
// n is the size of an array
sum0
for i1 to n do

Dept. of CSE, SJEC Page 10

Design and Analysis of Algorithms Module 1

sumsum+x[i]
return sum

Space for array x : n words

Space for sum,i,n : 3 words
Hence, the total space : >=(n+3) words

Recursive algorithms:
3) Add(x,n) // x is an array of integer elements
// n is the size of an array
Return Add(x,n-1)+x[n]

Here stack is used for recursion. The internal stack for recursion includes space for
formal parameters, local variables and return address.
Here, each call to function Add requires atleast 3 words:
 Space for n values
 Space for return address
 Space for pointer to x[]
Depth of recursion: n+1 ( n times call + 1 time return call )
Therefore, the total space: >=3(n+1)

Time Complexity:
The time T(p) taken by a program P is the sum of the compile time and runtime. The compile
time does not depend on the instance characteristics. Also a compiled program can run several
times without recompilation. Hence we consider only the runtime of a program denoted by
tP(instance characteristics).
A Program step is loosely defined as a syntactically or semantically meaningful segment of a
program that has an execution time that is independent of the instance characteristics.
 The number of steps any program statement is assigned depends on the kind of statement.
For example, comments and declarative statements such as int,struct,#include,class etc
count as zero steps.
 Assignment statements which does not involve any calls to other programs is counted as
one step
 For the iterative statements like for,while etc we consider step count only for control part
of the statement.
 Control part for while and for statements have the following forms:
for(i=<expr>;i<=<expr1>;i++)
while (<expr>)

We can determine the number of steps needed by a program to solve a particular problem
instance in one of the 2 ways
1. Introduce a variable called count into the program. This is a global variable initialized to
zero. Statements to increment count by appropriate amount are introduced into the
program. Each time a statement in the program is executed count is incremented.
2. Build a table in which we list the total number of steps contributed by each statement.

Method 1: Using count variable:

float Sum(float a[], int n)

Dept. of CSE, SJEC Page 11

Design and Analysis of Algorithms Module 1

{
float s=0.0;
count++;
for(int i=1;i<=n;i++)
{
count++;
s+=a[i];count++;
}
count++;
count++;
return s;
}

Program 1: to find sum of n numbers.

Since we are interested in knowing only the change in count variable, the above program
can be simplified as follows

float Sum(flaot a[],int n)

{
for(int i=1;i<=n;i++) count+=2;
count+=3;
}
In the for loop ,the value of count will increase by a total of 2n. If count is zero to start
with, then it will be 2n+3 on termination. So each invocation of Sum takes total of 2n+3
steps.

Ex2: floar RSum(float a[],int n)

{
count++;
if (n<=0){
count++;
return 0.0;
}
else{
count++;
return (RSum(a,n-1)+a[n]);
}
}
We obtain a recursive formula for step count

Dept. of CSE, SJEC Page 12

Design and Analysis of Algorithms Module 1

This recursive formulas are referred to as recurrence relations.Inorder to solve a recurrence

relation we make repeated substitutions for tRSum on the right hand side until all such occurrences

disappear.
So the step count of RSum is 2n+2.

*Find the step count of Matrix Addition.----Refer Notes

Method 2: Refer Notes

5. Asymptotic Notations: Big-Oh notation (O), Omega notation (Ω), Theta

notation (Q), and Little-oh notation (o), Mathematical analysis of Non-
Recursive and recursive Algorithms with Examples (T1:2.2, 2.3, 2.4).
The order of growth is expressed using asymptotic notation. They are the notations using which
two algorithms can be compared w.r.t. efficiency based on the order of growth of an algorithm‘s
basic operation.

Asymptotic Notations:
 The efficiency analysis concentrates on the order of growth of an algorithm‘s basic operation
count. To compare and rank such orders of growth, the following three asymptotic notations
are used
O(big oh)
Ω(big omega)
Θ(big theta).
 Let t (n) and g(n) can be any nonnegative functions defined on the set of natural numbers.
t(n) will be an algorithm‘s running time and g(n) will be some simple function to compare
the count with.

O- Notation:
 O(g(n)) is the set of all functions with a lower or same order of growth as g(n) ( to within a
constant multiple, as n goes to infinity).

 Definition: A function t(n) is said to be in O(g(n)), denoted t (n)  O(g(n)), if t (n) is bounded
above by some constant multiple of g(n) for all large n, i.e., if there exist some positive
constant c and some nonnegative integer n0 such that
t (n) ≤ c.g(n) for all n ≥ n0.
Where n is the size of input
g(n) is a function
( t(n) is the time efficiency of an algorithm)
 The definition is illustrated in the following Figure 2.1 where, n is extended to be a real
number.
Here, a graph of time, t versus input, n is plotted for the functions t(n) and c.g(n)
The function t(n) lies below c.g(n)

Dept. of CSE, SJEC Page 13

Design and Analysis of Algorithms Module 1

 Big-O is a method of expressing the upper bound of an algorithm‘s running time. And it is a
measure of the longest amount of time it could take for the algorithm to complete.
 Examples: the following assertions are all true:
n  O(n2), 100n + 5  O(n2), 1 n(n-1)  O(n2).
2
 Let us formally prove one of the assertions made: 100n + 5  O(n2).
(i) 100n + 5 ≤ 100n + n (for all n ≥ 5) = 101n ≤ 101n2.
Thus, we can take c=101 and n0=5.
(ii) We could also reason that,
100n + 5 ≤ 100n + 5n (for all n ≥ 1) = 105n ≤ 105n2
with c = 105 and n0 = 1.
 On the other hand,
n3  O(n2), 0.00001n3  O(n2), n4+n+1  O(n2)
3 3
Indeed, the functions n and 0.00001n are both cubic and hence have a higher order of
growth than n2, and so has the fourth-degree polynomial n4 + n + 1.

Ω – Notation:
 Ω(g(n)), stands for the set of all functions with a higher or same order of growth as g(n).
 Definition: A function t (n) is said to be in Ω(g(n)), denoted t(n)  Ω(g(n)), if t(n) is bounded
below by some positive constant multiple of g(n) for all large n, i.e., if there exist some
positive constant c and some nonnegative integer n0 such that
t (n) ≥ cg(n) for all n ≥ n0.
 So, if we draw a graph t(n) and c.g(n) versus n, the graph of t(n) lies above the graph of
c.g(n) for sufficiently large value of n as shown below:

 Examples:
n3  Ω(n2), 1 n(n-1)  Ω(n2), but 100n+5  Ω(n2)
2
Dept. of CSE, SJEC Page 14
Design and Analysis of Algorithms Module 1

Let us prove that n3  Ω(n2):

n3 ≥ n2 for all n ≥ 0,
i.e., we can select c = 1 and n0 = 0.

Θ-Notation:
 Θ(g(n)) is the set of all functions that have the same order of growth as g(n).
 Definition: A function t(n) is said to be in Θ(g(n)), denoted t(n)  Θ(g(n)), if t(n) is bounded
both above and below by some positive constant multiples of g(n) for all large n, i.e., if there
exist some positive constants c1 and c2 and some nonnegative integer n0 such that
c2g(n) ≤ t (n) ≤ c1g(n) for all n ≥ n0.

 So, if we draw a graph t(n), c1.g(n) and c2.g(n) versus n, the graph of t(n) lies above the graph
of c2.g(n) and lies below the graph c1.g(n) for sufficiently large value of n as shown below:

 Examples:
(1) Every quadratic function an2+bn+c with a>0 is in Θ(n2).
(2) Let us prove that 1 n(n-1)  Θ(n2).
2
First, we prove the right inequality (upper bound):

Useful Property Involving the Asymptotic Notations:

The following property is useful in analyzing algorithms that comprise two consecutively
executed parts.

THEOREM If t1(n)  O(g1(n)) and t2(n)  O(g2(n)), then

t1(n) + t2(n)  O(max{g1(n), g2(n)}).
(The analogous assertions are true for the Ω and Θ notations as well.)

Dept. of CSE, SJEC Page 15

Design and Analysis of Algorithms Module 1

PROOF The proof extends to orders of growth the following simple fact about four arbitrary
real numbers a1, b1, a2, b2: if a1 ≤ b1 and a2 ≤ b2, then a1 + a2 ≤ 2 max{b1, b2}.

Since t1(n)  O(g1(n)), there exist some positive constant c1 and some nonnegative integer n1
such that
t1(n) ≤ c1g1(n) for all n ≥ n1.
Similarly, since t2(n)  O(g2(n)),
t2(n) ≤ c2g2(n) for all n ≥ n2.
Let us denote c3 = max{c1, c2} and consider n ≥ max{n1, n2} so that we can use both inequalities.
Adding them yields the following:
t1(n) + t2(n) ≤ c1g1(n) + c2g2(n)
≤ c3g1(n) + c3g2(n) = c3[g1(n) + g2(n)]
≤ c32 max{g1(n), g2(n)}.

Hence, t1(n) + t2(n)  O(max{g1(n), g2(n)}), with the constants c and n0 required by the O
definition being 2c3 = 2 max{c1, c2} and max{n1, n2}, respectively.

It implies that the algorithm‘s overall efficiency is determined by the part with a higher order of
growth, i.e., its least efficient part:
| t1(n)  O(g1(n)) | t1(n) + t2(n)  O(max{g1(n), g2(n)})
| t2(n)  O(g2(n)) |

For example, If a sorting algorithm used in the first part makes no more than ½ n(n − 1)
comparisons (and hence is in O(n2)) the second part makes no more than n − 1
comparisons (and hence is in O(n)), then the efficiency of the entire algorithm
will be in O(max{n2, n}) = O(n2).

Using Limits for Comparing Orders of Growth:

Though the formal definitions of O, Ω and Θ are indispensable for proving their abstract
properties, they are rarely used for comparing the orders of growth of two specific functions. A
much more convenient method for doing so is based on computing the limit of the ratio of two
functions in question.
Three principal cases may arise:
0 implies that t (n) has a smaller order of growth than g(n)
lim t(n) = c>0 implies that t(n) has the same order of growth as g(n),
n→∞ g(n) ∞ implies that t(n) has a larger order of growth than g(n).

Note :
 first two cases mean that t(n)  O(g(n)),
 the last two mean that t(n)  Ω(g(n)),
 the second case means that t(n)  Θ(g(n)).

The limit-based approach is often more convenient than the one based on the definitions because
it can take advantage of the powerful calculus techniques developed for computing limits, such
as L‘H ˆ opital‘s rule

Dept. of CSE, SJEC Page 16

Design and Analysis of Algorithms Module 1

and Stirling‘s formula

EXAMPLE 1 : Compare the orders of growth of 1 n(n-1) and n2

2

Since the limit is equal to a positive constant, the functions have the same order of growth or,
symbolically, ½ n(n − 1)  Θ(n2).

EXAMPLE 2: Compare the orders of growth of log2 n and n

Since the limit is equal to zero, log2 n has a smaller order of growth than n.

EXAMPLE 3: Compare the orders of growth of n! and 2n.

Taking advantage of Stirling‘s formula, we get

Thus, though 2n grows very fast, n! grows still faster. We can write symbolically that n!  Ω (2n).

Rules of sum manipulations:

Summation formulas:

Mathematical Analysis of Nonrecursive Algorithms:

General Plan for Analyzing the Time Efficiency of Nonrecursive Algorithms:

1. Decide on a parameter (or parameters) indicating an input‘s size.
2. Identify the algorithm‘s basic operation.
3. Check whether the number of times the basic operation is executed depends only on the size of
an input.

Dept. of CSE, SJEC Page 17

Design and Analysis of Algorithms Module 1

If it also depends on some additional property, the worst-case, average-case, and, if

necessary, best-case efficiencies have to be investigated separately.
4. Set up a sum expressing the number of times the algorithm‘s basic operation is executed.
5. Using standard formulas and rules of sum manipulation,
either find a closed form formula for the count
or, at the very least, establish its order of growth.

EXAMPLE 1: Consider the problem of finding the value of the largest element in a list of n
numbers in an array.
 Pseudocode for solving this problem:
ALGORITHM MaxElement(A[0..n − 1])
//Determines the value of the largest element in a given array
//Input: An array A[0..n − 1] of real numbers
//Output: The value of the largest element in A
maxval ←A[0]
for i ←1 to n − 1 do
if A[i]>maxval
maxval←A[i]
return maxval
 Analysis:
o Parameter to be considered is n (size of input).
o The operations that are going to be executed most often are in the algorithm‘s for loop.
There are two operations in the loop‘s body:
 the comparison A[i]> maxval
 the assignment maxval←A[i].
Since the comparison is executed on each repetition of the loop, we should consider it to
be the algorithm‘s basic operation.
o There is no need to distinguish among the worst, average, and best cases here since the
number of comparisons will be the same for all arrays of size n.
o Let C(n) denote the no. of times comparisons made.
The algorithm makes one comparison on each execution of the loop
And this is repeated once for each value of the loop‘s variable i
The variable i is within the bounds 1 and n − 1.
Therefore, we get
n-1
C(n) = 1
i=1
u
i.e. C(n) = n-1 [ based on the summation formula, 1 = u-m+1, m<=u ]
im
 C(n)  Θ(n)

EXAMPLE 2: element uniqueness problem: check whether all the elements in a given array of
n elements are distinct.

 Pseudocode for solving this problem:

ALGORITHM UniqueElements(A[0..n − 1])
//Determines whether all the elements in a given array are distinct
//Input: An array A[0..n − 1]

Dept. of CSE, SJEC Page 18

Design and Analysis of Algorithms Module 1

//Output: Returns ―true‖ if all the elements in A are distinct and ―false‖ otherwise
for i ←0 to n − 2 do
for j ←i + 1 to n − 1 do
if A[i]= A[j ] return false
return true
 Analysis:
o Parameter to be considered is n (size of input).
o Since the innermost loop contains a single operation (the comparison of two elements),
we should consider it as the algorithm‘s basic operation.
o The number of element comparisons depends not only on n but also on whether there are
equal elements in the array and, if there are, which array positions they occupy.
We will limit our investigation to the worst case only. An inspection of the innermost
loop reveals that there are two kinds of worst-case inputs:
 arrays with no equal elements
 Arrays in which the last two elements are the only pair of equal elements.
o Here, one comparison is made for each repetition of the innermost loop, i.e., for each
value of the loop variable j between its limits i + 1 and n − 1;
This is repeated for each value of the outer loop, i.e., for each value of the loop variable i
between its limits 0 and n − 2.

Accordingly, we get
n2 n 1
Cworst(n) =  1
i 0 j i 1
n2

 [( n  1)  (i  1)  1]
u
=
i 0
[based on summation formula 1 = l-u+1, l<=u]
i l
n2

=  (n  1  i)
i 0
n2 n2

=  ( n  1) - i
i 0 i 0
n2

 (n  2)(n  1) n( n  1)
n
= (n-1) 1 -
i 0 2
[  i=i 1 2
]

(n  2)(n  1)
= (n-1)2 -
2
( n  1) n 1
=  n2  Θ(n2)
2 2
EXAMPLE 3: Given two n × n matrices A and B, find the time efficiency of the definition-
based algorithm for computing their product C = AB.
 Pseudocode:
ALGORITHM MatrixMultiplication(A[0..n − 1, 0..n − 1], B[0..n − 1, 0..n − 1])
//Multiplies two square matrices of order n by the definition-based algorithm
//Input: Two n × n matrices A and B
//Output: Matrix C = AB
for i ←0 to n − 1 do
Dept. of CSE, SJEC Page 19
Design and Analysis of Algorithms Module 1

for j ←0 to n − 1 do
C[i, j ]←0.0
for k←0 to n − 1 do
C[i, j ]←C[i, j ]+ A[i, k] * B[k, j]
return C

 Analysis:
o Parameter to be considered is n (size of input).
o There are two arithmetical operations in the innermost loop here: multiplication and
addition.
We consider multiplication as the basic operation.
o Let M(n) be the sum for the total number of multiplications executed by the algorithm.
o There is just one multiplication executed on each repetition of the algorithm‘s innermost
loop, which is governed by the variable k ranging from the lower bound 0 to the upper
bound n − 1.
o Therefore,
 the number of multiplications made for every pair of specific values of variables i
n 1
and j is 1
k 0
 the total number of multiplications M(n) is expressed by the following triple sum:
n 1 n 1 n 1
M(n) =
i 0
 1
j 0 k 0
n 1 n 1
=  n
i 0 j 0
n 1
= n
i 0
2

= n3
o now, the running time of the algorithm on a particular machine can be estimated as
T (n) ≈ cmM(n) = cmn3,
where cm is the time of one multiplication on the machine in question.
We would get a more accurate estimate if we took into account the time spent on the
additions, too:
T (n) ≈ cmM(n) + caA(n) = cmn3 + can3 = (cm+ ca)n3,
where ca is the time of one addition.

EXAMPLE 4: Find the number of binary digits in the binary representation of a positive
decimal integer.
 Pseudocode:
ALGORITHM Binary(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n‘s binary representation
count ←1
while n > 1 do
count ←count + 1
n← n / 2
return count

Dept. of CSE, SJEC Page 20

Design and Analysis of Algorithms Module 1

o the most frequently executed operation here is not inside the while loop but rather the
comparison n > 1 that determines whether the loop‘s body will be executed.
o loop variable takes on only a few values between its lower and upper limits; therefore, we
have to use an alternative way of computing the number of times the loop is executed.
Since the value of n is about halved on each repetition of the loop, the answer should be
about log2n.
o The exact formula for the number of times the comparison n>1 will be executed is
actually + log2 n + 1

Mathematical Analysis of Recursive Algorithms:

General Plan for Analyzing the Time Efficiency of Recursive Algorithms

1. Decide on a parameter (or parameters) indicating an input‘s size.
2. Identify the algorithm‘s basic operation
3. Check whether the number of times the basic operation is executed can vary on different
inputs of the same size; if it can, the worst-case, average-case, and best-case efficiencies must
be investigated separately.
4. Set up a recurrence relation, with an appropriate initial condition, for the number of times the
basic operation is executed.
5. Solve the recurrence or, at least, ascertain the order of growth of its solution.

EXAMPLE 1: Compute the factorial function F(n) = n! for an arbitrary nonnegative integer n.

 Recursive algorithm for computing n!:

ALGORITHM F(n)
//Computes n! recursively
//Input: A nonnegative integer n
//Output: The value of n!
if n = 0
return 1
else
return F(n − 1) * n

 Analysis:
o Parameter to be considered is n (size of input).
o The basic operation of the algorithm is multiplication
o The total number of executions made by the basic operation is denoted by M(n).

The function F(n) is computed according to the formula

F(n) = F(n − 1) . n for n > 0,
The number of multiplications M(n) needed to compute it, must satisfy the equality
M(n) = M(n − 1) + 1 for n > 0.
to compute to multiply
F(n−1) F(n-1) by n

Such equations are called recurrence relations or recurrences.

Initial condition:
By inspection, we find that, if n = 0 return 1.
Dept. of CSE, SJEC Page 21
Design and Analysis of Algorithms Module 1

Therefore, the initial condition is

M(0) = 0.

Thus, we succeeded in setting up the recurrence relation and initial condition for the
algorithm‘s number of multiplications M(n):
M(n) = M(n − 1) + 1 for n > 0
M(0) = 0.

o Now we need to solve the above recurrence :

Here, we use method of backward substitutions to solve recurrence relations:
M(n) = M(n − 1) + 1 substitute M(n − 1) = M(n − 2) + 1
= [M(n − 2) + 1]+ 1
= M(n − 2) + 2 substitute M(n − 2) = M(n − 3) + 1
= [M(n − 3) + 1]+ 2
= M(n − 3) + 3.
…………..
After inspecting the first three lines, we see a general formula for the pattern:
M(n) = M(n − i) + i.
To get the result, we need to substitute i = n in the pattern‘s formula, since the initial
condition specified for n = 0:
i.e.,M(n) = M(n − 1) + 1
...
= M(n − i) + i
...
= M(n − n) + n
= M(0) + n
= 0+n
=n
Thus, complexity of factoral function is given by t(n)  Θ(n)

EXAMPLE 2: Tower of Hanoi puzzle. In this puzzle, we have n disks of different sizes that can
slide onto any of three pegs. Initially, all the disks are on the first peg in order of size, the largest
on the bottom and the smallest on top. The goal is to move all the disks to the third peg, using the
second one as an auxiliary, if necessary. We can move only one disk at a time, and it is forbidden
to place a larger disk on top of a smaller one.

The problem has an elegant recursive solution, which is illustrated in Figure 2.4.

Dept. of CSE, SJEC Page 22

Design and Analysis of Algorithms Module 1

o To move n>1 disks from peg 1 to peg 3 (with peg 2 as auxiliary),

 we first move recursively n − 1 disks from peg 1 to peg 2 (with peg 3 as auxiliary)
 then move the largest disk directly from peg 1 to peg 3
 Finally, we move recursively n − 1 disks from peg 2 to peg 3 (using peg 1 as
auxiliary).
o if n = 1, we simply move the single disk directly from the source peg to the destination
peg.(initial condition)

 Analysis:
o Parameter to be considered is n (no.of disks).
o The basic operation is movement of disk.
o The number of moves M(n) depends on n only, and
we get the following recurrence equation:
M(n) = M(n − 1) + 1+ M(n − 1) for n > 1.
M(1) = 1 initial condition :

Therefore, recurrence relation for the number of moves M(n):

M(n) = 2M(n − 1) + 1 for n > 1 (2.3)
M(1) = 1.

o We solve this recurrence by the method of backward substitutions:

M(n) = 2M(n − 1) + 1 sub. M(n − 1) = 2M(n − 2) + 1
= 2[2M(n − 2) + 1]+ 1
= 22M(n − 2) + 2 + 1 sub. M(n − 2) = 2M(n − 3) + 1
= 22[2M(n − 3) + 1]+ 2 + 1
= 23M(n − 3) + 22 + 2 + 1
……….

Therefore, after i substitutions, we get

a (r n  1)
M(n) = 2iM(n − i) + 2i−1 + 2i−2 + . . . + 2 + 1 ar0 + ar1+ . . . + arn-1=
r 1

Dept. of CSE, SJEC Page 23

Design and Analysis of Algorithms Module 1

1(2 i  1)
= 2iM(n − i) + 2i – 1 here, a=1, r=2, n=i , i.e.=
2 1

Since the initial condition is specified for n = 1, which is achieved for i = n − 1, we get
the following solution:
M(n) = 2n−1M(n − (n − 1)) + 2n−1 − 1
= 2n−1M(1) + 2n−1 − 1
= 2n−1+ 2n−1 – 1
= 2n − 1.

Thus, we have an exponential algorithm and the complexity of Towers of Hanoi function
is given by t(n)  Θ(2n-1)  Θ(2n)

Note:
o When a recursive algorithm makes more than a single call to itself, it can be useful for
analysis purposes to construct a tree of its recursive calls.
In this tree, nodes correspond to recursive calls, and we can label them with the value of
the parameter (or, more generally, parameters) of the calls.
o For the Tower of Hanoi example, the tree is given below.

o By counting the number of nodes in the tree, we can get the total number of calls made
by the Tower of Hanoi algorithm:
n 1
C(n) =  2 l (where l is the level in the tree in Figure 2.5) = 2n − 1.
l 0

EXAMPLE 3: To find the number of binary digits in a decimal integer recursively

 Pseudocode :
ALGORITHM BinRec(n)
//Input: A positive decimal integer n
//Output: The number of binary digits in n‘s binary representation
if n = 1 return 1
else return BinRec( n / 2 ) + 1

 Analysis:
o Parameter to be considered is n (size of input).
o The basic operation is
BinRec( n / 2 ) + 1

Dept. of CSE, SJEC Page 24

Design and Analysis of Algorithms Module 1

o The number of additions made by the algorithm be A(n).

The number of additions made in computing BinRec( n / 2 ) is A( n / 2 ), plus one more
addition is made by the algorithm to increase the returned value by 1.
This leads to the recurrence:
A(n) = A( n / 2 ) + 1 for n > 1.
Since the recursive calls end when n is equal to 1 and there are no additions made then,
the initial condition is
A(1) = 0.
o Here, we use the standard approach to solve such a recurrence by solving it only for
n = 2k and then using smoothness rule .
Therefore, for n = 2k recurrence takes the form
A(2k) = A(2k−1) + 1 for k > 0
A(20) = 0.
Now we use backward substitutions:
A(2k) = A(2k−1) + 1 substitute A(2k−1) = A(2k−2) + 1
k−2
= [A(2 ) + 1]+ 1
= A(2k−2) + 2 substitute A(2k−2) = A(2k−3) + 1
k−3
= [A(2 ) + 1]+ 2
= A(2k−3) + 3
...
...
= A(2k−i) + i
...
= A(2k−k) + k.
= A(1) + k
=k
k
However, n = 2 and hence k = log2n,
A(n) = log2n   (log n).

6. Important Problem Types: Sorting, Searching, String processing, Graph

Problems, Combinatorial Problems. (T1:1.3)
Important Problem Types
The most important problem types:
 Sorting
 Searching
 String processing
 Graph problems
 Combinatorial problems
 Geometric problems
 Numerical problems

Sorting
 The sorting problem is to rearrange the items of a given list in ascending order.
 Further, sorting makes many questions about the list easier to answer. The most
important of them is searching. Also it is used as an auxiliary step in several important
algorithms in other areas, e.g., geometric algorithms; greedy approach requires a sorted
input.

Dept. of CSE, SJEC Page 25

Design and Analysis of Algorithms Module 1

 Two properties of sorting algorithms:

1. Stability: A sorting algorithm is called stable if it preserves the relative order of any
two equal elements in its input. i.e., if an input list contains two equal elements in
positions i and j where i < j, then in the sorted list they have to be in positions i’ and
j‘, respectively, such that i’< j’.
This property can be desirable if, for example, we have a list of students sorted
alphabetically and we want to sort it according to student GPA: a stable algorithm
will yield a list in which students with the same GPA will still be sorted
alphabetically.
2. The amount of extra memory the algorithm requires: An algorithm is said to be
in-place if it does not require extra memory, except, possibly, for a few memory
units. There are important sorting algorithms that are in-place and those that are not.
Searching
 The searching problem deals with finding a given value, called a search key, in a given
set.
Examples: sequential search, binary search

String Processing
 A string is a sequence of characters from an alphabet.
 Strings of particular interest are
 text strings, which comprise letters, numbers, and special characters;
 bit strings, which comprise zeros and ones;
 Gene sequences, which can be modeled by strings of characters from the four-
character alphabet {A, C, G, T}.
 String-processing algorithms have been found important for computer science for a long
time in conjunction with computer languages and compiling issues.
 One particular problem: string matching.
Several algorithms that exploit the special nature of this type of searching have been
invented.

Graph Problems
 A graph can be thought of as a collection of points called vertices, some of which are
connected by line segments called edges. Graphs can be used for modeling a wide variety
of applications, including transportation, communication, social and economic networks,
project scheduling, and games
 Basic graph algorithms include
 graph-traversal algorithms
 shortest-path algorithms
 topological sorting for graphs with directed edges
 Some graph problems are computationally very hard; the most well-known examples are
the traveling salesman problem and the graph-coloring problem.
 The traveling salesman problem (TSP) is the problem of finding the shortest tour
through n cities that visits every city exactly once. In addition to obvious applications
involving route planning, it arises in such modern applications as circuit board and
VLSI chip fabrication, X-ray crystallography, and genetic engineering.
 The graph-coloring problem seeks to assign the smallest number of colors to the
vertices of a graph so that no two adjacent vertices are the same color. This problem
arises in several applications, such as event scheduling: if the events are represented

Dept. of CSE, SJEC Page 26

Design and Analysis of Algorithms Module 1

by vertices that are connected by an edge if and only if the corresponding events
cannot be scheduled at the same time, a solution to the graph-coloring problem yields
an optimal schedule.

Combinatorial Problems
 These are problems that ask, explicitly or implicitly, to find a combinatorial object—such
as a permutation, a combination, or a subset—that satisfies certain constraints. A desired
combinatorial object may also be required to have some additional property such as a
maximum value or a minimum cost.
 From a more abstract perspective, the traveling salesman problem and the graph coloring
problem are examples of combinatorial problems.
 Combinatorial problems are the most difficult problems in computing, from both a
theoretical and practical standpoint.

Note:
 Among several ways to classify algorithms, the two principal alternatives are:
1. According to types of problems they solve.
2. According to underlying design techniques they are based upon.

 Algorithms operate on data. This makes the issue of data structuring critical for efficient
algorithmic problem solving. The most important elementary data structures are the array
and the linked list. They are used for representing more abstract data structures such as the
list, the stack, the queue, the graph (via its adjacency matrix or adjacency lists), the binary
tree, and the set.

7. Fundamental Data Structures: Stacks, Queues, Graphs, Trees, Sets and

Dictionaries. (T1:1.4)
Data Structure can be defined as a particular scheme of organizing related data items.
Linear Data Structure
 The two most elementary data structures are the array and the linked list.
 A one dimensional array is a sequence of n items of the same data type that are stored
contiguously in computer memory and made accessible by specifying a value of the
arrays index. As shown in Figure 1.

Item[0] Item[1] Item[n-1]

Figure 1: Array of n elements

 The index is an integer either between 0 and n-1 or between 1 and n.

 Arrays can also be used to implement other data Structures like the String, which is a
sequence of characters terminated by a special character call the NULL String.
 Strings composed of zeros and ones are called binary strings or bit strings.
 A Linked list is a sequence of zero or more elements called nodes each containing two
kind of information: data and one more links called pointers to other nodes of the linked
list.
 In a singly linked list, each node except the last one contains a single pointer to the next
element.

Dept. of CSE, SJEC Page 27

Design and Analysis of Algorithms Module 1

 To access a particular node, we start with the first node and traverse the pointer chain
until the particular node is reached.
 Doubly Linked List in which every node except the first and the last contains pointers to
both its successor and its predecessor.

 Arrays and Linked list are principal choices in representing more abstract data structure
called a linear list
 Basic operations performed on a Linear List are insertion, deletion and searching an
element.
 Example for Lists are Stacks and Queues.
 Stack is a list in which insertion and deletions can be done only at one end. This end is
called as the Top. Hence stack is commonly referred to LIFO(Last in First Out).
 A Queue is a list in which elements are inserted from one end called the rear(enqueue)
and deleted from the other end called the front(dequeue).
 A priority queue is a collection of data items from a totally ordered universe(integers and
real numbers)
 The main operations on a priority queue are finding the largest element,deleting the
largest element and adding a new element.

Graphs
A graph G=(V,E) is defined by a pair of two sets : a finite set V of items called vertices and
a set E of pairs of these lines called edges.
 If these pairs of vertices are unordered i.e a pair of vertices (u,v) is same as vertices(v,u)
then we say that vertices u and v are adjacent to each other and that they are connected by
undirected edge(u,v)
 If a pair of vertices(u,v) is not the same as the pair (v,u) we say that the edge (u,v) is
directed from the vertex u, called the edge‘s tail to the vertex v called the edge‘s head.

The graph in (a) has 6 vertices and eight directed edges.

V={a,b,c,d,e,f} E={(a,c),(a,d),(b,c),(b,f),(c,e),(d,e),(e,f)}
The Digraph in figure (b) has six vertices and eight directed edges
V={a,b,c,d,e,f}

Dept. of CSE, SJEC Page 28

Design and Analysis of Algorithms Module 1

E={(a,c),(b,c),(b,f),(c,e),(d,a),(d,e),(e,c),(e,f)}
 Graph Representations:graphs for computer algorithms can be represented in two
principal ways: the adjacency matrix and adjacency lists.
 The adjacency matrix of a graph with n vertices is a n by n Boolean matrix with one row
and one coloumn for each the graph vertices,in which the element in the ith row and the
jth coloumn is equal to 1 if there is an edge from the ith vertex to the jth vertex, and
equal to 0 if there is no such edge .
 Adjacency list of a graph or a diagraph is a collection of linked list,one for each vertex
that contain all the vertices adjacent to the lists vertex.

(a) Adjacency matrix (b) Adjacency list

Weighted Graphs:
 A weighted graph is a graph with numbers assigned to its edges. These numbers are
called weights or costs as shown below in figure (a).
 If a weighted graph is represented by its adjacency matrix then its element A[i,j] will
contain the weight of the edge from the ith to the jth vertex if there is such an edge and a
special symbol ……if there is no such edge. This matrix is called the weighted matrix or
cost matrix as shown below in figure (b).
 Adjacency lists for a weighted graph must include in their nodes the name of the adjacent
vertex and also the weight of the corresponding edge as shown in figure (c).

Paths and Cycles:

 A path from a vertex u to vertex v of a graph G can be defined as a sequence of adjacent
vertices that starts with u and ends with v. If all vertices of a path are distinct, the path is
said to be simple. The length of a path is the total number of vertices in a vertex sequence
defining the path minus one, which is same as number of edges in the path.
 A Directed path is a sequence of vertices in which every consecutive pair of the vertices
is connected by an edge directed from the vertex listed first to the vertex listed next. For
Example a,c,e,f is a directed path from a to f in the graph of figure b from Graphs
subsection..
 A graph is said to be connected if for every pair of its vertices u and v there is a path from
u to v.
Dept. of CSE, SJEC Page 29
Design and Analysis of Algorithms Module 1

 A connected component is a maximal connected subgraph of a given graph.

 A cycle is a path of a positive length that starts and ends at the same vertex and does not
traverse the same edge more than once. For eg. f,h,i,g,f is a cycle in the graph of figure
below.
 A graph with no cycles is called acyclic.

Figure: A graph that is not connected

Trees:
A tree is a connected acyclic graph. (Figure (a) below). A graph that has no cycles but is not
necessarily connected is called a forest. Each of the connected components is a tree.(figure
b).The number of edges in a tree is always one less than the number of vertices.|E|=|V|-1.

(a) Tree (b) Forest

Rooted Trees:
For every 2 vertices in a tree, there always exists one simple path from one of these vertices to
another. Any free tree can be converted to a rooted tree by choosing an arbitrary vertex as the
root of the rooted tree. The root is placed at the level 0 of the tree, the vertices adjacent to the
root below it(level 1),the vertices two edges apart from the root below that(level 2) and so figure
below shows transformation of a free tree to a rooted tree.

Dept. of CSE, SJEC Page 30

Design and Analysis of Algorithms Module 1

 For every vertex v in a tree T, all the vertices on the simple path from the root to that
vertex are called ancestors of v. If (u,v) is the last edge of the simple path from the root to
vertex v ,u is said to be the parent of v and v is called a child of u,
 Vertices who have the same parent are called siblings.
 A vertex with no children is called the leaf. A vertex with atleast one child is called
parental. All vertices for which a vertex v is an ancestor are said to be descendants of v.
 The depth of a vertex v is the length of the simple path from the root to v. The height of a
tree is the length of the longest simple path from the root to a leaf. Example Depth of the
vertex c in the tree in figure 9 is 2 and height of the tree is 3.

Ordered Trees:
An ordered tree is a rooted tree in which all the children of each vertex are ordered. A binary tree
can be defined as an ordered tree in which very vertex has no more than 2 children and each
child is designates as either the left child or the right child. In a binary search tree the nodes to
the left of the parent node has to be smaller than the parent node and the nodes to the right has to
larger than the parent node (figure a, b).

(a) Binary tree (b) Binary search tree

Figure below shows the standard implementation of binary search tree of figure (b) above.

Below figure (a) shows the first child-next sibling representation of the graph and figure (b)
shows its binary representation.

Dept. of CSE, SJEC Page 31

Design and Analysis of Algorithms Module 1

Sets and Dictionaries

 The set can be described as an ordered collection of distinct items called elements of the
set. A set can be defined either by an explicit listing of its elements eg. S={2,3,5,7} or by
specifying a property which the elements of the set must satisfy.
 Eg: S={n:n is a prime number and n<10}
 A multiset or a bag is an unordered collection of items that are not necessarily
distinct.The operations that need to be performed on a set or a multiset are searching,
inserting and deleting an item.
 A data structure that implements these three operations is called the dictionary.

Additional algorithms and their analysis:

Selection Sort
 We start selection sort by scanning the entire given list to find its smallest element and
exchange it with the first element, putting the smallest element in its final position in the
sorted list.
 Then we scan the list, starting with the second element, to find the smallest among the last
n − 1 elements and exchange it with the second element, putting the second smallest element
in its final position.
 Generally, on the ith pass through the list, which we number from 0 to n − 2, the algorithm
searches for the smallest item among the last n − i elements and swaps it with Ai :
A0 ≤ A1 ≤ . . . ≤ Ai–1 | Ai, . . . , Amin, . . . , An–1
in their final positions the last n – i elements

 After n − 1 passes, the list is sorted.

 Pseudocode of this algorithm:
ALGORITHM SelectionSort(A[0..n − 1])
//Sorts a given array by selection sort
//Input: An array A[0..n − 1] of orderable elements
//Output: Array A[0..n − 1] sorted in nondecreasing order
for i ←0 to n − 2 do

Dept. of CSE, SJEC Page 32

Design and Analysis of Algorithms Module 1

min←i
for j ←i + 1 to n − 1 do
if A[j ]<A[min] min←j
swap A[i] and A[min]

 Example: the action of the algorithm on the list 89, 45, 68, 90, 29, 34, 17 is illustrated below:

| 89 45 68 90 29 34 17
17 | 45 68 90 29 34 89
17 29 | 68 90 45 34 89
17 29 34 | 90 45 68 89
17 29 34 45 | 90 68 89
17 29 34 45 68 | 90 89
17 29 34 45 68 89 | 90

 Analysis of selection sort :

o Parameter to be considered is n (size of input).
o The basic operation is the key comparison A[j ]<A[min].
o The number of times basic operation executed depends only on the array size and is given
by the following sum:
n2 n 1
C(n ) =  1
i 0 j i 1
n2
= 
i 0
[(n-1)-(i+1)+1]
n2
= 
i 0
(n-1-i)
n2 n2

=  (n  1) -  i
i 0 i 0
n2

1 - (n  2)(2 n  1) n( n  1)
n
= (n-1) [  i= ]
i 0 i 1 2

(n  2)(n  1)
= (n-1)2 -
2
( n  1) n 1
=  n2  Θ(n2)
2 2

o Thus,
 Selection sort is a Θ(n2) algorithm on all inputs.
 The number of key swaps is only Θ(n), or, more precisely, n – 1. This property
distinguishes selection sort from many other sorting algorithms.
Bubble Sort:

 Bubble sort makes use of Brute-force strategy by comparing adjacent elements of the list and
exchanging them if they are out of order. By doing it repeatedly, we end up ―bubbling up‖
the largest element to the last position on the list.
Dept. of CSE, SJEC Page 33
Design and Analysis of Algorithms Module 1

 The next pass bubbles up the second largest element,

 The ith pass : Pass i (0 ≤ i ≤ n − 2) of bubble sort :
?
A0, . . . , Aj↔Aj+1, . . . , An−i−1 | An−i≤ . . . ≤ An−1
in their final positions
 After n − 1 passes, the list is sorted.

 Pseudocode:
ALGORITHM BubbleSort(A[0..n − 1])
//Sorts a given array by bubble sort
//Input: An array A[0..n − 1] of orderable elements
//Output: Array A[0..n − 1] sorted in ascending order
for i ←0 to n − 2 do
for j ←0 to n − 2 − i do
if A[j + 1]<A[j ] swap A[j ] and A[j + 1]
 Example: the action of the algorithm on the list 89, 45, 68, 90, 29, 34, 17 is illustrated below:
89 
?
45 68 90 29 34 17
45 89  68
?
90 29 34 17
45 68 89  90  29
? ?
34 17
45 68 89 29 90  34
?
17
45 68 89 29 34 90 
?
17
45 68 89 29 34 17 | 90

45 
?
68 
?
89 
?
29 34 17 | 90
45 68 29 89  34
?
17 | 90
45 68 29 34 89  17 | 90
?

45 68 29 34 17 | 89 90
…………………………… etc
 Analysis:
o Parameter to be considered is n (size of input).
o The basic operation is the key comparison A[j+1 ]<A[j].
o The number of key comparisons for the bubble-sort is the same for all arrays of size n;
and it is given below:
n2 n  2 i
C(n) = 
i 0
1
j 0
n2
=  [( n  2  i)  0  1]
i 0
n2
=  (n  1  i)
i 0

( n  1) n
=  Θ(n2) [ same as selection sort analysis ]
2
 The number of key swaps, however, depends on the input. In the worst case of decreasing
arrays, it is the same as the number of key comparisons:
( n  1) n
Sworst(n) = C(n) =  Θ(n2)
2

Dept. of CSE, SJEC Page 34

Design and Analysis of Algorithms Module 1

Reference Questions:
1. What is an algorithm? What are the essential properties of an algorithm? Explain
2. Explain in brief, the basic asymptotic efficiency classes
3. What are space and time complexities of an algorithm?
4. Explain how you would compute the time complexity of non-recursive algorithms.
5. Explain how you would compute the time complexity of recursive algorithms.
6. What are asymptotic notations? What is their significance? Explain.
7. Explain different asymptotic notations used to analyze an algorithm
8. Explain asymptotic notations and efficiency classes.
9. Explain the general framework of algorithms. Explain best-case, worst-case and average-
case efficiencies with an example.
10. Define the asymptotic notations for the best-case, worst-case and average-case
efficiencies with an example.
11. With at least two examples each, Define and explain O, Θ and 
12. If t1(n)  O(g1(n)) and t2(n)  O(g2(n)), then prove that, t1(n)+t2(n)  O(max{g1(n), g2(n)}).
13. Explain the method of comparing orders of growth of two functions using limits. Also
compare the following functions:
(i) log2n and n (ii) (log2n)2 and log2n2
14. Calculate the efficiency of an algorithm for finding the largest element in an array.
15. Write an algorithm to perform matrix multiplication and calculate its efficiency.
16. Describe the non-recursive algorithm for finding the binary representation of a given
decimal number. Also calculate its efficiency.
17. For the following algorithms, indicate:
(a) Natural size metric for the inputs (b) Basic operations (c) Whether the basic
operation counts can be different for inputs of the same size
(i) Computing sum of ‗n‘ numbers
(ii) Computing xn
(iii) Finding largest element in a list of ‗n‘ numbers
(iv) Euclid‘s algorithm

18. Prove that (i) ½ n(n-1)  Θ(n2) (ii) 3n3+2n2  O(n3) (iii) n!   (2n)
19. Indicate whether the first function in each of the following pair, has smaller, same or
larger order of growth than the second function.
(i) n(n+1) and 2000n2 (ii) 100 n2 and 0.01 n3 (iii) log2n and ln n
(iv) 2n-1 and 2n (v) log2n and log2n2 (vi) (n-1)! And n!
20. Consider the following algorithm
Algorithm X(int N)
{
int P;
for i =0 to N
{
Printf(―\n %d\t * \t %d = %d ―, N,i,P);
P=P+N;
}
}
(i) What does this algorithm compute?
(ii) What is the basic operation?
(iii) How many times basic operation is executed?

Dept. of CSE, SJEC Page 35

 Design and Analysis of Algorithms Module 1

(iv) What is the efficiency class of this algorithm?

21. Consider the following algorithm
Algorithm Mystery(int N)
int S=0
for i =1 to N
S=S+i*i;
Return S
(i) What does this algorithm compute?
(ii) What is the basic operation?
(iii) How many times basic operation is executed?
(iv) What is the efficiency class of this algorithm?
22. Consider the following algorithm
Algorithm PQ(A[0..n-1, 0…n-1])
for i=0 to n-2
for j =i+1 to n-1 do
if A[i,j]≠A[j,i]
return false;
end for
end for
return true
(i) What does this algorithm compute?
(ii) What is the basic operation?
(iii) How many times basic operation is executed?
(iv) What is the efficiency class of this algorithm?

23. Compute the time complexity of following recurrence relations

(i) T(n) = 4T(n/2) +n , T(1) = 1 (ii) T(n) = 4T(n/2) +n2 , T(1) = 1
(iii) T(n) = 4T(n/2) +n3 , T(1) = 1 (iv) T(n) = 2T(n/2)+Cn , T(1) = 0
24. Explain with formal definitions of asymptotic notation, write a recursive algorithm to
find the sum of cubes of first n numbers. Obtain its complexity.
25. Consider the following recursive algorithm for computing the sum of first n cubes,
S(n)= 13+23+33+…………+n3
Algorithm S(n)
if(n==1) return 1
else return ( S(n-1)+n*n*n)
end
Set up and solve the recurrence relation for the above algorithm
26. Design an algorithm to check whether all elements in an array are distinct. What is its
time complexity?
27. Give the general plan for analyzing recursive algorithms. Mathematically analyze the
Towers of Hanoi, and calculate its efficiency.
28. Given a positive decimal integer ‗n‘, design a recursive algorithm to compute the number
of binary digits present for n. obtain a recurrence relation and calculate its time
efficiency.
29. Explain selection sort algorithm. Calculate its time efficiency.
30. Write an algorithm to search for a key element in an array using linear search. Compute
its best, worst and average case efficiencies.

*******************************************************************
Dept. of CSE, SJEC Page 36
Copy protected with Online-PDF-No-Copy.com