Design and Analysis of Algorithms

Unit-I
 Algorithm
• Definition
An algorithm is a finite set of instructions that
accomplishes a particular task.
All algorithms must satisfy the following criteria.
• Criteria
1. Input : Zero or more quantities are externally
supplied.
2. Output : At least one quantity is produced.
3. Definiteness : Each instruction is clear and
unambiguous.
Statements such as “add 6 or 7 to x”
or “Compute 5/0” are not permitted”
4. Finiteness : The algorithm should terminate after
a finite number of steps.
5. Effectiveness : Instruction is basic enough to be
carried out.
Pseudo code for expressing algorithms
• We present most of our algorithms using pseudo code that
resembles C and Pascal.

1. Comments begin with // and continue until end of the line.

2. Blocks are indicated with matching braces: { and }.
i. A compound statement
ii. Body of a procedure.

3.
i. An identifier begins with a letter.
ii. The data types of variables are not explicitly declared.
iii. The types will be clear from the context.
iv. Whether a variable is global or local to a procedure will also be evident from
the context.
v. We assume simple data types such as integer, float, char, boolean, and so
on.
vi. Compound data types can be formed with records.
node = record
{ C style :-

datatype_1 data_1; struct node

: {
datatype_n data_n; datatype_1 data_1;
:
node *link datatype_n data_n;
}
data items of a record can be accessed struct node *link

with → and period( . ) }

4. Assignment of values to variables is done using the
assignment statement.
< variable > := < expression >

5. There are two boolean values true and false. To

produce these values, logical operators and, or and
not and the relational operators <, ≤,=, ≠, ≥ and > are
provided.

6. Elements of multidimensional arrays are accessed

using [ and ]. For example the (i,j)th element of the
array A is denoted as A[i,j].

7. The following looping statements are used: for, while,

and repeat until.
The general form of a while loop:
while( condition ) do
{
statement_1;
:
statement_n;
}
The general form of a for loop:
for variable := value1 to value2 step step do

– Here value1, value2, and step are arithmetic expressions.

– The clause “step step” is optional and taken as +1 if it does not
occur.
– step could be either positive or negative.
– variable is tested for termination at the start of each iteration.
The for loop can be implemented as a
while loop as follows:
variable:=value1;
incr:=step;
while( ( variable – value2)*step ≤ 0) do
{
<statement 1>
:
<statement n>
variable :=variable+incr;
}
• The general form of a repeat-until loop:

repeat
<statement 1>
:
<statement n>
until ( condition )

The statements are executed as long as

condition is false.
8. A conditional statement has the following
forms:
➢ if < condition > then < statement >

➢ if < condition > then

< statement 1>
else
< statement 2>

9. Input and output are done using the instructions read

and write.
Ex: read n;
write n;
10. Procedure or function starts with the word
Algorithm.

General form :
Algorithm Name( <parameter list> )
{
body
}
where Name is the name of the procedure.
– Simple variables to procedures are passed by value.
– Arrays and records are passed by reference
Ex:-Algorithm that finds and returns the
maximum of n given numbers
Algorithm max(a,n)
// a is an array of size n
{
Result:=a[1];
for i:=2 to n do
if a[i]>Result then Result:=a[i];
return Result;
}
Ex:-Write an algorithm to sort an array of n
integers using bubble sort.

Algorithm(a,n)
// a is an array of size n
{
for i:=1 to n-1 do
{
for j:=1 to n-i do
{
if( a[j] >a[j+1] ) then
t=:a[j]; a[j]:=a[j+1]; a[j]:=t;
}
}

}
 Performance Analysis
(machine independent)
• There are many things upon which the
performance will depend.

– Does the program efficiently use primary and

Secondary storage?
– Is the program's running Time acceptable for the
task?
– Does it do what we want it to do?
– Does it work correctly according to the
specifications of the task?
– Does the program contain documentation that
shows how to use it and how it works?
– Is the program's code readable?
 Practice
Practice
Practice
How to achieve them?
– Good programming style, experience, and practice
– Discuss and think.

Think
Think
Think
 Space Complexity
• The space needed by a program has the
following components.
– Instruction space
• Instruction space is the space needed to store the
compiled version of the program instructions.
• The amount of instruction space that is needed
depends on the compiler used to compile the
program into machine code.
– Data space
• Data space is the space needed to store all
constant and variable values. Data space has two
components.
– Space needed by constants( ex; 1 and 2 in max of n num
algorithm) and simple variables( such as i, j, n etc).
– Space needed by a dynamically allocated objects such
as arrays and class instances.
• Space taken by the variables and constants varies
from language to language and platform to
platform.
– Environmental stack space
• The environment stack is used to save information
needed to resume execution of partially completed
functions.
• Each time a function is invoked the following data
are saved on the environment stack.
– The return address .
– The values of all local variables and formal parameters in
the function being invoked( necessary for recursive
functions only).
Recursive algorithm
Algorithm rfactorial(n)
// n is an integer
{ fact=1;
if (n=1 or n=0) return fact;
else
fact=n*rfactorial(n-1);
return fact;
}
Each time the recursive function rfactorial is invoked,
the current values of n and fact and the program location to return to
on completion are saved in the environment stack.
 Execution n: 2
R: n: 3
int rfactorial(n){ //3500
void main() int rfactorial(n){ //2500
fact=1
{int R=rfactorial(3); fact=1; if (n=0 or n=1)return fact
printf(R);} //2000 if (n=0 or n=1)return fact; else
else
fact=n*rfactorial(n-1); //4000
fact=n*rfactorial(n-1); //3000
return fact;}
return fact;
}
R n=3
R=6 fact=6
fact=1 n: 1
2000 n=2
3000 int rfactorial(n){
fact=1 4000
fact=2
fact=1 if (n=0 or n=1)
return 1;
else
fact=n*rfactorial(n-1);
return fact;
}

Stack
Summary of space complexity
• The space needed by a program depends on several
factors.
• We cannot make an accurate analysis of the space
requirements of a program unless we know the computer
or compile that will be used.
• However, we can determine the components that
depend on the characteristics of the problem instance
(e.x., the number of inputs and outputs or magnitude of the numbers
involved ) to be solved.

Ex:-1. Space requirements of a program that sorts n elements can

be expressed as a function of n.
2. Space requirements of a program that adds two m×n
matrices can be expressed as a function of m, n.
 Contd..

– The size of the instruction space is independent of the problem

instance to be solved.

– The contribution of the constants and simple variables to the

data space is also independent of the problem instance to be
solved.

– Most of the dynamically allocated memory( ex., arrays, class

instances etc) depends on problem instance to be solved.

– The environmental stack space is generally independent of the

problem instance unless recursive functions are in use.
 Contd..
Therefore, We can divide the total space needed by a
program into two parts:
i) Fixed Space Requirements (C)
Independent of the characteristics of the problem instance ( I )
• instruction space
• space for simple variables and constants.
ii) Variable Space Requirements (SP(I))
depend on the characteristics of the problem instance ( I )
• Number of inputs and outputs associated with I
• recursive stack space ( formal parameters, local variables,
return address ).
– Therefore, the space requirement of any problem P
can be written as
S(p)=C +Sp( Instance characteristics )
• Note:
– When analyzing the space complexity of an algorithm,
we concentrate only on estimating
Sp (Instance characteristics ).
– We do not concentrate on estimating fixed part c .
– We need to identify the instance characteristics of the
problem to measure Sp
Example1

Algorithm abc(a,b,c)
{
return a+b+b*c+(a+b-c)/(a+b)+4.0;
}
• Problem instance characterized by the specific values of a,b,and c.
• If we assume one word (4 bytes) is adequate to store the values of
each a, b, and c , then the space needed by abc is independent of
the instance characteristics.
Therefore, Sabc( instance characteristics)=0
Example2
Algorithm sum(a,n)
{
Recall: Address of the
s:=0;
first element of the array will
for i:=1 to n do
be passed .
s:=s+a[i];
return s;
}
• Problem instance characterized by n.
• The amount of space needed does depend on the value of n.

Therefore, Ssum(n)=n words + 3

Example3
Algorithm RSum(a,n)
{
if(n ≤ 0) then return 0;
else return RSum(a,n-1)+a[n];
}

Type Name Number of bytes

formal parameter: int a 2
formal parameter: int n 2
return address 2
( used internally)
Total per one recursive 6
call
Total no.of recursive calls n, therefore SRSum (n)=6(n+1)
Example4

Calculate space complexity sp(n) for the following algorithm

Algorithm rfactorial(n)
{
fact=1;
if (n=0 or n=1)return fact;
else
fact=n*rfactorial(n-1);
return fact;
}
 Contd..

• Time Complexity:
T(P)=C+TP(I)
– The time, T(P), taken by a program, P, is the sum of
its compile time C and its run (or execution) time,
TP(I).
– The compile time does not depend on the instance
characteristics.
– We will concentrate on estimating run time Tp(I).
• If we know the characteristics of the compiler to be used, we
can determine the
– No. of additions , subtractions, multiplications, divisions, compares,
and so on.
• Then we can obtain an expression for Tp(n) Of the form
TP(n)=caADD(n)+ csSUB(n)+ cm MUL(n)+ cd DIV(n)+……..
where,
– n denotes the instance characteristics.
– ca, cs, cm, cd and so on denote the time needed for an addition ,
subtraction, multiplication, division and so on. And
– ADD, SUB, MUL, DIV, and so on are functions whose values are the
no.of additions, subtractions, multiplications, divisions, and so on.
• Obtaining such an exact formula is an impossible task,
since time needed for an addition, subtraction, and so
on, depends an numbers being added, subtracted, and
so on.
• The value of Tp(n) for any given n can be obtained only
experimentally.
• Even with the experimental approach, one could face
difficulties.
• In a multi user system the execution time of a program p
depends on the number of other programs running on
the computer at the time program p is running.
 Contd..
• As there were some problems in determining the execution
time using earlier methods, we will go one step further and
count only the number of program steps.
program step
program step is loosely defined as a syntactically or
semantically meaningful program segment whose
execution time is independent of the instance
characteristics.
– Example
result = a + b + b * c + (a + b - c) / (a + b) + 4.0;
sum = a + b + c;
• The number of steps assigned to any program statement
depends on the kind of statement.
• Comments are counted as zero number of steps.
• An assignment statement which does not involve any
calls to other functions counted as one step.
• Loops, such as the for, while, and repeat-until, we
consider the step counts only for the control part of the
statement.
• The control parts for for and while statements have the
following forms:
for i:= <expr1> to <expr2> do
while ( <expr> ) do
• Each execution of the control part of a while statement is one,
unless <expr> is a function of instance characteristics.

• The step count for each execution of the control part of a for
statement is one, unless <expr1> and <expr2> are functions of the
instance characteristics.

• The step count for each execution of the condition of a conditional

statements is one, unless condition is a function of instance
characteristics.
• If any statement ( assignment statement, control part, condition etc.)
involves function calls, then the step count is equal to the number of
steps assignable to the function plus one.
• Methods to compute the step count
1) Introduce global variable count into programs with
initial value zero.
• Statements to increment count by the appropriate amount
are introduced into the program.
• The value of the count by the time program terminates is the
number steps taken by the program.
2) Tabular method
• Determine the total number of steps contributed by each
statement per execution  frequency
• Add up the contribution of all statements
Method-I: Introduce variable count into programs

EX:- Iterative sum of n numbers

Algorithm sum(a, n)
{
s:=0;
count:=count+1; // for assignment statement
for i:=1 to n do
{ count:=count+1; // For for
s:=s+a[i];
count:=count+1; // for assignment statement
}
count:=count+1; // for last time of for
return s;
count:=count+1; // for return
}

2n + 3 steps
Note : Step count tells us how the run time for a program changes with
changes in the instance characteristics.
Ex:- Addition of two m×n matrices
Algorithm Add(a,b,c,,m,n)
{
for i:=1 to m do
{
for j:=1 to n do
{
c[i,j]:=a[i,j]+b[i,j];

}
}
2mn + 2m+1 steps
}
EX:- Recursive sum of n numbers
Algorithm RSum(a,n)
{
count:=count+1; // for the if conditional
if(n ≤ 0) then
{
return 0;
count:=count+1; // for the return
}
else
{
return RSum(a,n-1)+a[n];
count:=count+1; // For the addition, function invocation and return
}
}
• When analyzing a recursive program for its step count, we often
obtain a recursive formula for the step count.
• We obtain the following recursive formula for above (RSum)
algorithm.

2 If n=0
tRSum (n)= 2+ tRSum(n-1) If n>0
• One way of solving such recursive formula is by repeated
substitutions for each occurrence of the function tRSum on the right
side until all such occurrences disappear:

tRSum (n) = 2+tRSum(n-1)

= 2+2+tRSum(n-2)
= 2(2)+tRSum(n-2)
= 2+2+2+tRSum(n-3)
= 2(3)+tRSum(n-3)
:
:
= n(2) +tRSum(n-n)
=2n+tRSum(0)
= 2n+2
The step count for Rsum is 2n+2
Method-II: Tabular method
• EX:- Iterative sum of n numbers
Statement s/e frequency Total steps
Algorithm sum(a, n) 0 -- 0
{ 0 -- 0
s:=0 ; 1 1 1
for i:=1 to n do 1 n+1 n+1
s:=s+a[i]; 1 n n
return s; 1 1 1
} 0 -- 0

Total 2n+3
• EX:- Addition of two m×n matrices

Statement s/e frequency Total steps

Algorithm Add(a,b,c,m, n) 0 -- 0
{ 0 -- 0
for i:=1 to m do 1 m+1 m+1
for j:=1 to n do 1 m(n+1) mn+m
c[i,j]:=a[i,j]+b[i,j] ; 1 mn mn
} 0 -- 0

Total 2mn+2m+1
• EX:- Recursive sum of n numbers

Statement s/e Frequency Total steps

n=0 n>0 n=0 n>0
Algorithm RSum(a,n) 0 -- -- 0 0
{ 0 -- -- 0 0
if( n ≤ 0 ) then 1 1 1 1 1
return 0; 1 1 0 1 0
else return
Rsum(a,n-1)+a[n] ; 1+x 0 1 0 1+x
} 0 -- -- 0 0

Total 2 2+x

x=tRSum(n-1)
 Best, Worst, Average Cases
✓ Not all inputs of a given size take the same number of program
steps.
✓ Sequential search for K in an array of n integers:
• Begin at first element in array and look at each element in
turn until K is found.
1. Best-Case Step count:-
Minimum number of steps executed by the algorithm for the
given parameters.
2. Worst-Case Step count:-
Maximum number of steps executed by the algorithm for the
given parameters.
3. Average-Case Step count:-
Average number of steps executed by an algorithm.
 Contd..

5 ms Worst -case time

4 ms

3 ms
Running
Best-case time
Time 2 ms

1 ms

A B C D E F G

Input

A time + B time+ ………..+G time

Average-case time = ----------------------------------------------------
7 ( total number of possible inputs )
 Inexactness of step count.
• Both the instructions x=y; and x=y+z+(x/y) count as one
step.
• Therefore, two analysts may arrive at 4n2+6n+2 and
7n2+3n+4 as the step count for the same program.

• Any step count of the form c1n2+c2n+c3 could be a correct

step count for the program.

• Because of the inexactness of what a step count stands for,

the exact step count is not very useful for comparison of
algorithms.
 Asymptotic efficiency

• Asymptotic efficiency means study of algorithms

efficiency for large inputs.

• To compare two algorithms with running times f(n)

and g(n), we need a rough measure that
characterizes how fast each function grows as n
grows.

• Hint: use rate of growth

• Compare functions asymptotically!
(i.e., for large values of n)
– To get a feel for how the various functions grow
with n, you are advised to study the following figs:
Functions ordered by growth rate

Function Name
1 Growth is constant
logn Growth is logarithmic
n Growth is linear
nlogn Growth is n-log-n
n2 Growth is quadratic
n3 Growth is cubic
2n Growth is exponential
n! Growth is factorial

1 < logn < n < nlogn < n2 < n3 < 2n < n!

2
 Rate of Growth
• Ex:- f(n)=n2+100n+log10n+1000
n f(n) n2 100n log10n 1000

value value % value % value % value %

1 1,101 1 0.1 100 9.1 0 0.0 1,000 90.83

10 2,101 100 4.76 1,000 47.6 1 0.05 1,000 47.60

100 21,002 10,000 47.6 10,000 47.6 2 0.001 1,000 4.76

1,000 1,101,003 1,000,000 90.8 100,000 9.1 3 0.0003 1,000 0.09

10,000 101,001,004 100,000,000 99.0 1,000,000 0.99 4 0.0 1,000 0.001

1,00000 10,010,001005 10,000,000,000 99.9 10,000,000 0.099 5 0.0 1,000 0.00

• The following fig gives the time needed by a 1 billion
instructions per second computer to execute a
program of complexity f(n) instructions.
• The low order terms and constants in a function are relatively
insignificant for large n
n2 + 100n + log10n + 1000 ~ n2

i.e., we say that n2 + 100n + log10n + 1000 and n2 have the same
rate of growth

Some more examples

• n4 + 100n2 + 10n + 50 is ~ n4
• 10n3 + 2n2 is ~ n3
• n3 - n2 is ~ n3
• constants
➢ 10 is ~ 1
➢ 1273 is ~ 1
 Asymptotic/order Notations

• Asymptotic/order notation describes the

behavior of functions for the large inputs.

• Big Oh(O) notation:

– The big oh notation describes an upper bound
on the asymptotic growth rate of the function f.

Definition: [Big “oh’’]

– f(n) = O(g(n)) (read as “f of n is big oh of
g of n”) iff there exist positive
constants c and n0 such that
f(n)  cg(n) for all n, n  n0.
• The definition states that the function f(n) is at most c
times the function g(n) except when n is smaller than n0.
• In other words, f(n) grows slower than or same rate as”
g(n).
• When providing an upper –bound function g for f, we
normally use a single term in n.
• Examples
– f(n) = 3n+2
• 3n + 2 <= 4n, for all n >= 2, 3n + 2 =  (n)

– f(n) = 10n2+4n+2
• 10n2+4n+2 <= 11n2, for all n >= 5,  10n2+4n+2 =
 (n2)

– f(n)=6*2n+n2=O(2n) /* 6*2n+n2 7*2n for n4 */

• It also possible to write 10n2+4n+2 = O(n3) since
10n2+4n+2 <=7n3 for n>=2

• Although n3 is an upper bound for 10n2+4n+2, it is not a

tight upper bound; we can find a smaller function (n2) that
satisfies big oh relation.

• But, we can not write 10n2+4n+2 =O(n), since it does not

satisfy the big oh relation for sufficiently large input.
• Omega () notation:
– The omega notation describes a lower bound
on the asymptotic growth rate of the function f.

Definition: [Omega]
– f(n) = (g(n)) (read as “f of n is
omega of g of n”) iff there exist
positive constants c and n0
such that f(n)  cg(n) for all n,
n  n0.
• The definition states that the function f(n) is at least c times the
function g(n) except when n is smaller than n0.
• In other words,f(n) grows faster than or same rate as” g(n).
• Examples
– f(n) = 3n+2
• 3n + 2 >= 3n, for all n >= 1, 3n + 2 =  (n)

– f(n) = 10n2+4n+2
• 10n2+4n+2 >= n2, for all n >= 1,  10n2+4n+2 =  (n2)

• It also possible to write 10n2+4n+2 = (n) since 10n2+4n+2 >=n for

n>=0

• Although n is a lower bound for 10n2+4n+2, it is not a tight lower

bound; we can find a larger function (n2) that satisfies omega
relation.

• But, we can not write 10n2+4n+2 = (n3), since it does not satisfy
the omega relation for sufficiently large input.
• Theta () notation:
– The Theta notation describes a tight bound
on the asymptotic growth rate of the
function f.

Definition: [Theta]
– f(n) = (g(n)) (read as “f of n is theta
of g of n”) iff there exist positive
constants c1, c2, and n0 such that
c1g(n)  f(n)  c2g(n) for all n, n  n0.
• The definition states that the function f(n) lies between c1 times the
function g(n) and c2 times the function g(n) except when n is smaller
than n0.
• In other words,f(n) grows same rate as” g(n).

• Examples:-
– f(n) = 3n+2
• 3n <= 3n + 2 <= 4n, for all n >= 2,  3n + 2 =  (n)

– f(n) = 10n2+4n+2
• n2<= 10n2+4n+2 <= 11n2, for all n >= 5,  10n2+4n+2 =  (n2)
• But, we can not write either 10n2+4n+2= (n) or 10n2+4n+2= (n3),
since neither of these will satisfy the theta relation.
• Little Oh(O) notation:
– The big oh notation describes a strict
upper bound on the asymptotic growth
rate of the function f.

Definition: [Little “oh’’]

– f(n) = o(g(n)) (read as “f of n is little

oh of g of n”) iff

Lim f(n) =0
n->∞ g(n)
• The definition states that the function f(n) is less than c times the
function g(n) except when n is smaller than n0.
• In other words, f(n) grows slower than” g(n).
• Examples
– f(n) = 3n+2=o(n2)
• Since 3n+2
Lim =0
n->∞ n2

– However, 3n+2 ≠ o(n)

Big-Oh, Theta, Omega and Little-oh

Tips :
• Think of O(g(n)) as “less than or equal to” g(n)
– Upper bound: “grows slower than or same rate as” g(n)

• Think of Ω(g(n)) as “greater than or equal to” g(n)

– Lower bound: “grows faster than or same rate as” g(n)

• Think of Θ(g(n)) as “equal to” g(n)

– “Tight” bound: same growth rate

• Think of o(g(n)) as “less than to” g(n)

– Strict Upper bound: “grows slower than ” g(n)

• (True for large N)