UNIT I: INTRODUCTION

Algorithm, Performance Analysis-Space complexity, Time complexity, Asymptotic Notation- Big oh notation,
Omega notation, Theta notation and Little oh notation.
Divide and conquer: General method, applications-binary search, quick sort, merge sort, Strassen’s matrix
multiplication.

ALGORITHM:
Algorithm was first time proposed a purshian mathematician Al-Chwarizmi in 825 AD.
According to web star dictionary, algorithm is a special method to represent the procedure to
solve given problem.

OR

An Algorithm is any well-defined computational procedure that takes some value or set of
values as Input and produces a set of values or some value as output. Thus algorithm is a
sequence of computational steps that transforms the input into the output.

Formal Definition:

An Algorithm is a finite set of instructions that, if followed, accomplishes a

particular task. In addition, all algorithms should satisfy the following criteria.

1. Input. Zero or more quantities are externally supplied.

2. Output. At least one quantity is produced.
3. Definiteness. Each instruction is clear and unambiguous.
4. Finiteness. If we trace out the instructions of an algorithm, then for all cases, the
algorithm terminates after a finite number of steps.
5. Effectiveness. Every instruction must very basic so that it can be carried out, in
principle, by a person using only pencil & paper.

Areas of study of Algorithm:

 How to device or design an algorithm– It includes the study of various design

techniques and helps in writing algorithms using the existing design techniques
like divide and conquer.
 How to validate an algorithm– After the algorithm is written it is necessary to
check the correctness of the algorithm i.e for each input correct output is produced,
known as algorithm validation. The second phase is writing a program known
as program proving or program verification.
 How to analysis an algorithm–It is known as analysis of algorithms or
performance analysis, refers to the task of calculating time and space complexity
of the algorithm.
 How to test a program – It consists of two phases . 1. debugging is detection and
correction of errors. 2. Profiling or performance measurement is the actual amount
of time required by the program to compute the result.
Algorithm Specification:

Algorithm can be described in three ways.

1. Natural language like English:
2. Graphic representation called flowchart:
This method will work well when the algorithm is small& simple.
3. Pseudo-code Method:
In this method, we should typically describe algorithms as program, which resembles
language like Pascal &algol.

Pseudo-Code for writing Algorithms:

1. Comments begin with // and continue until the end of line.

2. Blocks are indicated with matching braces {and}.
3. An identifier begins with a letter. The data types of variables are not explicitly
declared.
4. Compound data types can be formed with records. Here is an example,
Node. Record
{
data type – 1 data-1;
data type – n data – n;
node * link;
}
Here link is a pointer to the record type node. Individual data items of a
record can be accessed with  and period.
5. Assignment of values to variables is done using the assignment statement.
<Variable>:= <expression>;

6. There are two Boolean values TRUE and FALSE.

Logical Operators AND, OR, NOT

Relational Operators <, <=,>,>=, =, !=

7. The following looping statements are employed.

For, while and repeat-until

While Loop:

While < condition >do{

<statement-1>
.. .
<statement-n>
}
For Loop:

For variable: = value-1 to value-2 step step do

{
<statement-1>
..
<statement-n>
 }
One step is a key word, other Step is used for increment or decrement.

repeat-until:

repeat{
<statement-1>
….
<statement-n>
}until<condition>
8. A conditional statement has the following forms.
(1) If <condition> then <statement>

(2) If <condition> then <statement-1>

Else <statement-2>

Case statement:

Case
{ :<condition-1>:<statement-1>
….
:<condition-n>:<statement-n>
:else:<statement-n+1>
}
9. Input and output are done using the instructions read & write.
10. There is only one type of procedure:
Algorithm, the heading takes the form,

Algorithm Name (<Parameter list>)

As an example, the following algorithm fields & returns the maximum of ‘n’ given
numbers:

Algorithm Max(A,n)
// A is an array of size n
{
Result := A[1];
for I:= 2 to n do
if A[I] > Result then
return Result;
}

In this algorithm (named Max), A & n are procedure parameters. Result & I are
Local variables.
Performance Analysis:
There are many Criteria to judge an algorithm.
– Is it correct?
– Is it readable?
– How it works
Performance evaluation can be divided into two major phases.

1. Performance Analysis (machine independent)

– space complexity: The space complexity of an algorithm is the amount of

memory it needs to run for completion.

– time complexity: The time complexity of an algorithm is the amount of

computer time it needs to run to completion.

2. Performance Measurement (machine dependent).

Space Complexity:

The Space Complexity of any algorithm P is given by S(P)=C+SP(I),C is constant.

1. Fixed Space Requirements (C)

Independent of the characteristics of the inputs and outputs
– It includes instruction space
– space for simple variables, fixed-size structured variable, constants
2. Variable Space Requirements (SP(I))
depend on the instance characteristic I
– number, size, values of inputs and outputs associated with I
– recursive stack space, formal parameters, local variables, return address
Example Program 1 :

Simple arithmetic function Algorithmabc( x, y, z)

{
return x* y*z + (x- y) ;
}

Here we have 3 independent variables x,y,z

So C=3, SP(I)=0

Hence S(P)=3+0=3=Constant Therefore Space Complexity is O(1)

 Example Program 2 :

int sum( int x, int y, int z)

{
int t=x+y+z ;
return t;
}
Considering integer occupies 2 Bytes, Here we have x,y,z,t and return t, each one requires 2 Bytes. So total space
required is 10 Bytes.
C=10, SP(I)=0

Hence S(P)=10+0=Constant Therefore Space Complexity is O(1)

Example Program 3:
Iterative function for sum a list of numbers:

Algorithm sum( list[ ], n)

{
tempsum = 0;
for i = 0 to ndo
tempsum += list [i];
return tempsum;

In the above example list[] is dependent on n. Hence S P(I)=n. The remaining variables
are i,n, tempsum each requires one location.

Hence S(P)=3+n Therefore Space Complexity is O(n)

Example Program 4:
Algorithm int sum(int a[],int n)
{
int r=0;
for(int i=0;i<=n;i++)
{
r=r+a[i];
}
return r;
}
In the above example a[] is dependent on i. Hence SP(I)=n. The remaining variablesare
i, n, r, return r each requires one location.

Hence S(P)=4+n Therefore Space Complexity is O(n)

Example Program 5:
void matrixAdd(int a[],int b[],int c[],int n)
{
for(int i=0;i<=n;i++)
{
c[i]=a[i]+b[j];
}
}
In the above example a[],b[],c[] are dependent on i. Hence SP(I)=3n. The remaining
variablesare i, n each requires one location.

Hence S(P)=3n+2 Therefore Space Complexity is O(n)

Example Program 5: Recursive function for sum a list of numbers

Algorithm rsum( list[ ], n)

{

If (n<=0) then

return 0.0

else
return rsum(list, n-1) + list[n];
}
In the above example the recursion stack space includes space for formal parameters, local variables
and return address. Each call to rsum requires 3 locations i.e for list[],n and return address. As the
length of recursion is n+1.

So S(P)>=3(n+1)

Time complexity:

T(P)=C+TP(I)

It is combination of-Compile time (C)

independent of instance characteristics
-run (execution) time TP
dependent of instance characteristics
Time complexity is calculated in terms of program step as it is difficult to know the
complexities of individual operations.
Definition: A program step is a syntactically or semantically meaningful
programsegment whose execution time is independent of the instance
characteristics.
 Program steps are considered for different statements as : for comment zero steps .
assignment statement is considered as one step. Iterative statements such as “for, while
and until-repeat” statements, we consider the step counts based on the expression .

Methods to compute the step count:

1) Introduce variable count into programs
2) Tabular method(Sequency)
– Determine the total number of steps contributed by each statement
step per execution(s/e)  frequency
– add up the contribution of all statements
Count Method:
Program 1.with count statements
Algorithm sum( list[ ], n)
{
tempsum := 0;
count=count+1; /* for assignment */
for i := 1 to n do
{
count=count+1; /*for the for loop */
tempsum := tempsum + list[i];
count=count+1;/* for assignment */
}
count=count+1; /* last execution of
for */
return tempsum;
count=count+1; /* for return */
Hence T(n)=2n+3

Program 2.with count statements

Algorithm sum( x, n)
{
total:= 0;
count=count+1; /* for assignment */
for i := 1 to n do
{
count=count+1; /*for the for loop */
total := total + i;
count=count+1;/* for assignment */
}
count=count+1; /* last execution of
for */
}
Hence T(n)=2n+2
Program 3:with count statements
Algorithm add( a, b)
{
for i := 1 to n do
{
count=count+1; /*for the for loop */
for j := 1 to n do
{
count=count+1; /*for the for loop */
c[i][j]=a[i][j]+b[i][j];
count=count+1; /* for assignment */
}
count=count+1; /* last execution of for */
}
count=count+1; /* last execution of for */
}
Hence T(n)=2n2+2n+1

II Tabular method.

Complexity is determined by using a table which includes steps per execution(s/e)

i.e., amount by which count changes as a result of execution of the statement.

Frequency – number of times a statement is executed.

Example:1

Statement s/e Frequency Total steps

Algorithm sum( list[ ], n) 0 - 0
{ 0 - 0
tempsum := 0; 1 1 1
for i := 0 ton do 1 n+1 n+1
tempsum := tempsum + list [i]; 1 n n
return tempsum; 1 1 1
} 0 0 0

Total 2n+3
 Example: 2
Statement s/e Frequency Total steps
Algorithm sum( x, n) 0 - 0
{ 0 - 0
total := 0; 1 1 1
for i := 1 to n do 1 n+1 n+1
{ 0 - 0
total:= total + i; 1 n n
} 0 - 0

Total 2n+2

Example: 3
Statement s/e Frequency Total steps
Algorithm add(a,b,c,n) 0 - 0
{ 0 - 0
for i:=1 to n do 1 n+1 n+1
for j:=1 to n do 1 n(n+1) n2+n
c[i,j]:=a[i,j]+b[i,j]; 1 n*n n2
} 0 - 0

Total 2n2+2n+1

Example:4
Statement s/e Frequency Total steps
Algorithm add(a,b,c,m,n) 0 - 0
{ 0 - 0
for i:=1 to m do 1 m+1 m+1
for j:=1 to n do 1 m(n+1) mn+m
c[i,j]:=a[i,j]+b[i,j]; 1 mn mn
} 0 - 0

Total 2mn+2m+1

Complexity of Algorithms:

The complexity of an algorithm M is the function f(n) which gives the running time and/or
storage space requirement of the algorithm in terms of the size ‘n’ of the input data. Mostly,
the storage space required by an algorithm is simply a multiple of the data size ‘n’.
Complexity shall refer to the running time of the algorithm.
The function f(n), gives the running time of an algorithm, depends not only on the size ‘n’
of the input data but also on the data. The complexity function f(n) for certain cases is:

1. Best Case : The minimum possible value of f(n) is called the best-case.

2. Average Case : The average value of f(n).

3. Worst Case : The maximum value of f(n) for any key possible input.
The field of computer science, which studies efficiency of algorithms, is known as analysis
of algorithms.
Algorithms can be evaluated by a variety of criteria. Most often we shall be interested in
the rate of growth of the time or space required to solve larger and larger instances of a
problem. We will associate with the problem an integer, called the size of the problem,
which is a measure of the quantity of input data, Rate of Growth.

The following notations are commonly use notations in performance analysis and used to
characterize the complexity of an algorithm:

Asymptotic notation:
Asymptotic notations are the mathematical tools that describe the behaviour of the algorithm and
determine the rate of growth of a function.
Rate of Growth: It is the rate at which the running time of algorithm grows as size of the input grows.

Big O Notation: Big O notation is used to measure the performance of any algorithm by
providing the order of growth of the function.
It gives the upper bound on a function by which we can make sure that the function will never
grow faster than this upper bound.
 If f(n) and g(n) are the two functions , then
f(n)=O(g(n))
if there exists constants c and n0 such that f(n)<=c*g(n) for all n>= n0

[This means f(n) doesn’t grow faster than g(n)]

Ex: 1
f(n)=n g(n)=2n
Is f(n) =O(g(n))?
Ans: f(n)<=c*g(n)
n<=c*2n for c=1 and n0=1
Yes it is f(n)=O(g(n)) for all n>=1. f(n) grows linearly.

Ex: 2
f(n)=4n+3 g(n)=n
Is f(n) =O(g(n))?
Ans: f(n)<=c*g(n)
4n+3<=c*n we must find suitable c value which satisfies equation.
For c=5, 4n+3<=5n, Now we must find suitable n0 value.
3<=5n-4n
n>=3
Yes, it is f(n)=O(g(n)) for all n>=3. f(n)grows linearly.

Ex: 3
f(n)=3n+2 g(n)=n
Is f(n) =O(g(n))?
Ans: f(n)<=c*g(n)
3n+2<=c*n we must find suitable c value which satisfies equation.
For c=4, 3n+2<=4n, Now we must find suitable n0 value.
2<=4n-3n
n>=2
Yes, it is f(n)=O(g(n)) for all n>=2. f(n)grows linearly.
 Big Ω(Omega) Notation: Big Ω notation is used to measure the performance
of any algorithm by providing the order of growth of the function.
It gives the lower bound on a function by which we can make sure that the
function will never grow slower than this lower bound.

If f(n) and g(n) are the two functions , then

f(n)=Ω(g(n))
if there exists constants c and n0 such that f(n)>=c*g(n) for all n>= n0

[This means f(n) grows at same rate or faster than g(n)]

Ex: 1
f(n)=3n+2 g(n)=n
Is f(n) =Ω(g(n))?
Ans: f(n)>=c*g(n)
3n+2>=c*n
2>=cn-3n
2>=n(c-3)
n<=2/(c-3)
for c=4, n0=2
Yes, it is f(n)=Ω(g(n))

Ex: 2
f(n)=5n2 g(n)=n
Is f(n) =Ω(g(n))?
Ans: f(n)>=c*g(n)
5n2 >=c*n
5n>=c
for n0=1 c=5
Yes, it is f(n)=Ω(g(n))
 Ex: 3
f(n)=5n2+2n-3 g(n)=n2
Is f(n) =Ω(g(n))?
Ans: f(n)>=c*g(n)
5n2+2n-3 >=c*n2
Here 2n-3 must be positive.
2n-3>=0
2n>=3
n>=3/2=2
for n0=2, c=5
Yes, it is f(n)=Ω(g(n))

Big θ (Theta) notation:

Big θ notation is used to measure the performance of any algorithm by
providing the order of growth of the function.
It gives the both lower bound and upper bound on a function.

[This means f(n) and g(n) grow at same rate]

If f(n) and g(n) are the two functions , then

f(n)= θ (g(n)) iff there exist positive constants
c1, c2 and n0 such that
C1*g(n) ≤f(n)≤C2*g(n) for all n, n≥0
Ex: 1
f(n)=3n+2 g(n)=n
Is f(n) = θ (g(n))?
Ans: c1*n<=3n+2<=c2*n
This can be solved in two parts

c1*n<=3n+2 and 3n+2<=c2*n

i) c1*n<=3n+2
3n+2>=c*n
2>=cn-3n
 2>=n(c-3)
n<=2/(c-3)
for c1=4, n0=2
ii) 3n+2<=c2*n
For c2=4, 3n+2<=4n, Now we must find suitable n0 value.
2<=4n-3n
n>=2
Yes, it is f(n)= θ (g(n)) for c1=4,c2=4,n>=2

Little oh: o
The function f(n)=o(g(n)) (read as “f of n is little oh of g of n”) iff

0<= f(n)<c.g(n)
for all c>0 and n>=n0
OR

Lim f(n)/g(n)=0 for all n, n≥0

n∞

Ex: 1
Is 3n+2=o(n2) ?
Ans: 3n+2< n2 for c=1 and n>=4
c=2 and n>=3
c=3 and n>=2 …..etc

Therefore 3n+2=o(n2)

Ex:2 Is 2n=o(n) ?

Ans: 2n< n for c=1 and n=1 condition fails

Therefore 2n≠o(n)

Ex:3 Is 2n=o(n2) ?

Ans; for c=1 and n>=3

c=2 and n>=2
c=3 and n>=1 …..etc

Therefore 2n=o(n2)
 Ex:4 Is 2 n2=o(n2) ?

Ans; for c=1 and n=1 condition fails

Therefore 2 n2≠o(n2)

Ex:5 Is 2 n2=o(n3) ?

Ans; for c=1 and n >=3

for c=2 and n >=2
for c=3 and n >=1

Therefore 2 n2=o(n3)

Little Omega:ω
The function f(n)=ω (g(n)) (read as “f of n is little omega of g of n”) iff

0<= c.g(n) < f(n)

For all c>0 and n>=n0

OR

Lim f(n)/g(n)= ∞ for all n, n≥0

n∞

Ex: 1
Is n2 = ω (n) ?
Ans: c * n < n2 for c=1 and n>=2
c=2 and n>=3
c=3 and n>=4 …..etc

Therefore n2 = ω (n)

Ex: 2
Is n2 = ω (n) ?
Ans: c * n < n2 for c=1 and n>=2
c=2 and n>=3
c=3 and n>=4 …..etc

Therefore n2 = ω (n)
Ex: 3
Is n2/2 = ω (n) ?
Ans: c * n < n2 /2 for c=1 and n>=2
c=2 and n>=5
c=3 and n>=7 …..etc

Therefore n2/2 = ω (n)

Ex: 4
Is n2/2 = ω (n2) ?
Ans: c * n2 < n2 /2 for c=1 and n=1 condition fails

Therefore n2/2 ≠ ω (n2)

Analyzing Algorithms
Suppose ‘M’ is an algorithm, and suppose ‘n’ is the size of the input data. Clearly
the complexity f(n) of M increases as n increases. It is usually the rate of increase
of f(n) we want to examine. This is usually done by comparing f(n) with some
standard functions. The most common computing times are:
O(1), O(log2n), O(n), O(n. log2n), O(n2), O(n3), O(2n), n! and nn

Best, Worst and Average case complexity:

Best case is the minimum number of steps that can be achieved for given parameters.

Ex: Consider five elements of any data structure in the order 7,17,27,37,47
In this our algorithm task is finding particular element present at which position.

In Best case efficiency, finding the element 7 requires one comparison.

CBest(n)=1

Worst case is the maximum number of steps that can be achieved for given parameters. And
also in case of failed, it checks the maximum number of steps.

CWorst(n)=n

CFailed(n)=n

Average case is the situation in which it is not either best case or worst case.
Element is found somewhere in the middle of the list.
DIVIDE AND CONQUER
General method:
Given a function to compute on ‘n’ inputs the divide-and-conquer strategy suggests
splittingthe inputs into ‘k’ distinct subsets, 1<k<=n, yielding ‘k’ sub problems.

These sub problems must be solved, and then a method must be found to
combine subsolutions into a solution of the whole.

If the sub problems are still relatively large, then the divide-and-conquer strategy can
possibly be reapplied. Often the sub problems resulting from a divide-and-conquer
design are of the same type as the original problem.
For those cases the re application of the divide- and-conquer principle is naturally
expressed by a recursive algorithm. DAndC(Algorithm) is initially invoked as
DandC(P), where ‘P’ is the problem to be solved. Small(P) is a Boolean- valued
function that determines whether the input size is small enough that the answer can
be computed without splitting. If this so, the function ‘S’ is invoked. Otherwise, the
problem P is divided into smaller sub problems. These sub problems P1, P2 …Pk
are solved by recursive application of DAndC. Combine is a function that determines
the solution to P using the solutions to the ‘k’ sub problems. If the size of ‘P’ is n and
the sizes of the ‘k’ sub problems are n1, n2 ….nk, respectively, then the computing
time of DAndC is described by the recurrence relation.
T(n)= { g(n) n small
T(n1)+T(n2)+……………+T(nk)+f(n); otherwise.
Where T(n) is the time for DAndC on any input of size ‘n’.
g(n) is the time of compute the answer directly for small inputs. f(n) is the time for dividing P
& combining the solution to sub problems.

Algorithm DAndC(P)
{
if small(P) then return S(P);
else
{
divide P into smaller instances
P1, P2… Pk, k>=1;
Apply DAndC to each of these sub problems;
return combine (DAndC(P1), DAndC(P2),…….,DAndC(Pk));
}
}
The complexity of many divide-and-conquer algorithms is given by recurrence relationof the
form
T(n) = T(1) n=1
=aT(n/b)+f(n) n>1
Where a & b are known constants.
We assume that T(1) is known & ‘n’ is a power of b (i.e., n=bk)
One of the methods for solving any such recurrence relation is called the substitution
method. This method repeatedly makes substitution for each occurrence of the
function. T is the right-hand side until all such occurrences disappear.

Example: 1

Consider the case in which a=2 and b=2. Let T(1)=1 & f(n)=n. We have,
T(n) = 2T(n/2)+n
= 2[2T(n/2/2)+n/2]+n
=[4T(n/4)+n]+n
= 4T(n/4)+2n
= 4[2T(n/4/2)+n/4]+2n
= 4[2T(n/8)+n/4]+2n
= 8T(n/8)+n+2n
= 8T(n/8)+3n

*
*

In general, we see that T(n)=2k T(n/2k)+k n for any log2 n >=k>=1

T(n/2k) = T(1)
n/2k = 1
n =2k
k = log n
Therefore T(n)=2k T(1)+k n
= n *1 + n log n
O( n log n)

OR

T(n) = 2T(n/2)+n
= 2[2T(n/2/2)+n/2]+n
=[4T(n/4)+n]+n
= 4T(n/4)+2n
= 4[2T(n/4/2)+n/4]+2n
= 4[2T(n/8)+n/4]+2n
= 8T(n/8)+n+2n
= 8T(n/8)+3n
*
*
=n T(n/n)+k n
T(n) = n T(1)+k n
T(n) = logn n
Therefore T(n) = n logn

O( n log n)
Example: 2

Consider the case in which a=1 and b=2. Let T(1)=1 & f(n)=c We have,
T(n) = T(n/2)+c
= [T(n/2/2)]+c+c
=[T(n/4)]+c+c
=[T(n/4/2)+c+c+c
=[T(n/8)]+3c

*
*
= T(n/n)+k c
= T(1)+k c
T(n) = log n * c

O( log n)

BINARY SEARCH

Given a list of n elements arranged in increasing order. The problem is to determine

whether a given element is present in the list or not. If x is present then determine the
position of x, otherwise position is zero.

Divide and conquer is used to solve the problem.

ITERATIVE METHOD:

Example: Search the key values 42, 55 and 30 from the list
3, 6,8,12,14,17,25,29,31,36,42,47,53,55,62
Ans:

Key Value = 42

3 6 8 12 14 17 25 29 31 36 42 47 53 55 62
Index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Here l=1 h=15 mid= (l + h)/2 =8

a[8]=29 ≠ 42 -----1st comparison

The element we are searching is right side of the mid value
Therefore l=mid+1 and h =h and mid =(9+15)/2=12

a[12]=47 ≠ 42 -----2nd comparison

The element we are searching is left side of the mid value

Therefore l=l and h =mid-1 and mid =(9+11)/2=10

a[10]=36 ≠ 42 -----3rd comparison

The element we are searching is right side of the mid value

Therefore l=mid+1 and h =h and mid =(11+11)/2=11

a[11]=42 = 42 -----4th comparison

Key value is found at the 4th comparison.

Search is Successful

Key Value=55

3 6 8 12 14 17 25 29 31 36 42 47 53 55 62
Index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Here l=1 h=15 mid= (l + h)/2 =8

a[8]=29 ≠ 55 -----1st comparison

The element we are searching is right side of the mid value

Therefore l=mid+1 and h =h and mid =(9+15)/2=12

a[12]=47 ≠ 55 -----2nd comparison

The element we are searching is right side of the mid value

Therefore l=mid+1 and h =h and mid =(13+15)/2=14

a[14]=55 = 55 -----3rd comparison

Key value is found at the 3rd comparison.

Search is Successful

Key Value = 30

3 6 8 12 14 17 25 29 31 36 42 47 53 55 62
Index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Here l=1 h=15 mid= (l + h)/2 =8

a[8]=29 ≠ 30 -----1st comparison

 The element we are searching is right side of the mid value
Therefore l=mid+1 and h =h and mid =(9+15)/2=12

a[12]=47 ≠ 30 -----2nd comparison

The element we are searching is left side of the mid value

Therefore l=l and h =mid-1 and mid =(9+11)/2=10

a[10]=36 ≠ 30 -----3rd comparison

The element we are searching is left side of the mid value

Therefore l=l and h =mid-1 and mid =(9+10)/2=9

a[9]=31 ≠ 30 -----4th comparison

The element we are searching is left side of the mid value

Therefore l=l and h =mid-1 and l value is 9, h value is 8 i.e., l>h which is not
possible

Key value is not found after the 4th comparison.

Search is Unsuccessful

Mid value tree can be drawn as follows:

1 3 5 7 9 11 13 15

Algorithm: Binary Search (Iterative Method)

Algorithm BinaySearch(A,n,key)
{
low=l; high=n;
while(low<=high)
{
mid=(low+high)/2;
if (key= =A[mid])
return mid;
if (key < A[mid])
high = mid-1;
else
low = mid +1;
}
return 0;
}

Algorithm: Binary Search (Recursive Method)

Algorithm RBinaySearch(low,high,key)
{
if (low = = high)
{
if (A[low]= = key)
return low;
else
return 0;
}
else
{
mid=(low+high)/2;
if (key = = A[mid])
return mid;
if (key < A[mid])
return RBinarySearch(low,mid-1,key);
else
return RBinarySearch(mid+1,high,key);
}
}

Complexity of Binary Search:

For the list of elements of size of n, based on the mid value, each time we are considering n/2
part of the list. Therefore, the complexity of Binary search is T(n)=T(n/2)+c which is in the
form of T(n)=a T(n/b)+f(n) where a=1, b=2 and f(n)=c

Hence O(log n)
QUICK SORT

The quick sort algorithm partitions the original array by rearranging it into two groups. The
first group contains those elements less than some arbitrary chosen value taken from the set,
and the second group contains those elements greater than or equal to the chosen value.
The chosen value is known as the pivot element. Once the array has been rearranged in this
way with respect to the pivot, the very same partitioning is recursively applied to each of the
two subsets. When all the subsets have been partitioned and rearranged, the original array is
sorted.
Procedure:
Select first element as the pivot element. Move ‘i’ pointer from left to right in search of an
element larger than pivot. (As long as i<pivot, increment i) and move the ‘j’ pointer from
right to left in search of an element smaller than pivot (As long as j>p, decrement j). If such
elements are found, the elements are swapped. This process continues till the ‘i’ pointer
crosses the ‘j’ pointer. If ‘i’ pointer crosses ‘j’ pointer, the position for pivot is found and
interchange pivot and element at ‘j’ position.
Let us consider the following example with 7 elements to analyze quicksort

50 15 25 49 5 10 16
1 2 3 4 5 6 7 Remarks
50 15 25 49 5 10 16
Pivot i J Initial set up
Pivot J We do not find i which is greater than pivot
element. So swap j and pivot element
16 15 25 49 5 10 50 50 is in the correct position
Pivot i j Again find i>pivot and j<pivot
i j Swap i and j
16 15 10 49 5 25 50
Pivot i j Again find i>pivot and j<pivot
16 15 10 49 5 25 50
Pivot i j Swap i and j
16 15 10 5 49 25 50
Pivot i j Again find i>pivot and j<pivot
16 15 10 5 49 25 50
Pivot j i i and j are overlapped. So swap j and pivot
element
5 15 10 16 49 25 50
Pivot i j 16 is in the correct position.
We do not find element j less than pivot
5 15 10 16 49 25 50 5 is in the correct position
Pivot i,j j is less than pivot, swap j and pivot
5 10 15 16 49 25 50 10 and 15 are in the correct position
pivot i,j j is less than pivot, swap j and pivot

5 10 15 16 25 49 50 List is sorted
Quicksort Algorithm:
Quicksort(A,i,j)
{
if ( i < j)
{
Ploc = Partition(A,i,j);
Quicksort(A,i, Ploc – 1);
Quicksort(A,Ploc + 1,j);
}
}
Partition Algorithm
Partition(A,i,j)
{
Pivot = A[i];
i=1;
j=n;
while(i<n)
{
while(A[i]<=pivot)
{
i++;
}
while(A[j]>pivot)
{
j--;
}
if(i<j)
{
Swap(A[i],A[j]);
}
else
Swap(A[pivot],A[j]);
return j;
}
Complexity of Quick Sort:
Worst Case:
x
Pivot --------------------------------------- n-1---------------------------------------------------
Let us suppose that x is the smallest element then no elements less than x
Again, let us suppose that list is already in sorted order,
x Y
Pivot --------------------------------------- n-2----------------------------------------------
 x Y z
Pivot ----------------------------------n-3----------------------------------------------
So Time complexity is
T(n)=T(0)+T(n-1)+n
=T(n-1)+ n
=T(n-1-1)+(n-1)
=T(n-2)+(n-1) + n
=T(n-2-1)+ (n-2) + n-1 + n
=T(n-3)+(n-2) + n-1 +n
*
*
*
= T(n-k) + (n-(k-1)) + …..+ (n-2)+(n-1)+n

Let us assume that n-k =0 i.e n=k

=T(0) + 1+2+……..+(n-1)+n
= 1 + n(n+1)/2
= (n2+n)/2
Hence Complexity is O(n2)
Best Case:
x
---------n/2------------------ Pivot -------------n/2--------------

T(n) = T(n/2) + T(n/2) +c

= 2 T(n/2) + c
This is in the form of T(n)= a T(n/b)+f(n) where a=2, b=2 and f(n)=c

Hence Complexity is O( log n )

MERGE SORT
Merge sort algorithm is a classic example of divide and conquer. To sort an array, recursively,
sort its left and right halves separately and then merge them.
This strategy is so simple, and so efficient but the problem here is that there seems to be no
easy way to merge two adjacent sorted arrays together in place (The result must be built up in
a separate array). The fundamental operation in this algorithm is merging two sorted lists.
Because the lists are sorted, this can be done in one pass through the input, if the output is put
in a third list.
Example: 415 213 700 515 712 715

415 213 700 515 712 715

1 2 3 4 5 6
Mid=(low+high)/2=(1+6)/2=3

415 213 700 515 712 715

1 2 3 1 2 3
Mid=(low+high)/2=(1+3)/2=2

415 213 700 515 712 715

1 2 1 2
id=(low+high)/2=(1+2)/2=1

515 712
415 213

Next step is to compare and sort the elements

213 415
515 712

515 712 715

213 415 700 j high
i mid

Compare i and j, as long as i<j add i to output list or add j to output list. If i>mid then add all
remaining j values to output list. Similarly, if j>high then add all remaining i values to output
list.
Therefore, final sorted list is 415 213 700 515 712 715

Merge sort Algorithm:

Merge() function
Merge(A,low,mid,high)
{
i=low, j=mid+1;
while(i<=mid && j<=high)
{
if (A[i] <=A[j])
{
B[k]=A[i];
i++; k++;
}
else
{
B[k]=A[j];
j++; k++;
}
}
if(i>mid)
{
while(j<=high)
{
B[k]=A[j];
j++;
k++;
}
}
else if(j>high)
{
while(i<=mid)
{
B[k]=A[i];
i++;
k++;
}
}
Mergesort(A,low,high)
{
if(low<=high)
{
mid=(low+high)/2;
Mergesort(A,low,mid);
Mergesort(A,mid+1,high);
Merge(A,low,mid),high);
}
}
Complexity of Merge Sort:

T(n) = T(n/2) + T(n/2) +n

T(n) = 2T(n/2)+n
= 2[2T(n/2/2)+n/2]+n
=[4T(n/4)+n]+n
= 4T(n/4)+2n
= 4[2T(n/4/2)+n/4]+2n
= 4[2T(n/8)+n/4]+2n
= 8T(n/8)+n+2n
= 8T(n/8)+3n

*
*

In general, we see that T(n)=2k T(n/2k)+k n for any log2 n >=k>=1

T(n/2k) = T(1)
n/2k = 1
n =2k
k = log n
Therefore T(n)=2k T(1)+k n
= n *1 + n log n
O( n log n)

Strassen’s Matrix Multiplication:

The matrix multiplication of algorithm due to Strassen is the most dramatic example of divide
and conquer technique.
Let A and B be two n×n Matrices. The product matrix C=AB is also a n×n matrix whose i,
jth element is formed by taking elements in the ith row of A and jth column of B and
multiplying them to get
The usual way C(i, j)=∑1≤𝑘≤𝑛 𝐴(i, 𝑘)𝐵(𝑘, j) --------- Eqn 1
Here 1≤ i & j ≤ n means i and j are in between 1 and n.
To compute C(i, j) using this formula, we need n multiplications.
The divide and conquer strategy suggest another way to compute the product of two n×n
matrices. For Simplicity assume n is a power of 2 that is n=2k, k is a nonnegative integer.
If n is not power of two then enough rows and columns of zeros can be added to both A and
B, so that resulting dimensions are a power of two.
To multiply two n x n matrices A and B, yielding result matrix ‘C’ as follows:
Let A and B be two n×n Matrices. Imagine that A & B are each partitioned into four square
sub matrices. Each sub matrix having dimensions n/2×n/2.
The product of AB can be computed by using Eqn 1 formula.
If AB is product of 2×2 matrices then

A11 A12 B11 B12 C11 C12

A21 A22 B21 B22 = C21 C22
Then cij can be found by the usual matrix multiplication algorithm,
C11 = A11 B11 + A12 B21

C12 = A11 B12 + A12 B22

C21 = A21 B11 + A22 B21

C22 = A21 B12 + A22 B22

This leads to a divide–and–conquer algorithm, which performs nxn matrix multiplication by

partitioning the matrices into quarters and performing eight (n/2)x(n/2) matrix multiplications
and four (n/2)x(n/2) matrix additions.
T(1) = 1
T(n) = 8T(n/2)

Which leads to T (n) = O (n3), where n is the power of 2.

Strassen’s insight was to find an alternative method for calculating the C ij, requiring seven
(n/2) x (n/2) matrix multiplications and eighteen (n/2) x (n/2) matrix additions and
subtractions:
P = (A11 + A22) (B11 + B22)
Q = B11(A21 + A22)
R = A11 (B12 -B22)
S = A22 (B21 - B11)
T = B22 (A11 + A12)
U = (B11 + B12) (A21 – A11)
V = (B21 + B22) (A12 – A22)
C11 = P + S – T +V
C12 = R+ T
C21 = Q+S
C22 = P + R – Q +U
This method is used recursively to perform the seven (n/2) x (n/2) matrix multiplications,
then the recurrence equation for the number of scalar multiplications performed is:
 T(1) = 1
T(n) = 7T(n/2)

Solving this for the case of n = 2k is easy:

T(2k) = 7T(2k–1)

= 72T(2k-2)

= ------------

= ------------
= 7iT(2k–i)
Put i =k
= 7kT(20)
As k is the power of 2
n
That is T(n) = 7log2
= log
n 2
= O(nlog 7)= O(n2.81)
So, concluding that Strassen’s algorithm is asymptotically more efficient than the standard
algorithm.
Strassen’s Matrix Multiplication Algorithm:

Algorithm MatricMultiplication(A,B,n)
{
if(n<=2)
{
c11= a11 b11 + a12 b21;
c12= a11 b12 + a12 b22;
c21= a21 b11 + a22 b21;
c22= a21 b12 + a22 b22;

else
{
mid=n/2;
MM(A11,B11,n/2)+MM(A12,B21,n/2);
MM(A11,B12,n/2)+MM(A12,B22,n/2);
MM(A21,B11,n/2)+MM(A22,B21,n/2);
MM(A21,B12,n/2)+MM(A22,B22,n/2);
}
}

Example: Apply Strassen’s matrix multiplication algorithm to compute AxB

A= 1 3 B= 6 8

7 5 4 2

a11=1 a12=3 a21=7 a22=5 b11=6 b12=8 b21=4 b22=2

Ans: Let C = C11 C12

C21 C22

P = (1 + 5) (6 + 2) = 48
Q = 6(7 + 5) = 72
R = 1 (8 -2) = 6
S = 5(4 - 6) = -10
T = 2(1 + 3) = 8
U = (6 + 8) (7 – 1) = 84
V = (4 + 2) (3 – 5) = -12
C11 = 48 + (-10) – 8 +(-12) = 18
C12 = 6+ 8 =14
C21 = 72 + (-10) = 62
C22 = 48 + 6 – 72 +84 = 66
Therefore C =
18 14
62 66
66
 UNIT-I IMP QUESTIONS
1) Show that the following equalities are correct:
a) 5n2 - 6n = θ (n2)
b) n! = O(nn)
c) 2n22n + n log n = θ (n22n)
d) n3 + 106n2 = θ (n3) (4+4+4+3) M
2) What are asymptotic notations? Explain with examples. Write and explain quick sort
algorithm with an example. (8+7) M
3) a) Write and explain randomized quick sort algorithm.
b) Discuss about Big oh O, Omega Ω and Theta θ notations with examples. (8+7) M
4) a) Using Merge sort, sort the following elements:
310, 285, 179, 652, 351, 423, 861, 254, 450, 520
b) Analyze the computing time complexity of binary search algorithm. (7+8) M
5) a) Discuss in detail various notations used for expressing the time complexity
of algorithms, with examples.
b) What is Performance Analysis? Explain Space Complexity and Time Complexity with
examples. (7+8) M
6) a) Explain the process of merge sort by a list of any 11 integers (distributed
randomly). Write the algorithm and analyze its time complexity.
b) Write an algorithm to find matrix multiplication using Strassen’s. (8+7) M
7) What is an algorithm? 2M
8) Define static space tree. 2M
9) a) Write and explain the general method of divide-and-conquer strategy.
b) Derive the time complexity of Strassen’s matrix multiplication. (5+5) M
10) a) Write and explain recursive algorithm of binary search method.
b) What is space complexity? Explain with suitable examples. (5+5) M